Transfer Optimisation System for Adaptive Automated Nature-Inspired Optimisation
Lead Research Organisation:
UNIVERSITY OF EXETER
Department Name: Computer Science
Abstract
Hard optimisation problems are ubiquitous across the breadth of science, engineering and economics. For example, in water system planning and management, water companies are often interested in optimising several system performance measures of their infrastructures. They are particularly interested in providing sustainable and resilient water/wastewater services that are able to cope with and recover from disruption, as well as wider challenges brought by climate change and population increase. As a classic discipline, significant advances in both theory and algorithms have been achieved in optimisation. However, almost all traditional optimisation solvers, ranging from classic methods to nature-inspired computational intelligence techniques, ignore some important facts: (i) real-world optimisation problems seldom exist in isolation; and (ii) artificial systems are designed to tackle a large number of problems over their lifetime, many of which are repetitive or inherently related. Instead, optimisation is run as a 'one-off' process, i.e. it is started from scratch by assuming zero prior knowledge each time. Therefore, knowledge/experience from solving different (but possibly related) optimisation exercises (either previously completed or currently underway), which can be useful for enhancing the target optimisation task at hand, will be wasted. Although the Bayesian optimisation considers incorporating some decision maker's knowledge as a prior, the gathered experience during the optimisation process is discarded afterwards. In this case, we cannot expect any automatic growth of their capability with experience. This practice is counter-intuitive from the cognitive perspective where humans routinely grow from a novice to domain experts by gradually accumulating problem-solving experience and making use of existing knowledge to tackle new unseen tasks. In machine learning, leveraging knowledge gained from related source tasks to improve the learning of the new task is known as transfer learning, an emerging field that considerable success has been witnessed in a wide range of application domains. There have been some attempts on applying transfer learning in evolutionary computation, but they do not consider the optimisation as a closed-loop system. Moreover, the recurrent patterns within problem-solving exercises have been discarded after optimisation, thus experience cannot be accumulated over time.
The proposed research will develop a revolutionary general-purpose optimiser (as known as a transfer optimisation system) that will be able to learn knowledge/experience from previous optimisation process and then continuously, autonomously, and selectively transfer such knowledge to new unseen optimisation tasks in open-ended dynamic environments. The transfer optimisation system places adaptive automation at the heart of the development process and explores novel synergies at the crossroads of several disciplines including nature-inspired computation, machine learning, human-computer interaction and high-performance parallel computing. The outputs will bring automation in industry, including an optimised/shortened production cycle, reduced resource consumption and more balanced and innovative products, which have great potentials to result in economic savings and an increase of turnover. The proposed methods will be rigorously evaluated by the industrial partners, first in water industry and will be expanded to a boarder range of sectors which put the optimisation at the heart of their regular production/management process (e.g. software engineering, renewable energy, healthcare, automotive, appliance and medicine manufacturers).
The proposed research will develop a revolutionary general-purpose optimiser (as known as a transfer optimisation system) that will be able to learn knowledge/experience from previous optimisation process and then continuously, autonomously, and selectively transfer such knowledge to new unseen optimisation tasks in open-ended dynamic environments. The transfer optimisation system places adaptive automation at the heart of the development process and explores novel synergies at the crossroads of several disciplines including nature-inspired computation, machine learning, human-computer interaction and high-performance parallel computing. The outputs will bring automation in industry, including an optimised/shortened production cycle, reduced resource consumption and more balanced and innovative products, which have great potentials to result in economic savings and an increase of turnover. The proposed methods will be rigorously evaluated by the industrial partners, first in water industry and will be expanded to a boarder range of sectors which put the optimisation at the heart of their regular production/management process (e.g. software engineering, renewable energy, healthcare, automotive, appliance and medicine manufacturers).
Publications
Chen J
(2024)
A Knee Point Driven Evolutionary Algorithm for Multiobjective Bilevel Optimization.
in IEEE transactions on cybernetics
Chen L
(2024)
Evolutionary Bilevel Optimization via Multiobjective Transformation-Based Lower-Level Search
in IEEE Transactions on Evolutionary Computation
Guan Y
(2023)
Multidimensional Resource Fragmentation-Aware Virtual Network Embedding for IoT Applications in MEC Networks
in IEEE Internet of Things Journal
Heng Yang
(2023)
Boosting Text Augmentation via Hybrid Instance Filtering Framework
Heng Yang
(2024)
Modeling Aspect Sentiment Coherency via Local Sentiment Aggregation
Li K
(2024)
A Survey of Multi-objective Evolutionary Algorithm Based on Decomposition: Past and Future
in IEEE Transactions on Evolutionary Computation
Li K
(2023)
Interactive Evolutionary Multiobjective Optimization via Learning to Rank
in IEEE Transactions on Evolutionary Computation
Li K
(2024)
A Data-Driven Evolutionary Transfer Optimization for Expensive Problems in Dynamic Environments
in IEEE Transactions on Evolutionary Computation
Li K
(2023)
DaNuoYi: Evolutionary Multi-Task Injection Testing on Web Application Firewalls
in IEEE Transactions on Software Engineering
Li K
(2023)
Batched Data-Driven Evolutionary Multiobjective Optimization Based on Manifold Interpolation
in IEEE Transactions on Evolutionary Computation
Li S
(2024)
Evolutionary Alternating Direction Method of Multipliers for Constrained Multi-Objective Optimization With Unknown Constraints
in IEEE Transactions on Evolutionary Computation
Lyu B
(2023)
MTLP-JR: Multi-task learning-based prediction for joint ranking in neural architecture search
in Computers and Electrical Engineering
Lyu B
(2023)
Efficient Spectral Graph Convolutional Network Deployment on Memristive Crossbars
in IEEE Transactions on Emerging Topics in Computational Intelligence
Phoenix Williams
(2024)
CamoPatch: An Evolutionary Strategy for Generating Camoflauged Adversarial Patches
Tanabe R
(2024)
Quality Indicators for Preference-Based Evolutionary Multiobjective Optimization Using a Reference Point: A Review and Analysis
in IEEE Transactions on Evolutionary Computation
Wang S
(2024)
Constrained Bayesian Optimization under Partial Observations: Balanced Improvements and Provable Convergence
in Proceedings of the AAAI Conference on Artificial Intelligence
Williams P
(2023)
Sparse Adversarial Attack via Bi-objective Optimization
Williams P
(2024)
Evolutionary Art Attack For Black-Box Adversarial Example Generation
in IEEE Transactions on Evolutionary Computation
Xu J
(2024)
An Automated Few-Shot Learning for Time-Series Forecasting in Smart Grid Under Data Scarcity
in IEEE Transactions on Artificial Intelligence
Xu J
(2024)
Multioutput Framework for Time-Series Forecasting in Smart Grid Meets Data Scarcity
in IEEE Transactions on Industrial Informatics
Yang L
(2024)
A many-objective evolutionary algorithm based on interaction force and hybrid optimization mechanism
in Swarm and Evolutionary Computation
Yu H
(2024)
An interpretable RNA foundation model for exploring functional RNA motifs in plants.
in Nature machine intelligence
Zhang H
(2024)
Solving Expensive Optimization Problems in Dynamic Environments With Meta-Learning.
in IEEE transactions on cybernetics
Zheng J.-H.
(2011)
Objective-reduction using the least squares method
in Kongzhi Lilun Yu Yingyong/Control Theory and Applications
Zheng S
(2024)
Mutual Knowledge Distillation based Personalized Federated Learning for Smart Edge Computing
in IEEE Transactions on Consumer Electronics
Zhou S
(2024)
Evolutionary Multi-objective Optimization for Contextual Adversarial Example Generation
in Proceedings of the ACM on Software Engineering
Zou J
(2024)
A Multipopulation Evolutionary Algorithm Using New Cooperative Mechanism for Solving Multiobjective Problems With Multiconstraint
in IEEE Transactions on Evolutionary Computation
| Description | We have developed two software packages. One is a graph-based fitness landscape analysis that can be used to demystify the black-box optimization problem and systems. The other is an automated multi-task security testing tool that is able to automatically generate test inputs for multiple types of injection attacks on Web application firewalls. |
| Exploitation Route | The software packages developed in this project can be beneficial not only for academia but also can be used as handy tools for optimization and data analysis. |
| Sectors | Digital/Communication/Information Technologies (including Software) |
| Description | One of our research outcome is a collaboration with Amazon AWS. We developed a new transfer learning driven test case generation tool to help identify vulnerabilities in the Web firewalls. This software product is going to be patented in 2025. |
| First Year Of Impact | 2024 |
| Sector | Digital/Communication/Information Technologies (including Software) |
| Impact Types | Economic |
| Description | ERC Consolidator Grant (UKRI Horizon Europe Guarantee) |
| Amount | £1,644,752 (GBP) |
| Funding ID | EP/Y009886/1 |
| Organisation | European Research Council (ERC) |
| Sector | Public |
| Country | Belgium |
| Start | 03/2023 |
| End | 03/2024 |
| Description | Kan Tong Po International Fellowship 2023 |
| Amount | £3,000 (GBP) |
| Funding ID | KTP\R1\231017 |
| Organisation | The Royal Society |
| Sector | Charity/Non Profit |
| Country | United Kingdom |
| Start | 11/2023 |
| End | 10/2024 |
| Description | Turing Fellowship |
| Amount | £0 (GBP) |
| Organisation | Alan Turing Institute |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 03/2024 |
| End | 03/2026 |
| Description | Collaboration with GE Healthcare |
| Organisation | GE Healthcare Limited |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | We started this collaboration with GE Healthcare through a co-supervision of a PhD student funded by the University of Exeter. We are working on an exciting topic about explainable AI for healthcare. |
| Collaborator Contribution | The partner group in GE Healthcare contributed the staff time for attending the discussion meetings. |
| Impact | This is a multi-disciplinary collaboration. There is no concrete outcome yet since we just started this in a couple months. |
| Start Year | 2023 |
| Description | Research collaboration with Prof. Kay Chen Tan in The Hong Kong Polytechnic University |
| Organisation | Hong Kong Polytechnic University |
| Country | Hong Kong |
| Sector | Academic/University |
| PI Contribution | This is a research collaboration supported by the Kan Tong Po International Fellowship from the Royal Society. During this fellowship, we aim to revolutionize the complex software configuration tuning process. Imagine if we can incorporate human knowledge and preferences into automatic software configuration, creating a collaboration between humans and machines. This synergy could yield configurations tailored to individual requirements, reducing errors, and improving performance. |
| Collaborator Contribution | The partner in the Hong Kong Polytechnic University provided the office space to host my visit in Hong Kong. Prof. Tan also organized two workshops in the University to create an active environment for collaboration and discussion. |
| Impact | We have collaborated in some paper manuscript underway for submission. In addition, we plan to have a collaborative grant proposal later in 2024. The target is Hong Kong GRF scheme. |
| Start Year | 2023 |
| Title | Graph-Based Fitness Landscape Analysis |
| Description | Leveraging the advanced tools developed in the field of network science to facilite the understanding of complex configurable systems in different domains. |
| Type Of Technology | Software |
| Year Produced | 2024 |
| Open Source License? | Yes |
| Impact | This software is just released for a couple weeks, so the impacts are not yet generated. |
| URL | https://colalab.ai/docs/research/landscapes/ |
| Title | OmniGenome: A Comprehensive Toolkit of Genomic Modeling and Benchmarking |
| Description | OmniGenome is a comprehensive toolkit for genomic modeling and benchmarking. It provides a unified interface for various genomic modeling tasks, including genome sequence classification, regression and so on. OmniGenome is designed to be easy to use and flexible, allowing users to customize their workflows and experiment with different algorithms and parameters. It also includes a set of benchmarking tools to evaluate the performance of different genomic foundation models and compare their results. OmniGenome is written in Python and is available as an open-source project on GitHub. |
| Type Of Technology | Webtool/Application |
| Year Produced | 2025 |
| Open Source License? | Yes |
| Impact | Not yet to impact. |
| URL | https://github.com/COLA-Laboratory/OmniGenomeBench |
| Description | General Chair of a Turing Workshop on Human-Centric AI |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Other audiences |
| Results and Impact | I organized a three-day Turing Workshop on Human-Centered AI at the end of April. We have 54 registrations in total. They are not only from the UK, but also from the USA and Africa. We have 2 keynote speeches and 9 talks. The speakers are across the UK, from the south to the north, ranging from early career to established professors. We also have an interdisciplinary pool of people. In addition, we have two high-profile industrial speakers from GE Healthcare and NVIDIA. At the end, we have a panel discussion session on the emerging topics related to human-centered AI. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Give a 110 mins tutorial in 2023 IEEE CEC |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | Around 40 researchers across the world attended for this 110 minutes tutorial. This tutorial is about Decomposition Multi-Objective Optimisation: Current Developments and Future Opportunities. Within this tutorial, a comprehensive introduction to MOEA/D were given and selected research results will be presented in more detail. More specifically, it (i) introduced the basic principles of MOEA/D in comparison with other two state-of-the-art EMO frameworks, i.e., Pareto- and indicator-based frameworks; (ii) presented a general overview of state-of-the-art MOEA/D variants and their applications; (iii) discussed the future opportunities for possible further developments. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Program Chair of the 12th International Conference on Evolutionary Multi-Criterion Optimization in Leiden, the Netherlands |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | This is the most important academic event in the evolutionary multi-objective optimization community. After the COVID-19 pandemic, this event is hosted in a hybrid style with most attendees are in person with others attending online. There are more than 400 researchers across the world register this conference. In total there are around 130 papers got accepted and being presented during the conference. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://emo2023.liacs.leidenuniv.nl/ |