Scalable Blackbox Optimisation for Machine Learning

My work is highly relevant for the EPSRC Research Area: Information and Communication Technologies, as I seek to solve problems impacting the forefront of Artificial Intelligence research.

Recent work has shown that quasi-random search methods such as Evolution Strategies (ES) can scale to millions of dimensions, achieving competitive performance vs. state of the art in a variety of reinforcement learning problems. This exciting result has led to a flurry of research activity, and challenged what many thought was possible for ES methods.

Despite this excitement, ES methods remain incredibly sample inefficient, since they essentially disregard all data after a single use. As such, they currently require large parallel computing resources to achieve their impressive results.

Conversely, Bayesian Optimisation methods are many orders of magnitude more efficient, often solving problems in only a handful of trials. Bayesian Optimisation has been successfully used in many industrial settings, and even led to dramatic improvements in the recent AlphaGo algorithm which beat the world's best Go player - a landmark result for Artificial Intelligence research. Furthermore, Bayesian Optimisation methods can achieve these results using a principled mathematical approach, which is severely lacking in most high performing ES algorithms. However, most Bayesian Optimisation success stories come from a lower dimensional regime, and they are generally regarded to lack the scalability of other methods.

Despite tackling the same problem, these two communities remain distinct. It is my goal to bring them together, to build sample efficient blackbox optimisation algorithms which can scale to high dimensions, with a theoretical underpinning. I believe there are significant opportunities in this area, which are yet to be explored due to the vast ideological differences involved. As of now, there has been very little cross-pollination, since the two methods are at opposite ends of the spectrum.

In the first year of my studies, I plan to tackle the problem from several different directions:
1) Bringing a Bayesian approach to the recently introduced Population Based Training algorithm, shown to dramatically improve performance for neural networks.
2) Using probabilistic measures of diversity to boost exploration in populations of ES agents during optimisation.
3) Using Bayesian principles to actively acquire data in a reinforcement learning setting.

In the second and third years, I hope to not only introduce more novel algorithms, but also demonstrate the effectiveness of these approaches in real-world settings.

I believe this research could have a large impact, since blackbox optimisers are used in a variety of settings such as reinforcement learning, or hyper-parameter tuning for industrial scale deep learning projects. All of these settings have multiple use cases downstream, magnifying the impact.