Improving Sample Efficiency of Reinforcement Learning

Deep reinforcement learning has had huge empirical success and is a major enabling technology for many applications of AI. However, recent RL algorithms still require millions of samples to obtain good performance. Since obtaining environment interactions is often costly and since challenging environments are rarely static, this inhibits many practical applications. This project will investigate ways of reducing this cost, aiming to find more sample-efficient RL algorithms. We aim for the algorithms to be deployable in realistic settings, where agents use deep networks to represent knowledge about the environment. It is also likely to lead to improved performance of other systems making automated decisions.

Research Strategy

The project will investigate two main avenues for improving sample efficiency. Firstly, using a Bayesian framework to gain additional information from samples, we hope to achieve improved exploration which will in turn lead to more informative samples. Secondly, using meta-learning, we hope to enable generalisation, reducing the amount of samples required to learn a task which is similar to other learned tasks.

Objectives and Applications

We aim to develop more sample efficient reinforcement learning algorithms and to gain new insights about exploration. The project will be carried out in collaboration with Microsoft Research Cambridge and will have immediate relevance for their computer games research, particularly for training game AI in complex worlds where samples are expensive. The project will have wider applications for any problem which involves decision making with limited data, including real world applications such as robotics and pricing strategies.