Information Geometry and Reflexive Reinforcement Learning

Experimentation in a reinforcement learning agent is a process by which actions are drawn and the outcome has some evaluative signal. This is not dissimilar to humans, whereby we acquire knowledge through trying actions to determine the effect they have. Where this agent has some task to accomplish, this paradigm necessitates that an agent selects actions that are both maximally interesting in terms of information gain, but also actions that reduce the prediction error, i.e., actions which it has some knowledge or understanding of. This dichotomy is often called the "exploration-exploitation dilemma". There is an inherent duality in these two problems as they both are performed over the same state-action space, and as such it is hoped that we may be able to use Information Geometry, a sub field of information theory, and use the duality present in that to help bring together a more theoretical approach to solving the exploration-exploitation dilemma.

In doing so we have developed what we believe to be a novel approach to Reinforcement Learning which we call Reflexive Reinforcement Learning, whereby an agent better uses the evaluative signals generated over the trial and error learning process. Combining this with IRL we hope will lead to an adaptive expert agent who can change its policy over time to improve the efficacy of IRL as this is an ill-posed problem.

The Centre will have immediate short-term impacts on people skills and pipeline, alongside advances in scientific knowledge and techniques. However, with the strength of the program's training emphasis on innovation and social/societal challenges we also target longer term economic and societal benefits.
People: Centre graduates will be grounded in fundamental RAS topics and acquire advanced specialist scientific knowledge of crucial interaction themes. They will be skilled at teamwork, with a broader appreciation of RAS ethical issues. They will have international contacts and experience, with public presentation experience. Most importantly, they will be Innovation Ready - skilled in the principles of how technical and commercial disruption occurs, understanding how finance and organization realize new products and services in startup, SME and corporate situations. Their economic impact will be as industrial leaders of the future, foundational in realizing new products and services. This impact will be accelerated by our #Cauldron training programme in the interlinked areas of Scientific Cohesion, Research and Creativity Skills, Social and Societal Challenges, and programmed engagements and activities with our User Partners who shape the Centre's direction.
Science: The Centre will realize scientific advances, e.g. greater understanding of AI vs biomimetic approaches to persistent autonomy, advanced empathetic multimodal interaction between people and machines in smart spaces, advanced robotic micro-sensing and computing in soft embodiments, adaptive compliant actuation at a multitude of scales and form factors, semantic understanding of environments from noisy sensor data and more. Not only the advances, but also the research methods and practice to achieve them will be realized, e.g. hardware-in-the-loop architectures for re-usability and easy, low cost experimentation. The impact of these advances will be enhanced by strongly supported opportunities for dissemination, including conference presentations and publications (and training in presentation and writing skills), reciprocal secondments with Associate Research Partners, international student robot competitions, public outreach activities, CDT hosted international researcher visitors and workshops.
Society: Robotic and autonomous systems decrease cost and risk, increasing productivity while removing human operators from the 'dull, dirty and dangerous' tasks across the industries of our User Partners. Centre graduates and technology will contribute to maintaining UK business competitiveness and exports in this emerging Euro15.5Billion market, whilst improving quality of life for example a) more interesting (and prestigious) day-to-day employment for workers, b) assisted healthcare for an ageing population (including the Centre Directors), and c) greater awareness of environmental impacts and changes leading to policy and legislation.