Learning Loco-Manipulation for Articulated Robots

Advancements in legged robot technology will enable a wide range of applications, from planetary and cave exploration, to traversing dangerous environments for search and rescue operations and benefiting conservation efforts through autonomous environmental monitoring. In the UK there has been a strong incentive towards deploying legged robots for inspection of nuclear and offshore power plants, with the goal of minimising the exposure of human workers to risks in hazardous and remote settings. In a domestic setting such robots have the potential the improve safety, convenience and quality of life, helping with tasks tasks like cleaning and security, and providing assistance to elderly or mobility-impaired people.

One of the greatest challenges with legged robots lies in enabling them to traverse the complex and rapidly changing environments around us. In this research project we will tackle this problem through the use of machine learning, and in particular Reinforcement Learning (RL). By allowing robots to learn from their experience we can greatly improve their performance and robustness to the dynamic nature of the everyday world around them.

The main goal of this project is to develop methods of control that can greatly enhance the locomotion and traversal capabilities of legged robots. To this end, we would investigate how a legged robot can learn to take in information from its surroundings to adapt to the uncertainty and changes that occur around it, known as short-horizon motion planning. This can boost their agility and balance, and allow them to safely walk through a wider range of harsh outdoor and indoor environments alike.

In nature, animals not only react to sudden disruptions but can actively anticipate disturbances and preemptively modify their plans and adapt their locomotion. As an example, through a combination of their senses and past knowledge, humans can anticipate a slippery surface and adjust their step to avoid slipping before it happens. Moreover, reactive behaviour is not sufficient when traversing highly irregular environments, which necessitate a greater level of foresight and planning in the long-term, several seconds or even minutes ahead. To perform highly agile motions requires not only purely reflexive behaviour, but a greater deal of reasoning about the properties of the world around the robot, and the ability to form complex motion plans.

To tackle this challenge we will employ a novel approach known as Deep Reinforcement Learning (DRL). DRLearning leverages big data through simulation-based training of neural networks to design much more agile and robust motion controllers. It allows such controllers to learn from their mistakes and their experience and continuously improve.
Recent works have shown that through RL legged robots can be taught to traverse a much more diverse set of outdoor terrains than traditional methods. In this project we would apply RL to much more complex environments, the traversal of which requires a combination of advanced decision-making, foresight and perception of the world. A strong emphasis throughout the project will be the deployment and testing of these motion controllers on real robots. This way we can ensure that our controllers are robust and flexible enough to leave the lab and be used in the real world. Ensuring the safety of these systems, especially when operating in the presence of humans, is crucial and remains an open problem in the field. Therefore, as part of our research questions we will further investigate how learning-based controllers can incorporate interpretability and safety considerations.

This project falls within the EPSRC Artificial intelligence and Robotics research area. As part of an industrial CASE project, we will work closely with Dyson, the industrial partner. This can help bridge the gap between academic research and industry, and promote the development of systems that can be robustly and safely deployed.