Physics-Informed Reinforcement Learning for Tactile Manipulation

Object manipulation in unstructured and unfamiliar environments is a powerful tool that can greatly increase the value of robotics. However, research effort in this area has been largely focused on using vision as input for control whilst little attention has been given to the application of touch. Tactile sensing is key for achieving human-level dexterity. Measurements such as contact compliance, contact location, shear, pressure, slip, and other important mechanical properties of contact, that would otherwise be inaccessible through vision, can be obtained even when objects are partially occluded. In my research, I hope to harness this power of touch to develop control frameworks to support robots' interaction with the real-world.

To fully realise the value of autonomous manipulation, robots need to operate under a wide range of pervasive and unsystematic scenarios that it nor its designers have foreseen before, many of which can be impossible to model. This calls for advanced control frameworks with high versatility and generality that is also robust enough to deal with a range of unpredictable situations. Reinforcement learning (RL) is a popular learning algorithm that can allow robots to learn a complex controller through trial and error. Adopting such technique would simplify the controller design problem, avoiding the need to develop specialised controller for every scenario in our unstructured world and significantly reducing the research effort required to achieve human-level dexterity. However, reinforcement learning can be extremely fragile. The learning process can be very sensitive to certain design parameters and small deviations from expected behaviour can lead to catastrophic failures. It also suffers from high sample complexity which makes real-world learning unfeasible. These drawbacks have meant that much of the recent advances in reinforcement learning for robotics have involved experimenting in simulations before transferring to the real-world, creating many sim-to-real problems.

Model-based reinforcement learning is a more recent approach that aims to tackle these shortcomings by first learning a representation of the transition dynamics before using it to derive the optimal controller. This has been shown to be more effective for robot control where high-capacity models such as neural networks or probabilistic models can be used to solve complex control problems with reduced number of interactions. This not only makes real-world learning possible, but the existence of a model can improve robustness amongst many other benefits. The application of model-based reinforcement learning to tactile robotics has very much been underexplored and I believe this combination could massively progress the capabilities of tactile manipulation. My research will aim to further this vision, with a focus on safe and efficient real-world learning and deployment. I hope to develop methodologies for synthesising controller that is; sample efficient in the online learning process to exhibit adaptive behaviour, robust in deployment to be able to deal with unpredictable situations, and also remain general in formulation such that it can be applied to a range of tactile manipulation tasks. A potential direction I hope to explore is to incorporate physics-priors to guide the learning process which can potentially mitigate the major efficiency issues associated with the trial-and-error nature of reinforcement learning. This research falls within the EPSRC artificial intelligence and robotics area.