PHYDL: Physics-informed Differentiable Learning for Robotic Manipulation of Viscous and Granular Media

Viscous and granular media are ubiquitous in our daily life, ranging from food like dough or beans to construction materials like concrete, soils, or sand. Humans can use their hands to transform sands to any shape with ease; cooks can manipulate dough while cooking with proper tools; construction workers can control various types of machinery to mine, transfer or pile rocks and soil. Apart from that, there exist many other activities that involve manipulating granular media, including disaster rescue, space exploration, underwater exploration, agriculture and so forth. As a result, improving techniques that can enable automatic manipulation of such substances in an applicable way is rewarding for many parts of our society, but, currently, it is a big challenge in robotic control.

Conventional robot motion planning for manipulation focuses on safe and optimal trajectory generation with the assumption of rigid bodies of objects in the environment, that is, objects can only move or rotate but not deform. The intricate deformable geometric features and consequently the high unpredictability of material deformation due to the viscosity and granularity would prohibit direct applications of traditional robotic motion planning that is usually not scalable for such problems due to the requirement of explicitly designed models for rigid bodies. Techniques based on deep artificial neural networks and learning through intelligent agents interacting with the environment to achieve specific goals, known as Deep Reinforcement Learning (DRL), have become more popular for motion planning and decision making in complex environments without explicitly modelling the environment.

DRL trains an agent or a robot by rewarding desired behaviours and/or punishing undesired ones, such that the DRL agent will learn to interpret its environment perception and take optimal actions through trial and error. Usually, the DRL agent is trained in a realistic simulation environment without deploying a real robot to interact with the real-world environment directly. However, most simulators only support rigid-body environments. On the other hand, numerical modelling for simulating such materials is usually computationally prohibitive and impractical for efficient DRL. Moreover, DRL requires a robot or agent to explore the environment with a large number of randomly selected actions in order to learn from getting rewards or penalties that are usually highly inefficient and unsafe.

To address the above issues, this project will, for the first time, unlock a transformative robot learning framework by introducing a new technique, named differentiable physics, into the learning and control loop of the robot agent. This differentiable physics-based numerical simulation would greatly accelerate the simulation process, while on the other hand allow us to directly compute optimal physically-plausible actions without exploring all possible actions that are infinitely unbounded. In other words, we will leverage the differentiability nature for calculating physically-plausible bounded actions, which will reduce the amount of randomness for action exploration and hence allow a robot to learn more efficiently.

This recent tendency has attracted increasing attention in different communities such as robot trajectory planning and differentiable physics. This project will unlock a new robot learning framework for highly efficient, physically-plausible, and safe deep reinforcement learning for autonomous robots to learn to manipulate viscous and granular materials.