Planning, control and learning for intelligent robotic manipulation

Lead Research Organisation: University of Leeds
Department Name: Sch of Computing


This project will develop intelligent robotic algorithms and systems for object manipulation tasks. Example tasks include a robot reaching into a cluttered shelf to grasp objects and a robot using tools to cut and drill objects)

Particular topics include:

Manipulation planning using physics-based predictions of object motion, Machine learning for manipulation planning and control, Manipulation based on tactile sensing (e.g. artificial skin), Multi-robot collaborative manipulation, Human-robot collaborative manipulation, and Manipulation planning for flexible manufacturing and assembly.


10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/N509681/1 30/09/2016 29/09/2021
1949457 Studentship EP/N509681/1 30/09/2017 30/07/2021 Rafael Papallas
Description We proposed an approach to robotic manipulation in cluttered environments. We tackle the problems where a robot needs to reach an object on a cluttered shelf or a fridge. This problem has the potential for near-term impact in warehouse management (where robots need to reach into shelves to retrieve objects) and personal home robots (where a robot might need to reach into a fridge or shelf to retrieve an object).

However, this is a challenging problem for today's robots because the robots need to reason how and which objects to push out of the way to make space to reach for that goal object. The robots need to consider a large number of possibilities for their solution and therefore it is infeasible for robots, today, to explore every single possibility in a reasonable amount of time. Successful reasoning in a reasonable time, therefore, is extremely hard for autonomous robots and state-of-the-art approaches that tackle these problems today suffer from long planning times (the time it takes to find a solution to a problem) and low success rates. We recognize that humans are capable of making these decisions in a split of a second, yet we are not able to explain how we make these decisions. Our approach leverages this human intuition and integrates it into the algorithm used by the robot to reason which objects to push out of the way. Our approach shows an increase in success rate and an improvement in planning times compared to state-of-the-art methods.

Our contribution, therefore, is the integration of human intuition in these problems for physics-based manipulation in cluttered environments. We presented an approach to integrate human-input in these problems and we demonstrated that this approach performs better than existing methods.
Exploitation Route The problems we are interested in have the potential for near-term impact in warehouse management and personal home robots. Our approach could be used in warehouse management where remote human-operators can assist robots working in the warehouse. Potentially our approach can be used in personal home robots to help people with disabilities to carry on daily activities like retrieving a bottle of water from the back of the fridge.
Sectors Other

Title MuJoCo UR5 Model 
Description A model for the physics-simulator for the UR5 robot. This model replicates the UR5 robot and extends MuJoCo robot repertoire. 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact MuJoCo (the physics simulator) is being used in this research extensively to produce research and the publication (ICRA 2020) made extensive use of this model and MuJoCo. 
Title OpenRAVE UR5 Controller 
Description OpenRAVE is a widely-used simulator in Robotics. This software extends OpenRAVE robot repertoire to also include the UR5 robot used in our research. The controller is capable of replicating the real-robot in simulation and executing solutions planned in OpenRAVE on the real-robot. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact The software made possible the experimentation of research methods. Without this software, it would not be possible to run experiments and evaluate the research ideas on the real-robot which is critical for research output. Moreover, the software, as an open-source project, attracted attention of other people working in this area and has been used extensively by another colleague.