Autonomous physics-based object manipulation

Lead Research Organisation: University of Leeds
Department Name: Sch of Computing

Abstract

Planning of object manipulation is an important problem in robotics. Imagine an autonomous home-helper robot reaching into a cluttered fridge shelf: the robot will need to make contact with multiple objects and push them out the way. The robot's actions must be carefully planned to avoid pushing objects off the shelf, causing them to fall down and break. Existing algorithms solve this planning problem by searching over possible sequences of actions. Each action is applied for a discrete time step, and the planner chooses the sequence which is predicted to reach the goal object without damaging other objects in the scene. The effect of each action during a time step is computed using a physics simulation, which can either be a complete dynamics simulation or a simplified quasi-static model. However, a major problem with existing algorithms is speed: The number of possible action sequences increases exponentially with the planning horizon, leading to unacceptably long planning times. New methods that reduce planning time can have a major impact in this area, enabling robots to perform manipulation tasks that currently exceed their capabilities. In this project we will develop new optimisation based approaches to autonomous physics-based object manipulation in clutter.

Studentship Projects

Project Reference Relationship Related To Start End Student Name
EP/N509681/1 01/10/2016 30/09/2021
1879668 Studentship EP/N509681/1 01/04/2017 31/03/2021 Wisdom Agboh
 
Description Our goal is to reduce the total time it takes for a robot to successfully complete a physics-based robotic manipulation task. We do this by developing faster planning algorithms/frameworks in general and/or using physics models that are not computationally expensive.

Specifically, we have achieved our objectives through four key ideas:

1. We proposed a new physics-based robotic manipulation planning framework. It involves an optimisation framework based on sampling that exploits parallel computing to reduce task completion times for several minutes in prior work to only seconds. A robot's manipulation plan can fail in the face of uncertainty in for instance object positions or physics predictions. We found that an online re-planning scheme significantly improved task success rates. Our approach allows the robot the quickly update its plans online during execution of an initial plan.

2. Robots typically use only slow and conservative actions to complete physics-based manipulation tasks. One reason for this is so that they can have a very good control over what happens during execution. In this way, the robot can quickly react to any undesired events before they occur. We have proposed a new planning and control framework that allows a robot to adapt its actions to the task. Just like humans do, we found that a robot can use fast or slow actions depending on the task. For example, it can use a fast action to throw a mobile phone onto a large bed, but slower actions to reach for a glass in the shelf. This framework allows the robot to complete tasks as fast as possible.

3. Physics models are computationally expensive. They are the major computational bottleneck for physics-based robotic manipulation. We proposed new physics models for robotic manipulation. We combined coarse physics models (cheap to compute but inaccurate) with fine physics models (expensive to compute but accurate) to generate a spectrum of physics models. A robot can explore this spectrum of physics models to pick a model that is suitable for a given task. For example, if the goal is to push an object into a large region on a wide table, a fast and coarse model may be enough for planning. However, if the goal is to reach into a shelf to pick up a wine glass, the robot chooses to plan with a fine physics model - otherwise it can break the glass.

4. We found that a robot can plan for different parts of a trajectory using different physics models. This is a hierarchical planning strategy that significantly reduced planning time (for example by an order of magnitude for problems where a robot needs to reach into a cluttered space to pick up a target object).
Exploitation Route Below are several ways findings from this research can be taken forward and used by others (e.g. other robotics researchers).

1. Beyond Pushing: This project has focused on pushing as the main manipulation primitive. Future work can investigate other primitives such as tilting and throwing within the proposed planning and control frameworks to generate new robot capabilities.

2. Coarse Physics Models for Robotic Manipulation: An exciting question is how to build coarse models for difficult manipulation tasks e.g. for manipulating a deformable object such as a rope. Future work can focus on building these models for different tasks.

3. Planning Fast and Slow: Our prior work generates a spectrum of physics models increasing in accuracy and computational cost from coarse to fine. An interesting problem for future work is to autonomously decide which physics model to use for a given manipulation task. In this way, planning time will vary depending on the task.
Sectors Agriculture, Food and Drink,Construction,Manufacturing, including Industrial Biotechology,Retail
 
Description Travel Grant for the 8th Workshop on Parallel-in-Time Integration
Amount € 1,500 (EUR)
Organisation Julich Research Centre 
Sector Academic/University
Country Germany
Start 04/2019 
End 05/2019
 
Description Human-Like Computing 
Organisation University of Leeds
Department Institute of Psychological Sciences
Country United Kingdom 
Sector Academic/University 
PI Contribution The Human-Like Comnputing Project involves creating robotic systems that generate human-like motions. Our research team contributed a trajectory optimization algorithm that generates human-like robot motions. It takes a high level human-like plan as input. For example, which objects to push and to what directions. Our team also contributed a real robot used to investigate the performance of the human-like robot plans in the real world.
Collaborator Contribution The partners contributed a virtual reality (VR) dataset. It involves participants reaching into cluttered spaces to retrieve target objects using VR. They also proposed a framework where a robot learns from this human data to generate a high level sequence of actions. For example, to retrieve an object in a cluttered environment, a high level plan can be which objects to push aside and to what directions.
Impact 1. A conference publication: Hasan M., Warburton M., Agboh W.C., Dogar M.R., Leonetti M., Wang H., Mushtaq F., Mon-Williams M., and Cohn A.G. "Introducing a Human-like Planner for Reaching in Cluttered Environments". International Conference on Robotics and Automation (ICRA), 2020.
Start Year 2019
 
Description Parallel-inTime Integration for Robotics 
Organisation Hamburg University of Technology
Country Germany 
Sector Academic/University 
PI Contribution The goal of the partnership is to apply parallel-in-time integration methods to Robotics. Specifically, to speed-up physics simulation during robotic manipulation. Our research team contributed planning and control algorithms that use a physics model for robotic manipulaiton. We also provided solutions to new problems encountered when parallel-in-time integration methods are used to generate physics models for robotic manipulation e.g. the infeasible state problem.
Collaborator Contribution The partner contributed knowledge and insight about parallel-in-time algorithms in general and how they can be applied to simulate physics for robotic manipulation.
Impact Outputs 1. Conference paper: Agboh W.C., Ruprecht D., and Dogar M. "Combining Coarse and Fine Physics for Manipulation using Parallel-in-Time Integration", International Symposium on Robotics Research (ISRR), 2019. 2. Journal article Agboh W.C., Grainger O., Ruprecht D., and Dogar M " Parareal with a Learned Coarse Model for Robotic Manipulation", Journal of Computing and Visualization in Science, 2019 (under review).
Start Year 2019