Grasping Grocery Items with Sensorised Robotic Manipulators and Reinforcement Learning

Robotic manipulation remains an unsolved challenge in fully autonomous industrial settings, such
as automated supermarkets (according to Ocado - the world's largest online-only supermarket).
Successful solutions must enable robotic systems to operate robustly in uncertain, unstructured, and
dynamic environments. Thus, methodologies inspired by Nature, i.e. humans and animals, may play
the ideal role model, in particular, for robotic grasping and manipulation systems Karmakar et al.,
2019. Application dependent solutions are required able to cope with different types and shapes of
objects on a day-today basis. The use-cases are vast - a challenging example, though, involves the
individually sorting and packing of warehouse products in a fully automatically way Sotiropoulos
et al., 2018. One solution to this problem may be based off robotic manipulators Bogue et al.,
2016. The rapidly growing research on soft robotics has seen its application in fields including
human-robot interaction Arnold et al., 2017 allowing inherently safe collaboration with humans
Endo et al., 2013. The idea of robotic softhand manipulation may help in tackling some of these
presented challenges. Projects, such as SoMa(Soft Manipulation) 6, with a close collaboration
between universities and industry, attracted multimillion investment of the European funding under
Horizon 2020, to benchmark several soft grippers andhands for fruit grasping tasks.
Robotic end effectors proposed for handling irregularly shaped, soft and easily damaged goods
(e.g., fruits and vegetables) are currently divided from state of the art grasping with machine
learning. Soft grippers achieve best performance on these grocery items Friedl et al. 2020, Angelini
et al., Mnyusiwalla et al. 2020; however, the most effective machine learning based approaches
only investigate rigid parallel-jaw grippers or suction [Levine et al., 2018, Mahler et al., 2019].
There is a need to investigate new robotic end-effectors that are suited for combing both soft
behaviour and machine learning approaches.
This PhD research thesis will focus on a developing a grasping manipulator that uses sensorial
feedback control (i.e.tactile sensors as touch sensing) to complete robust and effective
handling of irregularly shaped, soft and easily damaged goods. This gripper will integrate with
machine learning techniques, notably being suited for accurate simluation. Such a project will be
timely and innovative as it will deliver positive economic impact in the automation of the foreseen
future. We plan designing novel hardware for applications that require high complexity in the end to-end research pipeline. We will focus on coupling the design with the required automation, such
as task planning and collision-free path planning, using advanced perception systems through both
geometrical modelling of the environment and learning techniques.
The unique focus of the project that will guarantee a successful implementation, is the integration of
Robotics and AI expertise in the field for as close cross-departmental collaboration of two thriving
UCL university lecturers Dr. Dimitrios Kanoulas (Lecturer in Robotic Sensing and Manipulation
atUCL Computer Science) and Dr. Helge Wurdemann (Lecturer in Robotics and Haptics at UCL
Mechanical Engineeing). Dr. Kanoulas offers extensive research experience in the field of
perception and learning and would advise the algorithmic implementation and analysis of the
project proposal, while Dr.Wurdemann, with his expertise in the creation of novel and innovative
soft robotic systems, will advise on the design of the soft-hand manipulator and implementation.