MAN^3: huMAN-inspired robotic MANipulation for advanced MANufacturing

Across the past 50 years, the use of robots in industry has monotonically increased, and it has literally boomed in the last 10 years. In 2016, the average robot density (i.e. number of robot units per 10,000 employees) in the manufacturing industries worldwide was 74; by regions, this was 99 units in Europe, 84 in the Americas and 63 in Asia, with an average annual growth rate (between 2010 and 2016) of 9% in Asia, 7% in the Americas and 5% in Europe. From 2018 to 2020, global robot installations are estimated to increase by at least 15% on average per year.
The main market so far has been the automotive industry (i.e. an example of heavy manufacturing), where simple and repetitive robotic manipulation tasks are performed in very controlled settings by big and expensive robots, in dedicated areas of the factories where human workers are not allowed to enter for safety reasons. New growing markets for robots are consumer-electronics and food/beverages (i.e. examples of light manufacturing) as well as other small and medium sized enterprises (SMEs): in particular, the food and beverage industry has increased robot orders by 12% each year between 2011 and 2015, and by 20% in 2016. However, in many cases the production processes of these industries require delicate handling and fine manipulations of several different items, posing serious challenges to the current capabilities of commercial robotic systems.
With 71 robot units per 10,000 employees (in 2016), the UK is the only G7 country with a robot density below the world average of 74, ranking 22nd. The industry and SME sector is highly in need of a modernization that would increase productivity and improve the working conditions (e.g. safety, engagement) of the human workers: this requires the development and deployment of novel robotic technologies that could meet the needs of those businesses in which current robots are yet not effective.

One of the main reasons why robots are not effective in those applications is the lack of robot intelligence: the ability to learn and adapt that is typical of humans. Indeed, robotic manipulation can be enhanced by relying on humans, both through interaction (i.e. humans as direct teachers) and through inspiration (i.e. humans as models).

Therefore, the aim of this project is to develop a system for natural human demonstration of robotic manipulation tasks, combining immersive Virtual Reality technologies and smart wearable devices (to interface the human with the robot) with robot sensorimotor learning techniques and multimodal artificial perception (inspired by the human sensorimotor system). The robotic system will include a set of sensors that allow to reconstruct the real world, in particual by integrating 3D vision with tactile information about contacts; the human user will access this artificial reconstruction through an immersive Virtual Reality that will combine both visual and haptic feedback. In other words, the user will see through the eyes of the robot, and will feel through the hands of the robot. Also, users will be able to move the robot just by moving their own limbs. This will allow human users to easily teach complex manipulation tasks to robots, and robots to learn efficient control strategies from the human demonstrations, so that they can then repeat the task autonomously in the future.

Human demonstration of simple robotic tasks has already found its way to industry (e.g. robotic painting, simple pick and place of rigid objects), but still it cannot be applied to the dexterous handling of generic objects (e.g. soft and delicate objects), that would result in a much larger applicability (e.g. food handling). Therefore, the expected results of this project will boost productivity in a large number of industrial processes (economic impact) and improve working conditions and quality of life of the human workers in terms of safety and engagement (social impact).

The project will impact different parts of our society: 1) academia and research; 2) economic sectors; 3) general public.

1) The research outcomes of the project will target different communities, namely: robotics, computer vision, tactile/force sensing, machine learning, automatic control, Virtual Augmented Reality, Human-Computer Interaction. Results will go beyond the state of the art of those subjects and will be disseminated through: publications in the top robotics (e.g. IJRR, T-RO, RAS, AURO), sensors (e.g. IEEE Sens. J., Sens. Actuators, Sensors), machine learning and computer vision (e.g. JMLR, Neural Networks, PAMI) journals and presentations to both national (e.g. TAROS, BMVC, IEEE UKRI RAS Chapter Conference) and international (e.g. ICRA, IROS, RSS, NIPS, ICML, IEEE Sensors, HRI) conferences; organisation of workshops at international robotics conferences (e.g. ICRA, IROS); organisation of national workshops (e.g. BMVA, the UK Manipulation Workshop, EPSRC UK-RAS network events).

2) The project will impact at least two categories of stakeholders in the secondary and tertiary economic sectors, i.e.: advanced manufacturers (industry and SMEs) and providers of robotics and AI solutions. These are the "two sides" looking into intelligent robots as an integral part of their future businesses.
Many SMEs and Industries in UK are willing to introduce robots on their production lines to improve their production (e.g. increase throughput and flexibility, reduce costs): they target collaborative robots, that can share the workspace with human workers and do not require the whole production lines to be modified. While low-cost collaborative robot arms are now available on the market, what is still missing are (i) intelligent manipulation capabilities in the end-effectors, and (ii) easy and intuitive ways to program the robots. These aspects still account for as much as 80 percent or more of the total deployment costs, hampering the successful introduction of collaborative robots in industry and SMEs. Remarkably, our research aim is to fill this gap by advancing the state of the art of human-demonstrated intelligent robotic manipulation.
Moreover, although we target manufacturing, the solutions that we will investigate in the project are beneficial for a wide range of additional applications, including manipulaiton in hazardous environments (e.g. chemical, radioactive), space and deep sea exploration, agriculture (e.g. precision weeding and picking), service and assistive robots at home and in public spaces.

3) Through public engagement activities, the project will raise awareness about how the proposed technologies will change the landscape of industrial production toward: a safer and more rewarding working experience for the human workers (released from the most dangerous, tiresome and repetitive aspects of their jobs) and an increased throughput and flexibility of the production (that generates higher income for the companies and therefore the possibility to employ more human workers in engaging and well-paid jobs, focused on the creativity and decision-making aspects). Indeed, through public engagement activities we aim to mitigate the increasing fear and discomfort about the mass introduction of robots in industry (i.e. one of the key pillars of Industry 4.0) in a part of the UK population, who is afraid to lose their jobs and economic power, with a negative impact on their quality of life; this fear is caused by a misguided conception of the real objective of robotics research in this area, which is not to replace humans with robots, but to improve the quality of the human work by using robots, both in terms of personal satisfaction of the worker and of overall quality of the production. In addition, by generating further understanding and interest in STEM subjects, we also aim to encourage young people into STEM studies and careers, regardless their gender, nationalities and social/ethnic groups.