Soft Robotic Skill Learning from Human Demonstration

Lead Research Organisation: King's College London
Department Name: Informatics

Abstract

SoftSkills will look at ways in which ordinary users can program robots way by teaching them how to control their movements and forces through demonstration. It will focus on using redundancy (additional degrees of freedom that are not directly required for a task) and compliance (the extent to which the robot gives way when subjected to external forces) in order to ensure safety and stability while teaching and interacting with robots, by blending behaviours taught to the robot, with specialist engineered controllers internal to the robot. In doing so, it is hoped that this will lead to new knowledge on how humans themselves exploit redundancy and compliance in their own musculoskeletal system, as well as enabling non expert users to program robots intuitively, leading to new applications in areas, such as manufacture, automation in the home and work place, transport, logistics, healthcare and agriculture.

Planned Impact

It is anticipated that advanced robotic systems will increasingly enter into interaction with humans and everyday objects and environments, with application in many areas, including manufacture, automation in the home and work place, transport, logistics, healthcare and agriculture. Key to their acceptance will be whether ordinary people are able to use them safely, in an intuitive way, to perform useful tasks. SoftSkills will examine how non expert users are able to program robots through demonstration, that is, by showing the robot what to do, rather than through traditional means such as coding in front of a computer. It will develop methods ensure the best performance out of these human robot interactions are by automatically blending user-programmed behaviours with expert controllers ensuring safety and stability. In the industrial domain, it is anticipated that the continuing rise of so called cobots, i.e., , collaborative robots that work in cooperation with human co-workers will, increasingly fuel economic demand for these enhanced robot capabilities. For example, enabling users to train robots skills in physically interacting with rigid/deformable surfaces opens up new domains of application e.g., service tasks such as wiping a table clean, medical tasks such as pressing soft tissues such as the skin of the abdomen to detect abnormalities, agricultural tasks such as running fingers over plants to inspect for diseases, among many others. Having these kinds of capabilities in a single, reprogrammable machine (as opposed to specialist, single purpose machinery), present an appealing business case to companies whose operations need to be dynamic in face of changes in consumer demands. Many do not invest in automation for the simple fact that consumer tastes change and products are continuously reinvented rendering specialist machinery obsolete. An adaptable, easy to use robotic system can help them improve productivity at lower labour cost, or enhance the working conditions of staff by relieving them of dangerous, dull or dirty tasks. An added benefit of facilitating the use of robots by ordinary users is that it may allow them to develop new robot uses, beyond the applications envisaged by engineers. The methods developed in SoftSkills are aimed at enabling users to program their own behaviours in an intuitive fashion, with the robot automatically compensating for things like safety and stability. This frees the user to flexibly experiment with the robot and find new tasks where it can be applied.
 
Description The findings that developed from this award is mainly four parts: i) The efficiency by which non-experts are able to teach robots equipped with machine learning capability is enhanced by giving them feedback on the state of the robot's understanding. This can be delivered through visual feedback and rule-based guidance. ii) Robot learning and control systems can be devised that selectively combine the best practice of human behaviour when performing some task, while avoiding their limitations (the latter arising from the physical properties of the human musculoskeletal system). Such a 'blended' controller outperforms direct imitation of human behaviours. iii) Redundancy (i.e., degrees of freedom in the robot that are unused when imitating a human) can be used automatically to optimise secondary behavioural goals. This includes maximising manipulability (thereby avoiding task singularitis that can result in unpredictable and unstable behaviour) and improving energy efficiency, for example, through energy harvesting by using independent physical damping control.
Exploitation Route One of the outputs of the project has been an open-source software package, that allows others in academic or industrial research to apply the developed robot learning algorithms to their own problems and applications. As the package is open source, the community can also carry on developing new methods in this domain, without license restrictions. While still in an early stage, the findings in regard to non-expert user interactions with collaborative robots lay the foundations for developing training programmes for people to use collaborative robots effectively. We are currently exploring how these can be put to use in several businesses with diverse production line requirements in the UK.
Sectors Aerospace, Defence and Marine,Agriculture, Food and Drink,Construction,Creative Economy,Education,Energy,Healthcare,Manufacturing, including Industrial Biotechology,Retail,Transport

 
Description The academic impact of these findings, in brief, are publications in high quality robotics conferences (i.e., ICRA and IROS) and journals (e.g., IEEE RA-L). The audience will gain a deep understanding of what is skill acquisition techniques in modern soft robotic systems. Outside academia, there has been social impact of this award, for example, in the education of young (school age) people in the capabilities of modern robotic systems and the skills needed to work in this domain. For example, the PI delivered a Christmas lecture to secondary school pupils at Kings College London, and the SoftSkills team ran a summer program in designing robots for vertical farming. The impact is in encouraging young students to dive into the field of robotics, potentially inspiring them to take this as a career path. Further social/economic impact can be found in the heightened awareness of the uses of soft, collabrative robots in industry, especially horticulture and agriculture. The PI gave evidence to the All-Party Parliamentary Gardening and Horticulture Group on this, and this was included in their inquiry into the future of the gardening and horticulture industry. The project team have delivered several keynote talks to industry about the uses of this technology. One outcome of this has been that a large grower of fresh produce has agreed to sponsor a PhD scholarship to apply the technologies on the production line in their main processing plant (due to the disruption of Covid-19, the latter project is scheduled for complete in September 2022). The PI is leading a working group to deliver a white paper on training needs for non-technical users of collaborative robots, informed by the findings of this project. Moreover, he continues to seek EU and UKRI investment into research to further the goal of standardising best practice in teaching collaborative robots in the UK.
Sector Agriculture, Food and Drink,Education,Manufacturing, including Industrial Biotechology
Impact Types Societal,Economic,Policy & public services

 
Description Invited to give evidence to the APPGHG Inquiry into the future of the gardening and horticulture industry
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
 
Description Rick Trainor Scholarship Scheme
Amount £69,000 (GBP)
Organisation King's College London 
Sector Academic/University
Country United Kingdom
Start 01/2018 
End 12/2020
 
Title Teaching Human Teachers to Teach Robot Learners 
Description The method has addressed two problems commonly observed in demonstration data sets that arise due to poor teaching strategies; undemonstrated states and ambiguous demonstrations. Overcoming these issues through the use of visual feedback and simple heuristic rules is investigated as a potential way of guiding novice users to more effectively teach robot learners to generalise a task. 
Type Of Material Model of mechanisms or symptoms - human 
Year Produced 2018 
Provided To Others? Yes  
Impact The method is filling the gap in the programming by demonstration community where the teaching efficiency of the learning algorithm maybe effected by the teacher. In another word, there is lack of study to investigate how to be a good teacher by given a learning algorithm. The method has been accepted by the ICRA conference and the quantitative results has been published so that more attentions will be gained in this field of research. 
 
Title The combination of the human and robot `best practice' for impedance control 
Description Physiologically, human stiffness and damping is coupled at the muscle level, restricting the ability to modulate impedance according to task demands. On the other hand, robotic systems do not have this issue, but the challenge is to plan a feasible variable impedance profile. In the proposed method, the task critical impedance controller is first learned for imitation and robot-specific controller is then blended. The user can use this blending scheme to take advantage of both human and robot 'best practice'. 
Type Of Material Model of mechanisms or symptoms - human 
Year Produced 2018 
Provided To Others? Yes  
Impact The paper has been submitted and the results will be disseminated at IROS 2018. A website has been created for broad casting the project and promotion over social media. The algorithm contains learning algorithm has been released in Github. The data have been shared through Figshare. 
 
Title energy regeneration 
Description Increasing research efforts have been made to improve the energy efficiency of variable impedance actuators (VIAs) through reduction of energy consumption. However, the harvesting of dissipated energy in such systems remains underexplored. The method proposes a novel variable damping module design enabling energy regeneration in VIAs by exploiting the regenerative braking effect of DC motors. The proposed damping module uses four switches to combine regenerative and dynamic braking, in a hybrid approach that enables energy regeneration without reduction in the range of damping achievable. 
Type Of Material Improvements to research infrastructure 
Year Produced 2018 
Provided To Others? Yes  
Impact The developed method has demonstrated the energy regeneration capability. The hardware has been implemented and integrates the hybrid damping module into a sensorised VIA to measure the combined energy consumption and regeneration during operation. Such information can be used for optimal decision making for task performance and energy cost trade-off. 
 
Title singularity avoidance 
Description The proposed method provides an approach, especially for non-expert users, to learn the constraint of a task such that it can be used optimise an autonomous controller to avoid singularity, and thereby unpredictable behaviour, with carrying out the task, without having to explicitly know the tasks constraint. 
Type Of Material Model of mechanisms or symptoms - human 
Year Produced 2018 
Provided To Others? No  
Impact The singularity avoidance problem is mainly concerned in the kinematic systems, however, it exists in the task as well. For instance, if the task is reaching a circle, the middle of the circle is the singularity of this task. Stepping into the singularity configuration in the task-space can be risky for both human and robot so that this method provides a controller which can avoid the the singularity in the task-space to avoid unnecessary behaviour on the robot. 
 
Title Teaching Human Teachers to Teach Robot Learners 
Description This is a dataset collected from 30 participants in a study on human-robot interaction during programming by demonstration. It was conducted under KCL ethics approval LRS-16/17-3800. This data is accompanying data to a paper that was accepted to the IEEE International Conference on Robotics and Automation (ICRA) 2018. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? Yes  
Impact The user can use the database to test the TP-GMR algorithm for a reaching task. The user will be able to notice the differences and improvement by introducing visual guidance and visual guidance with feedback, so that they can design their own testing environment by using the method proposed in the paper: `Teaching Human Teachers to Teach Robot Learners'. 
URL http://doi.org/10.18742/RDM01-242
 
Description RAS-user training needs white paper 
Organisation Harper Adams University
Country United Kingdom 
Sector Academic/University 
PI Contribution I am the lead of a working group to produce a white paper to identify future training needs to apply RAS technology to the UK agri-food sector.
Collaborator Contribution All other partners except Lincoln are contributing authors to the white paper. Lincoln is providing administrative support.
Impact None yet.
Start Year 2021
 
Description RAS-user training needs white paper 
Organisation University of Lincoln
Country United Kingdom 
Sector Academic/University 
PI Contribution I am the lead of a working group to produce a white paper to identify future training needs to apply RAS technology to the UK agri-food sector.
Collaborator Contribution All other partners except Lincoln are contributing authors to the white paper. Lincoln is providing administrative support.
Impact None yet.
Start Year 2021
 
Description RAS-user training needs white paper 
Organisation University of Reading
Country United Kingdom 
Sector Academic/University 
PI Contribution I am the lead of a working group to produce a white paper to identify future training needs to apply RAS technology to the UK agri-food sector.
Collaborator Contribution All other partners except Lincoln are contributing authors to the white paper. Lincoln is providing administrative support.
Impact None yet.
Start Year 2021
 
Description RAS-user training needs white paper 
Organisation University of Sheffield
Country United Kingdom 
Sector Academic/University 
PI Contribution I am the lead of a working group to produce a white paper to identify future training needs to apply RAS technology to the UK agri-food sector.
Collaborator Contribution All other partners except Lincoln are contributing authors to the white paper. Lincoln is providing administrative support.
Impact None yet.
Start Year 2021
 
Title A Library for Constraint Consistent Learning 
Description The Constraint Consistent Learning (CCL) library implements a family of data-driven methods that are capable of (i) learning state-independent and -dependent constraints, (ii) decomposing the behaviour of redundant systems into task and null-space parts, and (iii) uncovering the underlying null space control policy. It is a powerful tool to analyse and decompose many everyday tasks, such as wiping, reaching and drawing. 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact Practically, it presents the first unified application programmers interface (API) implementing and enabling use of these algorithms in a common framework. Academically, the methods implemented has already used and published papers based on it for human behaviour analysis. Furthermore, this unified API has summarised and implemented the methods which have been published for the past few years and added C functions, making it more useful and extendable for user. At the same time, it leads to a submission of paper `A Library for Constraint Consistent Learning' for gaining more academic interest and influence. 
URL https://nms.kcl.ac.uk/rll/CCL_doc/index.html
 
Description Exhibition at Science Museum 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact We plan to exhibit outcomes of the project at the Science Museum at the end of this month.
Year(s) Of Engagement Activity 2018
 
Description Guest Column in NFU Horticulture Magazine 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact I wrote a guest column for the NFU Horticulture Magazine, detailing the current state of the art of machine learning and robotics for horticultural production. This is to appear in print in the Spring issue.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at AHDB SmartHort 2019 Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 50 people attended the presentation, including policy makers, academic and industrial participants from across the UK.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Cambridge Agrifood Workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Third sector organisations
Results and Impact I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 50 people attended the presentation, including academic and industrial participants from across the UK.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Cambridge Agritech East Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 100 people attended the presentation, including academic and industrial participants from across the UK.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Horticulture 4.0 Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 100 people attended the presentation, including policy makers, academic and industrial participants from across the UK.
Year(s) Of Engagement Activity 2018
URL http://wsga.co.uk/wsga-horticulture-4-0-presentations
 
Description Invited talk at ICRA 2017 Workshop on Robot Learning 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The PI gave an invited talk at the ICRA 2017 Workshop on Robot Learning for Manipulation in Singapore. The audience consisted of leading robotics researchers from academia and industry, and provoked discussion and questions.
Year(s) Of Engagement Activity 2017
 
Description Invited talk at IROS 2018 Workshop on Robot Impeance Skills: Control and Learning 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I gave an invited talk at the IROS 2018 Workshop on Variable Impedance Skills: Control and Learning. The talk reported the outcomes of the project, and sparked debate around its findings. Around 50 people attended the presentation, and over 2000 attended the conference.
Year(s) Of Engagement Activity 2018
URL http://iros18wsimpedance.weebly.com/
 
Description Invited talk at IROS 2021 Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 50 people attended the presentation, including academic and industrial participants from across the world.
Year(s) Of Engagement Activity 2021
 
Description Keynote talk at Agritech East AI in Agriculture 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Approximately 100 people from business, industry, and technology development registered for the event, in which I was one of 6 invited speakers. The audience engaged in a lively discussion of the impact of AI and robotics in the agricultural sector. Early feedback on the event from the audience was extremely positive.
Year(s) Of Engagement Activity 2018
URL https://www.agritech-east.co.uk/events/smart-farming-the-aim-of-machine-to-machine-learning-in-agric...
 
Description Keynote talk at NFU Kent Branch AGM 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact I gave a keynote talk to the AGM of the NFU Kent branch about current opportunities for innovation in automation for farmers. The talk sparked considerable interest from the audience, including a discussion of the impacts of Brexit and how robotic/machine learning technology could address them.
Year(s) Of Engagement Activity 2017
 
Description Talk at the Innovate UK Automation and robotics in the AgriFood sector 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact The PI presented gave a 'pitch talk' at this event. It resulted in greater awareness of the project work and stimulated follow up enquiries from interested parties, including Innovate UK representatives, and members of the UK Forestry Commission.
Year(s) Of Engagement Activity 2017
URL https://www.eventbrite.co.uk/e/automation-and-robotics-in-the-agrifood-sector-expression-of-interest...