Soft Robotic Skill Learning from Human Demonstration
Lead Research Organisation:
King's College London
Department Name: Informatics
Abstract
SoftSkills will look at ways in which ordinary users can program robots way by teaching them how to control their movements and forces through demonstration. It will focus on using redundancy (additional degrees of freedom that are not directly required for a task) and compliance (the extent to which the robot gives way when subjected to external forces) in order to ensure safety and stability while teaching and interacting with robots, by blending behaviours taught to the robot, with specialist engineered controllers internal to the robot. In doing so, it is hoped that this will lead to new knowledge on how humans themselves exploit redundancy and compliance in their own musculoskeletal system, as well as enabling non expert users to program robots intuitively, leading to new applications in areas, such as manufacture, automation in the home and work place, transport, logistics, healthcare and agriculture.
Planned Impact
It is anticipated that advanced robotic systems will increasingly enter into interaction with humans and everyday objects and environments, with application in many areas, including manufacture, automation in the home and work place, transport, logistics, healthcare and agriculture. Key to their acceptance will be whether ordinary people are able to use them safely, in an intuitive way, to perform useful tasks. SoftSkills will examine how non expert users are able to program robots through demonstration, that is, by showing the robot what to do, rather than through traditional means such as coding in front of a computer. It will develop methods ensure the best performance out of these human robot interactions are by automatically blending user-programmed behaviours with expert controllers ensuring safety and stability. In the industrial domain, it is anticipated that the continuing rise of so called cobots, i.e., , collaborative robots that work in cooperation with human co-workers will, increasingly fuel economic demand for these enhanced robot capabilities. For example, enabling users to train robots skills in physically interacting with rigid/deformable surfaces opens up new domains of application e.g., service tasks such as wiping a table clean, medical tasks such as pressing soft tissues such as the skin of the abdomen to detect abnormalities, agricultural tasks such as running fingers over plants to inspect for diseases, among many others. Having these kinds of capabilities in a single, reprogrammable machine (as opposed to specialist, single purpose machinery), present an appealing business case to companies whose operations need to be dynamic in face of changes in consumer demands. Many do not invest in automation for the simple fact that consumer tastes change and products are continuously reinvented rendering specialist machinery obsolete. An adaptable, easy to use robotic system can help them improve productivity at lower labour cost, or enhance the working conditions of staff by relieving them of dangerous, dull or dirty tasks. An added benefit of facilitating the use of robots by ordinary users is that it may allow them to develop new robot uses, beyond the applications envisaged by engineers. The methods developed in SoftSkills are aimed at enabling users to program their own behaviours in an intuitive fashion, with the robot automatically compensating for things like safety and stability. This frees the user to flexibly experiment with the robot and find new tasks where it can be applied.
Publications
Aran S
(2018)
Teaching Human Teachers to Teach Robot Learners
Manavalan J
(2020)
A library for constraint consistent learning
in Advanced Robotics
Manavalan J
(2018)
Learning Singularity Avoidance
Manavalan J
(2019)
Learning Singularity Avoidance
Sena A
(2019)
Quantifying teaching behavior in robot learning from demonstration
in The International Journal of Robotics Research
Wu F
(2020)
Energy Regenerative Damping in Variable Impedance Actuators for Long-Term Robotic Deployment
in IEEE Transactions on Robotics
Zhao YC
(2018)
A Library for Constraint Consistent Learning
Description | The findings that developed from this award is mainly four parts: i) The efficiency by which non-experts are able to teach robots equipped with machine learning capability is enhanced by giving them feedback on the state of the robot's understanding. This can be delivered through visual feedback and rule-based guidance. ii) Robot learning and control systems can be devised that selectively combine the best practice of human behaviour when performing some task, while avoiding their limitations (the latter arising from the physical properties of the human musculoskeletal system). Such a 'blended' controller outperforms direct imitation of human behaviours. iii) Redundancy (i.e., degrees of freedom in the robot that are unused when imitating a human) can be used automatically to optimise secondary behavioural goals. This includes maximising manipulability (thereby avoiding task singularitis that can result in unpredictable and unstable behaviour) and improving energy efficiency, for example, through energy harvesting by using independent physical damping control. |
Exploitation Route | One of the outputs of the project has been an open-source software package, that allows others in academic or industrial research to apply the developed robot learning algorithms to their own problems and applications. As the package is open source, the community can also carry on developing new methods in this domain, without license restrictions. While still in an early stage, the findings in regard to non-expert user interactions with collaborative robots lay the foundations for developing training programmes for people to use collaborative robots effectively. We are currently exploring how these can be put to use in several businesses with diverse production line requirements in the UK. |
Sectors | Aerospace, Defence and Marine,Agriculture, Food and Drink,Construction,Creative Economy,Education,Energy,Healthcare,Manufacturing, including Industrial Biotechology,Retail,Transport |
Description | The academic impact of these findings, in brief, are publications in high quality robotics conferences (i.e., ICRA and IROS) and journals (e.g., IEEE RA-L). The audience will gain a deep understanding of what is skill acquisition techniques in modern soft robotic systems. Outside academia, there has been social impact of this award, for example, in the education of young (school age) people in the capabilities of modern robotic systems and the skills needed to work in this domain. For example, the PI delivered a Christmas lecture to secondary school pupils at Kings College London, and the SoftSkills team ran a summer program in designing robots for vertical farming. The impact is in encouraging young students to dive into the field of robotics, potentially inspiring them to take this as a career path. Further social/economic impact can be found in the heightened awareness of the uses of soft, collabrative robots in industry, especially horticulture and agriculture. The PI gave evidence to the All-Party Parliamentary Gardening and Horticulture Group on this, and this was included in their inquiry into the future of the gardening and horticulture industry. The project team have delivered several keynote talks to industry about the uses of this technology. One outcome of this has been that a large grower of fresh produce has agreed to sponsor a PhD scholarship to apply the technologies on the production line in their main processing plant (due to the disruption of Covid-19, the latter project is scheduled for complete in September 2022). The PI is leading a working group to deliver a white paper on training needs for non-technical users of collaborative robots, informed by the findings of this project. Moreover, he continues to seek EU and UKRI investment into research to further the goal of standardising best practice in teaching collaborative robots in the UK. |
Sector | Agriculture, Food and Drink,Education,Manufacturing, including Industrial Biotechology |
Impact Types | Societal,Economic,Policy & public services |
Description | Invited to give evidence to the APPGHG Inquiry into the future of the gardening and horticulture industry |
Geographic Reach | National |
Policy Influence Type | Contribution to a national consultation/review |
Description | Rick Trainor Scholarship Scheme |
Amount | £69,000 (GBP) |
Organisation | King's College London |
Sector | Academic/University |
Country | United Kingdom |
Start | 01/2018 |
End | 12/2020 |
Title | Teaching Human Teachers to Teach Robot Learners |
Description | The method has addressed two problems commonly observed in demonstration data sets that arise due to poor teaching strategies; undemonstrated states and ambiguous demonstrations. Overcoming these issues through the use of visual feedback and simple heuristic rules is investigated as a potential way of guiding novice users to more effectively teach robot learners to generalise a task. |
Type Of Material | Model of mechanisms or symptoms - human |
Year Produced | 2018 |
Provided To Others? | Yes |
Impact | The method is filling the gap in the programming by demonstration community where the teaching efficiency of the learning algorithm maybe effected by the teacher. In another word, there is lack of study to investigate how to be a good teacher by given a learning algorithm. The method has been accepted by the ICRA conference and the quantitative results has been published so that more attentions will be gained in this field of research. |
Title | The combination of the human and robot `best practice' for impedance control |
Description | Physiologically, human stiffness and damping is coupled at the muscle level, restricting the ability to modulate impedance according to task demands. On the other hand, robotic systems do not have this issue, but the challenge is to plan a feasible variable impedance profile. In the proposed method, the task critical impedance controller is first learned for imitation and robot-specific controller is then blended. The user can use this blending scheme to take advantage of both human and robot 'best practice'. |
Type Of Material | Model of mechanisms or symptoms - human |
Year Produced | 2018 |
Provided To Others? | Yes |
Impact | The paper has been submitted and the results will be disseminated at IROS 2018. A website has been created for broad casting the project and promotion over social media. The algorithm contains learning algorithm has been released in Github. The data have been shared through Figshare. |
Title | energy regeneration |
Description | Increasing research efforts have been made to improve the energy efficiency of variable impedance actuators (VIAs) through reduction of energy consumption. However, the harvesting of dissipated energy in such systems remains underexplored. The method proposes a novel variable damping module design enabling energy regeneration in VIAs by exploiting the regenerative braking effect of DC motors. The proposed damping module uses four switches to combine regenerative and dynamic braking, in a hybrid approach that enables energy regeneration without reduction in the range of damping achievable. |
Type Of Material | Improvements to research infrastructure |
Year Produced | 2018 |
Provided To Others? | Yes |
Impact | The developed method has demonstrated the energy regeneration capability. The hardware has been implemented and integrates the hybrid damping module into a sensorised VIA to measure the combined energy consumption and regeneration during operation. Such information can be used for optimal decision making for task performance and energy cost trade-off. |
Title | singularity avoidance |
Description | The proposed method provides an approach, especially for non-expert users, to learn the constraint of a task such that it can be used optimise an autonomous controller to avoid singularity, and thereby unpredictable behaviour, with carrying out the task, without having to explicitly know the tasks constraint. |
Type Of Material | Model of mechanisms or symptoms - human |
Year Produced | 2018 |
Provided To Others? | No |
Impact | The singularity avoidance problem is mainly concerned in the kinematic systems, however, it exists in the task as well. For instance, if the task is reaching a circle, the middle of the circle is the singularity of this task. Stepping into the singularity configuration in the task-space can be risky for both human and robot so that this method provides a controller which can avoid the the singularity in the task-space to avoid unnecessary behaviour on the robot. |
Title | Teaching Human Teachers to Teach Robot Learners |
Description | This is a dataset collected from 30 participants in a study on human-robot interaction during programming by demonstration. It was conducted under KCL ethics approval LRS-16/17-3800. This data is accompanying data to a paper that was accepted to the IEEE International Conference on Robotics and Automation (ICRA) 2018. |
Type Of Material | Database/Collection of data |
Year Produced | 2018 |
Provided To Others? | Yes |
Impact | The user can use the database to test the TP-GMR algorithm for a reaching task. The user will be able to notice the differences and improvement by introducing visual guidance and visual guidance with feedback, so that they can design their own testing environment by using the method proposed in the paper: `Teaching Human Teachers to Teach Robot Learners'. |
URL | http://doi.org/10.18742/RDM01-242 |
Description | RAS-user training needs white paper |
Organisation | Harper Adams University |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | I am the lead of a working group to produce a white paper to identify future training needs to apply RAS technology to the UK agri-food sector. |
Collaborator Contribution | All other partners except Lincoln are contributing authors to the white paper. Lincoln is providing administrative support. |
Impact | None yet. |
Start Year | 2021 |
Description | RAS-user training needs white paper |
Organisation | University of Lincoln |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | I am the lead of a working group to produce a white paper to identify future training needs to apply RAS technology to the UK agri-food sector. |
Collaborator Contribution | All other partners except Lincoln are contributing authors to the white paper. Lincoln is providing administrative support. |
Impact | None yet. |
Start Year | 2021 |
Description | RAS-user training needs white paper |
Organisation | University of Reading |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | I am the lead of a working group to produce a white paper to identify future training needs to apply RAS technology to the UK agri-food sector. |
Collaborator Contribution | All other partners except Lincoln are contributing authors to the white paper. Lincoln is providing administrative support. |
Impact | None yet. |
Start Year | 2021 |
Description | RAS-user training needs white paper |
Organisation | University of Sheffield |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | I am the lead of a working group to produce a white paper to identify future training needs to apply RAS technology to the UK agri-food sector. |
Collaborator Contribution | All other partners except Lincoln are contributing authors to the white paper. Lincoln is providing administrative support. |
Impact | None yet. |
Start Year | 2021 |
Title | A Library for Constraint Consistent Learning |
Description | The Constraint Consistent Learning (CCL) library implements a family of data-driven methods that are capable of (i) learning state-independent and -dependent constraints, (ii) decomposing the behaviour of redundant systems into task and null-space parts, and (iii) uncovering the underlying null space control policy. It is a powerful tool to analyse and decompose many everyday tasks, such as wiping, reaching and drawing. |
Type Of Technology | Software |
Year Produced | 2018 |
Open Source License? | Yes |
Impact | Practically, it presents the first unified application programmers interface (API) implementing and enabling use of these algorithms in a common framework. Academically, the methods implemented has already used and published papers based on it for human behaviour analysis. Furthermore, this unified API has summarised and implemented the methods which have been published for the past few years and added C functions, making it more useful and extendable for user. At the same time, it leads to a submission of paper `A Library for Constraint Consistent Learning' for gaining more academic interest and influence. |
URL | https://nms.kcl.ac.uk/rll/CCL_doc/index.html |
Description | Exhibition at Science Museum |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Other audiences |
Results and Impact | We plan to exhibit outcomes of the project at the Science Museum at the end of this month. |
Year(s) Of Engagement Activity | 2018 |
Description | Guest Column in NFU Horticulture Magazine |
Form Of Engagement Activity | A magazine, newsletter or online publication |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | I wrote a guest column for the NFU Horticulture Magazine, detailing the current state of the art of machine learning and robotics for horticultural production. This is to appear in print in the Spring issue. |
Year(s) Of Engagement Activity | 2018 |
Description | Invited talk at AHDB SmartHort 2019 Conference |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 50 people attended the presentation, including policy makers, academic and industrial participants from across the UK. |
Year(s) Of Engagement Activity | 2018 |
Description | Invited talk at Cambridge Agrifood Workshop |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Third sector organisations |
Results and Impact | I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 50 people attended the presentation, including academic and industrial participants from across the UK. |
Year(s) Of Engagement Activity | 2018 |
Description | Invited talk at Cambridge Agritech East Conference |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 100 people attended the presentation, including academic and industrial participants from across the UK. |
Year(s) Of Engagement Activity | 2018 |
Description | Invited talk at Horticulture 4.0 Conference |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 100 people attended the presentation, including policy makers, academic and industrial participants from across the UK. |
Year(s) Of Engagement Activity | 2018 |
URL | http://wsga.co.uk/wsga-horticulture-4-0-presentations |
Description | Invited talk at ICRA 2017 Workshop on Robot Learning |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | The PI gave an invited talk at the ICRA 2017 Workshop on Robot Learning for Manipulation in Singapore. The audience consisted of leading robotics researchers from academia and industry, and provoked discussion and questions. |
Year(s) Of Engagement Activity | 2017 |
Description | Invited talk at IROS 2018 Workshop on Robot Impeance Skills: Control and Learning |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | I gave an invited talk at the IROS 2018 Workshop on Variable Impedance Skills: Control and Learning. The talk reported the outcomes of the project, and sparked debate around its findings. Around 50 people attended the presentation, and over 2000 attended the conference. |
Year(s) Of Engagement Activity | 2018 |
URL | http://iros18wsimpedance.weebly.com/ |
Description | Invited talk at IROS 2021 Conference |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | I gave an invited talk at this workshop. The talk reported the outcomes of the project, and sparked debate around its findings. Around 50 people attended the presentation, including academic and industrial participants from across the world. |
Year(s) Of Engagement Activity | 2021 |
Description | Keynote talk at Agritech East AI in Agriculture |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Industry/Business |
Results and Impact | Approximately 100 people from business, industry, and technology development registered for the event, in which I was one of 6 invited speakers. The audience engaged in a lively discussion of the impact of AI and robotics in the agricultural sector. Early feedback on the event from the audience was extremely positive. |
Year(s) Of Engagement Activity | 2018 |
URL | https://www.agritech-east.co.uk/events/smart-farming-the-aim-of-machine-to-machine-learning-in-agric... |
Description | Keynote talk at NFU Kent Branch AGM |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Industry/Business |
Results and Impact | I gave a keynote talk to the AGM of the NFU Kent branch about current opportunities for innovation in automation for farmers. The talk sparked considerable interest from the audience, including a discussion of the impacts of Brexit and how robotic/machine learning technology could address them. |
Year(s) Of Engagement Activity | 2017 |
Description | Talk at the Innovate UK Automation and robotics in the AgriFood sector |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Industry/Business |
Results and Impact | The PI presented gave a 'pitch talk' at this event. It resulted in greater awareness of the project work and stimulated follow up enquiries from interested parties, including Innovate UK representatives, and members of the UK Forestry Commission. |
Year(s) Of Engagement Activity | 2017 |
URL | https://www.eventbrite.co.uk/e/automation-and-robotics-in-the-agrifood-sector-expression-of-interest... |