FREEpHRI: Flexible, Robust and Efficient physical Human-robot Interaction with iterative learning and self-triggered role adaption
Lead Research Organisation:
University of Leeds
Department Name: Electronic and Electrical Engineering
Abstract
Globally, the number of robots in use in 2020 is over 2.25 million, which will multiply even faster in the next 10 years, reaching 20 million by 2030. The fast adoption of robots has contributed 10% of worldwide total GDP growth in the last five years. The technological advancements are bringing robots to humans' daily lives, and they are no longer working in an isolated environment, but sharing the same workspace and physically interacting with humans, e.g., rehabilitation robots, tele-operation robots and collaborative robots. The physical coupling between humans and robots, often termed as physical human-robot interaction (pHRI), facilitates new human performance capabilities and creates opportunities to explore the task sharing and the control between humans and robots. To maximise the benefit of human-robot system during joint tasks, the robot needs to understand what the human is trying to do, and intelligently adjusts its behaviour according to the performance of the human and the requirements of tasks, which requires novel tools to model the human behaviours and innovate strategies to modulate the control of the robot.
The ambition of this fellowship is to enable robots to real-time estimate human behaviours, intelligently detect the changes of human behaviours, automatically adjust the relationship between the human and the robot (from collaborative to competitive), and provide natural interaction behaviours even when the robot dynamics are partly unknown. I will pursue this goal by: 1) developing a flexible and robust pHRI control strategy. The control strategy uses a two-player differential game to model human-robot interaction behaviours, and learning techniques to compensate the effects of unknown dynamics and external disturbances. A cost function implying motor capability will be assigned to the human partner, and the robot will adjust its role (collaborator or competitor) according to the real-time estimation of the human cost function. 2) introducing an efficient self-triggered role adaption mechanism. The triggering mechanism uses the performance of the human-robot system and the estimated human behaviour to detect the role changes of the human, and triggers the robot to change its role when necessary; 3) evaluating the reliability and functionality of the proposed techniques through an exemplar application in physical robot-assisted rehabilitation. The proposed techniques will be used to achieve typical training strategies (e.g., passive, assist-as-needed, challenge-based) initially in laboratory settings, and then in the Leeds Teaching Hospital rehabilitation service.
This fellowship targets at two fundamental issues in pRHI: (1) how to efficiently update the robot's control strategy to ensure desired interactions; and (2) how to deal with uncertainties in the human-robot system. The technologies developed in this fellowship will provide a general framework for designing an interactive robot control system, which has a large group of applications in both healthcare and manufacturing. The fellowship objectives and milestones will be delivered collaboratively with partners from the University of Leeds, the University of the West of England Bristol, the University of Manchester, Leeds Teaching Hospitals NHS Trust, Devices for Dignity, YIRUIDE Medical and DIH/Hocoma.
The ambition of this fellowship is to enable robots to real-time estimate human behaviours, intelligently detect the changes of human behaviours, automatically adjust the relationship between the human and the robot (from collaborative to competitive), and provide natural interaction behaviours even when the robot dynamics are partly unknown. I will pursue this goal by: 1) developing a flexible and robust pHRI control strategy. The control strategy uses a two-player differential game to model human-robot interaction behaviours, and learning techniques to compensate the effects of unknown dynamics and external disturbances. A cost function implying motor capability will be assigned to the human partner, and the robot will adjust its role (collaborator or competitor) according to the real-time estimation of the human cost function. 2) introducing an efficient self-triggered role adaption mechanism. The triggering mechanism uses the performance of the human-robot system and the estimated human behaviour to detect the role changes of the human, and triggers the robot to change its role when necessary; 3) evaluating the reliability and functionality of the proposed techniques through an exemplar application in physical robot-assisted rehabilitation. The proposed techniques will be used to achieve typical training strategies (e.g., passive, assist-as-needed, challenge-based) initially in laboratory settings, and then in the Leeds Teaching Hospital rehabilitation service.
This fellowship targets at two fundamental issues in pRHI: (1) how to efficiently update the robot's control strategy to ensure desired interactions; and (2) how to deal with uncertainties in the human-robot system. The technologies developed in this fellowship will provide a general framework for designing an interactive robot control system, which has a large group of applications in both healthcare and manufacturing. The fellowship objectives and milestones will be delivered collaboratively with partners from the University of Leeds, the University of the West of England Bristol, the University of Manchester, Leeds Teaching Hospitals NHS Trust, Devices for Dignity, YIRUIDE Medical and DIH/Hocoma.
Publications
Jin L
(2022)
Flexible unimodal strain sensors for human motion detection and differentiation
in npj Flexible Electronics
Qian K
(2023)
Robust Iterative Learning Control for Pneumatic Muscle With Uncertainties and State Constraints
in IEEE Transactions on Industrial Electronics
Qian K
(2023)
Data-Driven Adaptive Iterative Learning Control of a Compliant Rehabilitation Robot for Repetitive Ankle Training
in IEEE Robotics and Automation Letters
Wang C
(2023)
A Novel Series Elastic Actuator with Variable Stiffness
Zhang J
(2022)
Boosting Personalised Musculoskeletal Modelling with Physics-informed Knowledge Transfer
in IEEE Transactions on Instrumentation and Measurement
Zhang J
(2023)
Physics-Informed Deep Learning for Musculoskeletal Modeling: Predicting Muscle Forces and Joint Kinematics From Surface EMG
in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Zhao Y
(2023)
An EMG-driven musculoskeletal model for estimation of wrist kinematics using mirrored bilateral movement
in Biomedical Signal Processing and Control
Description | Device for Dignity MedTech Co-operative |
Organisation | Device for Dignity MedTech Co-operative |
Country | United Kingdom |
Sector | Charity/Non Profit |
PI Contribution | We collaborate with the steering group organized by Devices for Dignity to assess the regulatory requirements of the rehabilitation technology we developed. |
Collaborator Contribution | Devices for Dignity (D4D) is a UK organization focusing on developing medical devices. They have extensive experience in navigating the complex regulatory pathways involved in developing and commercializing medical devices, including obtaining regulatory approvals and meeting safety and quality standards. By collaborating with D4D, we will be able to leverage their expertise in regulatory issues to ensure that any technology developed through this collaboration meets the necessary regulatory requirements for safety. In addition to their expertise in regulatory issues, D4D also has a strong network of industry partners and stakeholders, including clinical experts and patient groups. This will provide the project with access to a broader range of perspectives and insights into the specific needs We are now working together to develop a pilot study protocol and they are also providing specialists to make sure the developed system is compliant with assessment and trial requirements. |
Impact | A steering group including both stroke survivors, physiotherapists and multi-disciplinary researchers have been created. |
Start Year | 2022 |
Description | Textile sensing for human monitoring, University of Manchester |
Organisation | University of Manchester |
Department | School of Materials Manchester |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | We work together with the research team in materials to develop soft sensing technologies that can be used to monitor the daily activities of users. During the collaboration, we contribute expertise in machine learning and signal processing. |
Collaborator Contribution | The teams in Manchester contribute their expertise in manufacturing textile sensors and also the facilities to test the usability of piezoelectric materials. |
Impact | A soft wrist brace prototype that can capture two degrees-of-freedom wrist movement has been developed. A soft gloves-based air writing interface that can independently measure bending, shearing and twisting movements of fingers has been developed. |
Start Year | 2022 |
Description | Interview for engineering magazine |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Media (as a channel to the public) |
Results and Impact | I was interviewed by a magazine named "The Engineering". I talked about the challenges and opportunities in the research area of human-robot interaction and introduce this project. The Interview improve the visibility of the project and spanked industry collaboration. |
Year(s) Of Engagement Activity | 2022 |
Description | Kick off meeting |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Schools |
Results and Impact | 6 people from the University of Leeds, Leed teaching Hospital, and the University of Manchester have been invited. The different research tasks have been discussed during the meeting. |
Year(s) Of Engagement Activity | 2022 |
Description | Seminar for the manipulation of soft objects |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Professional Practitioners |
Results and Impact | An academic from the University of York is invited to give a talk on robot manipulation. 13 people attended the seminar, which sparked questions and discussions on the challenges in assistive robotics |
Year(s) Of Engagement Activity | 2023 |
Description | consumer research advisory meeting |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Patients, carers and/or patient groups |
Results and Impact | 10 people including 2 physiotherapists and 3 stroke survivors attended the meeting. The user-required functions for knee and ankle rehabilitation have been collected. |
Year(s) Of Engagement Activity | 2022 |