Versatile and Dexterous Adaptive Control of Dual-Arm Robots for Personalized Assistance

Lead Research Organisation: Plymouth University
Department Name: Sch of Computing & Mathematics

Abstract

In the near future, robots are expected to co-habit with human beings and work closely with us in the fields of manufacturing and medicine as well as in other aspects of our daily lives. Unfortunately, most of the current robot control technologies are designed for industrial robots which operate behind safeguarding and for predefined tasks, and thus are not able to cope with the varying tasks in unknown dynamic environments. This project will develop human-like adaptive control techniques to provide versatile and dexterous assistance to humans . In particular, the proposed new control techniques will enable robots to offer personalized assistance to different human users according to their motion patterns. For example, when the robot is interacting with an elderly person, it will automatically slow down; while it would speed up when working with a young man. To achieve these goals, this project will perform physiological studies on human control in the following two aspects: i) apply advanced control theory to analyze bimanually coordinated control strategies of human-beings in various scenarios; ii) investigate how human users develop efficient cooperation with a robot from a machine learning point of view .

The establishment of human control models can help us in at least two aspects: developing human-like adaptive controllers with high versatility and dexterity; and personalizing robot control to suit a specific human user. Based on this understanding on human bimanual control and human-human cooperation, this project will: (1) enable the robot to cope with different objects by bimanual haptic exploration; (2) enhance the robustness of robot control in interacting with different human users and in handling various objects; (3) improve robot manipulative dexterity at human-robot interaction scenarios and under geometric constraints/limitations caused by hardware and safety issues; (4) detect motion intention of human user and enhance dexterity with robot joint compliance; (5) introduce a measurement of user experience to quantify how human users are satisfied with the assistance offered by robots; and (6) enable the robot to identify and adapt to motor behavior patterns of a specific human user.

In the short term, this project will provide new control technologies for robotics researchers and builders to achieve better manipulation performance in terms of dexterity, versatility, and personalization. In the long run, it will benefit socially and economically to the UK by helping to boost the personal robot industry. It will further make contributions to UK by advancing its technology in the field of Robotics and Autonomous Systems, which is identified by the UK government as one of eight great British Technologies propelling the UK towards future growth.

Planned Impact

This project will actively engage with a number of leading robot experts in the UK and will access their wider networks in both the academic and industrial communities. The proposed project has great potential in generating economic and social impacts in the following areas:

KNOWLEDGE:
The project proposes to create new knowledge on robot control and assistance, i.e., improve robot control performance in terms of dexterity and versatility, and develop a "from human, for human" control design approach to optimize and to personalize the assistance robots can provide according to user experience.

PEOPLE:
The proposed project will have a major impact on the PI who will carry out the research. The project will consolidate and extend the research skills and experiences the PI built in his previous Marie Curie Fellowship project. This project will be extremely useful for the PI as an early career academic to be equipped with a robot experimental platform, which can help him to build up his future research career. Through collaborations and academic visits, the project partners will also be able to gain new skills and knowledge by applying their experience and techniques in this new research direction. The requested robot equipment will also benefits students in PI's research team, providing them state-of-art experiment platform. In addition, this project will also create a few undergraduate-level final year projects and MSc final projects for the BEng/MSc programme delivered by Plymouth University. These projects will bring fresh new ideas into education, and will benefit students by exposing them to the very forefront of robotics research; preparing them for possible future careers in academic or industrial robotics.

ECONOMY:
Robotics and automation is one of the 'eight great technologies' identified by the UK government to propel the UK to future growth. A 2010 report from the Japan Robotics Association valued the robotics market at $24.9bn, with a forecasting value of $66.4bn by 2025 (over 50% of this growth is expected to come from personal robotics). Moreover, the International Robotics Federation forecasts sales of 15.6 million service and edutainment robot units between 2012-2015, for a market value of $5.9bn. Sales of assistance robots for the elderly and disabled, while at current stage relatively small at 4,600 units, are expected to "increase substantially within the next 20 years" (IFR, 2012). This project is of great importance in developing future robot control techniques, and will be particularly useful in scenarios where robots assist or cooperate with humans. The success of this project might provide significant commercial opportunities, and could benefit relevant businesses by delivering technologies developed from this project.

SOCIETY:
Versatile and dexterous robot control will help the robot to provide personalized assistance and has great potential in making benefits to society through various applications in which robots could support humans, e.g., manufacturing, construction, assembling, medical operation, etc. The improved control efficiency and optimized assistance would directly benefit human users.
In addition, extensive studies will be carried out in the project to investigate human motor control in order to analyze and understand human motions. This would also support the development of power assistance technologies for the disabled, and rehabilitation technologies to restore motor function for neuropathy patients.

The above mentioned impacts on society and economy are potential and long-term, and hence will not be fully delivered during the short duration of the proposed project, which only aims to lay the foundations for such impacts in the future.
 
Description 1. Development of both kinematics and dynamics model of the Baxter dual-arm robot with experimental test and verification.
2. Stiffness extraction from human operator's muscle electromyography (EMG), with experimental test and verification.
3. Development of a novel haptic teaching interface for transfer of human variable stiffness skills to a robot based on EMG signals.
4. Development of a novel human to human via a robot teaching interface for motor skills transfer from one human subject to the other.
5.Optimal parameter tuning for adaptive control of of robot manipulator using fuzzy logic system.
6. Development and experimental test of personalized human robot shared control techniques using haptic device Phantom Omni.
7. Development and experimental test of automatic obstacle avoidance enhanced dexterous robot manipulation.
8. Development and experimental test of automatic obstacle avoidance enhanced human robot shared control method.
9. Development and experimental test of robot self body perception based on visual Kinect sensor.
10. Development and experimental test of neural network based versatile robot manipulation.
11. Development and experimental test of sterero visual servo control of dual arm robot.
12. Development and experimental test of user experience driven interface design for human robot interaction.
13. Theoretical development of general Newton-Euler modeling process of robot manipulator.
14. Theoretical development of neural enhanced adaptive optimal control for robot manipulator.
Exploitation Route The versatile and dexterous robot manipulation control methods developed in this project would be very beneficial to industrial partners working on robot arms based automation technologies. These algorithms have been reported in our publications, and are easy to access. These algorithms can be implemented with slight modification to their robot arms.
Sectors Aerospace, Defence and Marine,Creative Economy,Education,Electronics,Healthcare,Manufacturing, including Industrial Biotechology

 
Description 1. The Baxter robot model developed has been adopted in the widely used MATLAB robot simulation toolkit by Peter Corke (http://petercorke.com/Robotics_Toolbox.html) 2. Partial research outputs informed to the public through presentation at Science Cafe sponsored by the British Science Association.
First Year Of Impact 2014
Sector Aerospace, Defence and Marine,Creative Economy,Education,Electronics
Impact Types Societal,Economic

 
Title Human Robot Coupling Device 
Description we have developed a mechatronic human-robot coupling interface for kinaesthetic guidance based human robot teaching interface, by integration of a 6-axis force sensor, a coupling mechanism and wearable electromyography (EMG) arrays. Experimental results indicated that the coupling design nearly caused no disturbance to human upper limb stiffness estimation. Moreover, human arm kinematics and force/torque in task space can be transferred to the robot in a natural way with guaranteed safety. This coupling interface design can be easily extended to other robot platforms and has great potentials in application of human robot skill transfer. 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact we have developed a mechatronic human-robot coupling interface for kinaesthetic guidance based human robot teaching interface, by integration of a 6-axis force sensor, a coupling mechanism and wearable electromyography (EMG) arrays. Experimental results indicated that the coupling design nearly caused no disturbance to human upper limb stiffness estimation. Moreover, human arm kinematics and force/torque in task space can be transferred to the robot in a natural way with guaranteed safety. This coupling interface design can be easily extended to other robot platforms and has great potentials in application of human robot skill transfer. 
 
Title A Discrete Time Algorithm For Stiffness Extraction From sEMG 
Description We have developed a new discrete-time algorithm of stiffness extraction from muscle surface electromyography (sEMG) collected from human operator's arms and have applied it for antidisturbance control in robot teleoperation. The variation of arm stiffness is estimated from sEMG signals and transferred to a telerobot under variable impedance control to imitate human motor control behaviours, particularly for disturbance attenuation. In comparison to the estimation of stiffness from sEMG, the proposed algorithm is able to reduce the nonlinear residual error effect and to enhance robustness and to simplify stiffness calibration. In order to extract a smoothing stiffness enveloping from sEMG signals, two enveloping methods are employed in this paper, namely, fast linear enveloping based on low pass filtering and moving average and amplitude monocomponent and frequency modulating (AM-FM) method. Both methods have been incorporated into the proposed stiffness variance estimation algorithm and are extensively tested. The test results show that stiffness variation extraction based on the two methods is sensitive and robust to attenuation disturbance. 
Type Of Material Computer model/algorithm 
Year Produced 2016 
Provided To Others? Yes  
Impact Promotion of research reputation 
 
Title Adaptive Newton Euler Modelling 
Description Adaptive model identification method based on Devanit-Hartenberg convention using recursive Newton-Euler (RNE) formula has been developed. To obtain sufficient excitation for modeling of the robot, the particle swarm optimization method has been employed to optimize the trajectory of each joint, such that satisfied parameter estimation can be obtained. 
Type Of Material Computer model/algorithm 
Year Produced 2016 
Provided To Others? Yes  
Impact It provides a new method to perform modelling and control at the same time for uncertain robots. 
 
Title kinematics model of Baxter robot 
Description A kinematics model of Baxter robot has been developed to enable simulation of Baxter robot in MATLAB. This model has been adopted in the popular MATLAB based robot simulation toolkit by Peter Corke. 
Type Of Material Computer model/algorithm 
Year Produced 2014 
Provided To Others? Yes  
Impact This model has been reported in our recent publication and provides researcher on Baxter robot a verified kinematics model. 
 
Description collaboration with Imperial College 
Organisation Imperial College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Collaboration on algorithm design and experimental test.
Collaborator Contribution Data analysis and provision of models.
Impact 1 joint journal publications and 3 joint conference publications. Co-organization of one journal special issue on "Brain-computing-based and Bio-mechatronics Systems, Control, and Applications" on IEEE Transaction on System, Man Cybernetics: System Joint publication of 1 editorial letter: An Overview of Biomedical Robotics and Bio-Mechatronics Systems and Applications, IEEE Transactions on Systems, Man and Cybernetics: Systems, vol. 46, no. 7, July 2016
Start Year 2014
 
Description collaboration with Portsmouth University 
Organisation University of Portsmouth
Country United Kingdom 
Sector Academic/University 
PI Contribution Collaboration on algorithm design and experimental test.
Collaborator Contribution Collaboration on algorithm design and informing experience of experiment.
Impact 1 joint journal publication. Joint organization of journal special issue on "Innovative Design of Control System and User Interface for Teleoperation Robots" on Advances in Mechanical Engineering.
Start Year 2014
 
Description collaboration with University of Bristol 
Organisation University of Bristol
Country United Kingdom 
Sector Academic/University 
PI Contribution Discussion of algorithm design, experiment and organization of conference session/journal special issue, e.g., "Intelligent Control in Discrete-Time for Autonomous Systems" on Discrete Dynamics in Nature and Society.
Collaborator Contribution Discussion of algorithm design, experiment and organization of conference session/journal special issue, e.g., "Intelligent Control in Discrete-Time for Autonomous Systems" on Discrete Dynamics in Nature and Society.
Impact Joint organization of journal special issue "Intelligent Control in Discrete-Time for Autonomous Systems" on Discrete Dynamics in Nature and Society Joint publication of an editorial letter Joint organization of conference special session at IEEE International Conference on Advanced Robotics and Mechatronics (ARM), Macau, China, 18-20 Aug, 2016 Joint publication of 3 conference papers
Start Year 2014
 
Description Conference Session Organization 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Organization and Chair of sessions "Nonlinear System I" and "Intelligent Control III" at the IEEE International Conference on Information and Automation (ICIA), 8-10 August, Lijiang, Yunnan, China, 2015
Year(s) Of Engagement Activity 2015
 
Description Conference Session Organization 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Organization and Chair of sessions "Intelligent Robot Systems" and "Human Robot Interacton I" the 8th International Conference on Intelligent Robotics and Applications (ICIRA 2015), Aug 24-27, Portsmouth, UK
Year(s) Of Engagement Activity 2015
 
Description Program co-chair of 2016 IEEE International Conference on Advanced Robotics and Mechatronics 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Co-organization of the IEEE International Conference on Advanced Robotics and Mechatronics (ARM), Macau, 18-20 Aug, 2016. Over 200 paper submission and over 100 participants from more than 12 countries were attracted to the conference, and keynote speakers including 6 IEEE Fellows from Italy, US, China, Canada, Japan and Turkey joined the conference.
Year(s) Of Engagement Activity 2016
 
Description Science Cafe 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact More than 50 public members attended the presentation;Robotics in the Future of our Society; and a panel discussion between policy maker, industrial representative and academics including Wales Deputy Minister for Skills and Technology Julie James. Questionnaires collected showed that the talks were overwhelmingly favorable.
Year(s) Of Engagement Activity 2016
URL http://www.swansea.ac.uk/media-centre/news-archive/2016/sciencecafefuturedebate-roboticsinthefutureo...
 
Description Setup of a new IEEE Technical Committee on Bio-mechatronics and Bio-robotics Systems 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We set up a new IEEE Technical Committee on Bio-mechatronics and Bio-robotics Systems together with other co-chairs including 2 IEEE Fellows and IEEE Transactions Editor-in-Chiefs in 2016. This committee brings together global talents aiming to advance bio-inspired robot design.
Year(s) Of Engagement Activity 2016
URL http://www.ieeesmc.org/technical-activities/systems-science-and-engineering/bio-mechatronics-and-bio...