Human Centred Robotics for Next-generation Flexible Manufacturing

Lead Research Organisation: University of the West of England
Department Name: Faculty of Environment and Technology


The proposed research falls within the field of Robotics and Artificial Intelligence Systems; an area that has enormous potential to provide greater levels of throughput, repeatability, productivity and the introduction of more complex tasks to be carried out in a robot collaboration environment within the UK Manufacturing Sector. The introduction of robots in the production process has undeniable benefits: each robot can replace several human operators, performing repetitive tasks. However, reprogramming and operating robots for production purposes can pose significant challenges for businesses, which could be potential barriers to automation and corporate expansion. For example, each time a robot undertakes a new task it must be reprogrammed. Reprogramming multiple robots can take up to several months and involve the services of a specialist robot programmer. This results in high running costs and suboptimal productivity creating a barrier to the adoption of the technology within the wider manufacturing sector. Similarly, colocation of robots with humans and other machines requires higher levels of cognition, perception and autonomy to assimilate with different user experiences and individual preferences, without interfering with operational schedules.
The research is aimed to address these issues and ensure that robotics are more widely adopted, with the intention of producing software and hardware toolkits that once commercially available will enhance efficiency, reduce costs and facilitate corporate expansion. To achieve this two key approaches will be investigated:
Demonstrable (WS1) - which will develop a new skill transfer interface to teach the robots through body posture, hand gesture and voice commands. It will include (a) a comprehensive human motor skills capture system based on fusion of both physical signals including motion and force and physiological signals of muscle internal activities; (b) a user friendly intuitive teaching interface integrating the skill capture system with mixed reality and voice control, (c) a holistic approach to capture and transfer manipulative skills of arm and hand as coordinated system; and (d) skill generalization mechanism for robots to perform new tasks without additional demonstration.
Collaborative (WS2) - to deliver an intelligent control system for cobots to achieve optimal human-robot cooperation, so that a human's flexibility and creativity can be efficiently integrated with a robot's accuracy and repeatability. This involves (a) a reliable and efficient gesture/posture/voice based communication channel for human co-workers to command the robots easily; (b) improved cobot cooperation skills by embedding human intent perception into robot's control actions, (c) learning strategies to capture individual human co-worker's motion/force pattern for a cobot to provide customized support, and (d) validation in commercially available cobots such as KUKA iiwa and UR5-CBR, together with 3-finger Robotiq gripper and 5-finger Wessling Robotic Hand.
The collective outcome will be innovative, user friendly, technology that permits existing members of the workforce to train robots to undertake new tasks - reducing the cost of outsourcing to one fifth and enabling reprogramming to be completed at a rate that is approximately ten times faster than previous methods. This will have notable economic benefits for distributors of the software and companies as end-users within the manufacturing sector. Not only will existing production lines be more cost effective and profitable but new markets (i.e. customisation and the delivery of new products) will be accessible because of the ability to swiftly reprogram robots for new tasks. Therefore, corporate expansion will be facilitated, via the adoption of digital technology (a priority area for the UK Government), ultimately bolstering the UK Economy.

Planned Impact

Digital technology, especially Robotics and Artificial Intelligence Systems, can orchestrate a transformational effect on industry. Robots are able to provide manufacturers with a plethora of benefits; guaranteeing consistency in repeated tasks, high throughput rates, operating in environments and conditions that may be considered unsafe for humans, performing with accuracy, minus the necessity for breaks, holidays and compliance with health and safety regulations in the workplace.

However, automation can be both costly and disruptive. The initial outlay for a robotic unit is further exacerbated by the cost of reprogramming and operational challenges. An industrial robot will require specialist programming and integration for every new task (for instance the control system of an industrial robotic arm will take >200 hours to reprogram) in order to meet the emergent customisation and flexibility of production requirements. This is usually outsourced or requires the recruitment of a full-time specialist which is both time consuming and expensive (particularly for SMEs). Furthermore, employees are required to adapt to the new technology and difficulties with user experiences notably underpin costly delays in production.

This research, undertaken as part of the Fellowship, will target these challenges, resulting in commercially available software that will lead to significant economic benefits for distributors of the software and, on a wider scale, financial sustainability and expansion opportunities for the companies within the Manufacturing Sector. Moreover, long-term, it will facilitate the UK's economic development through the uptake and application of transformational technology.

Commercial transformation of the research outputs and the knowledge gained through further industrial participation will follow the IP protection of commercially relevant results. This is expected to yield substantive revenue for the prospective co-developers . The software will be sold to end-users, organisations within the manufacturing domain, generating wealth creation through the availability of a new product.

Organisations within the Manufacturing Sector and their employees will benefit from reduced costs, increased productivity and by gaining access to new markets through customisation and flexibility of production. As opposed to outsourcing, the software and hardware toolkits will enable existing employees to reprogram the robots to undertake new tasks. This will not only be cost effective but will motivate potentially upskill members of the workforce. Additionally, the new technology will allow robotic reprogramming to be accomplished at a markedly faster rate, approximately ten times faster than existing methods, avoiding costly delays in production and potentially facilitating increased revenue via entry into new markets (i.e the production of new products, customisation). Notably, this would previously have been challenging for many SMEs, given the amount of time and expense to teach robots on the production line to undertake new tasks. Finally, the ability to customise individual user experiences will ultimately increase efficiency.

The UK economy can be strengthened from expansion in the manufacturing sector with the ability to produce ever-more customised products, reversing the cost advantage of low-wage economies, incentivizing companies to re-shore activities to the UK, relocating closer to their customer base. This has already been the focus of key industrial nations in the EU, China, Japan and the USA to bolster their economies. The research offers a pathway for organisations within the Manufacturing Sector, who have already automated, to increase profits and expand production. It will also serve as an incentive for organisations who have yet to purchase robotics, by mitigating concerns about additional costs and maintenance and highlighting a route to increased productivity.


10 25 50

publication icon
Fan Y (2019) Neural adaptive global stability control for robot manipulators with time-varying output constraints in International Journal of Robust and Nonlinear Control

publication icon
He W (2020) Disturbance Observer-Based Neural Network Control of Cooperative Multiple Manipulators With Input Saturation. in IEEE transactions on neural networks and learning systems

publication icon
Luo J (2019) A Task Learning Mechanism for the Telerobots in International Journal of Humanoid Robotics

publication icon
Luo J (2020) A Teleoperation Framework for Mobile Robots Based on Shared Control in IEEE Robotics and Automation Letters

publication icon
Na J (2019) Unknown Dynamics Estimator-Based Output-Feedback Control for Nonlinear Pure-Feedback Systems in IEEE Transactions on Systems, Man, and Cybernetics: Systems

publication icon
Pan Y (2019) Composite learning adaptive backstepping control using neural networks with compact supports in International Journal of Adaptive Control and Signal Processing

publication icon
Peng G (2020) Force Sensorless Admittance Control With Neural Learning for Robots With Actuator Saturation in IEEE Transactions on Industrial Electronics

Related Projects

Project Reference Relationship Related To Start End Award Value
EP/S001913/1 29/06/2018 16/12/2018 £544,518
EP/S001913/2 Transfer EP/S001913/1 17/12/2018 28/06/2021 £492,351
Description -We have developed neural learning enhanced variable admittance controllers for human-robot physical collaboration, such that human effort can be reduced while the task performance being enhanced.
-We have established a robot learning framework based on adaptive admittance control and have built generalizable motion models
-We studied cooperation of multiple manipulators for a common task, and exploited disturbance observer technique to deal with input saturation. We also investigated admittance based multi-robot cooperation in the presence of output constraints.
-We studied force observer and have successfully applied it for admittance control design for robot manipulator of dynamics uncertainties, which are compensated for by neural learning approach. We also considered the problem of actuator saturation, which usually happen such that robot may not be able to provide enough large torques for actuation.
-We investigated random forest-based image processing technique to facilitate robot grasping based on vision guidance, and designed an adaptive method of shape approximation using Gaussian mixture models for robot grasping.
-We proposed a new force estimation method using muscle electromyography and developed a multimodal sensory data encoding approach for robotic skill learning.
-We have also investigated deep neural network techniques, and have successfully applied it on anthropomorphic manipulators for human-like redundancy optimization, as well as tool identification and calibration for bilateral teleoperation.
Exploitation Route Multimodal skill learning techniques for robot have been further developed in tihs project. Learning human skills will be very useful for robotic scanning of non-standard objects in the non-destructive testing (NDT), and collaboration with TWI is ongoing to investiage human robot collaborative scanning. The human robot collaboratation techniques developed have also been employed and tested in robotic surgery experiment, and will be interesting to researchers in surgeric robotics.
Sectors Digital/Communication/Information Technologies (including Software),Healthcare,Manufacturing, including Industrial Biotechology

Description The fellow sponsored by this grant recently took the co-chair position of the Technical Committee on Collaborative Automation for Flexible Manufacturing (CAFM) of the IEEE Robotics and Automation Society, the organizer of the world's largest robotics conferences ICRA (International Conference on Robotics and Automation) and IROS (IEEE/RSJ International Conference on Intelligent Robots and Systems). The fellow has also been appointed to the Associate Editor of these two major robotics conferences, in addition to his appointment of Associate Editorship of IEEE Transactions on Automation Science and Engineering, and of IEEE Transactions on System, Man and Cybernetics: Systems, in 2020 and 2019, respectively. These enables the fellow to influence widely to academic, industrial and general public communities. The research outcomes of this fellowship have been well disseminated through not only publications but also the fellow's active involvement of conference organizations. For example, the fellow was the program co-chair of the 19th UK Workshop on Computational Intelligence, and program chair of the 26th IEEE International Conference on Automation and Computing to be held in Portsmouth this September. These conferences create opportunities for researchers to exchange ideas with industrial participants and will promote public awareness of the importance and robotics and AI.
Sector Digital/Communication/Information Technologies (including Software),Healthcare,Manufacturing, including Industrial Biotechology
Impact Types Cultural,Societal,Economic

Title Optimization of Human-Robot Collaboration based on Adaptive Variable Admittance Control 
Description We developed a new method to optimize physical human robot interaction based on variable admittance control. By using measured electromyography to esimate human muscle activation, we adapt robot controller to reduce human effort and improve control performance simutaneously. 
Type Of Material Model of mechanisms or symptoms - human 
Year Produced 2020 
Provided To Others? Yes  
Impact This method can be very promising to improve efficiency of physical human robot collaboration. 
Description TWI Technology Centre (Wales) 
Organisation TWI ltd
Country United Kingdom 
Sector Private 
PI Contribution Collaboration on human robot collaborative NDT process for non-standard objects to be scanned
Collaborator Contribution Informing us the experiences in robotised NDT Providing advice on NDT relatd technologies Promoting the joint reserach outcomes
Impact The collaboration is ongoing and is multi-disciplinary, involving control engineering, machine learning and computer vision.
Start Year 2019
Description ? International Organizing Committee Chair, The 12th International Conference on Intelligent Robotics and Applications (ICIRA), August 8-11, 2019 Shenyang, China. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I was involved in the organization of the International Organizing Committee Chair of the 12th International Conference on Intelligent Robotics and Applications (ICIRA), August 8-11, 2019, at Shenyang, China.
Year(s) Of Engagement Activity 2019
Description Program co-chair of the 19th UK Workshop on Computational Intelligence, September 4-6, 2019, Portsmouth, UK 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact As the program co-chair, I jointly organized the 19th UK Workshop on Computational Intelligence, September 4-6, 2019, Portsmouth, UK, and co-edited a book titled Advances in Computational Intelligence Systems: Contributions Presented publised at Springer International Publishing. ISBN: 978-3-030-29933-0; DOI: 10.1007/978-3-030-29933-0
Year(s) Of Engagement Activity 2019
Description Review panel member of EPSRC HT Investigator-led Panel Meeting, 31 Jan 2019 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Supporters
Results and Impact I was invited to be a panel member of EPSRC HT Investigator-led Panel Meeting, 31 Jan 2019
Year(s) Of Engagement Activity 2019