Human Centred Robotics for Next-generation Flexible Manufacturing

Lead Research Organisation: Swansea University
Department Name: College of Engineering

Abstract

The proposed research falls within the field of Robotics and Artificial Intelligence Systems; an area that has enormous potential to provide greater levels of throughput, repeatability, productivity and the introduction of more complex tasks to be carried out in a robot collaboration environment within the UK Manufacturing Sector. The introduction of robots in the production process has undeniable benefits: each robot can replace several human operators, performing repetitive tasks. However, reprogramming and operating robots for production purposes can pose significant challenges for businesses, which could be potential barriers to automation and corporate expansion. For example, each time a robot undertakes a new task it must be reprogrammed. Reprogramming multiple robots can take up to several months and involve the services of a specialist robot programmer. This results in high running costs and suboptimal productivity creating a barrier to the adoption of the technology within the wider manufacturing sector. Similarly, colocation of robots with humans and other machines requires higher levels of cognition, perception and autonomy to assimilate with different user experiences and individual preferences, without interfering with operational schedules.
The research is aimed to address these issues and ensure that robotics are more widely adopted, with the intention of producing software and hardware toolkits that once commercially available will enhance efficiency, reduce costs and facilitate corporate expansion. To achieve this two key approaches will be investigated:
Demonstrable (WS1) - which will develop a new skill transfer interface to teach the robots through body posture, hand gesture and voice commands. It will include (a) a comprehensive human motor skills capture system based on fusion of both physical signals including motion and force and physiological signals of muscle internal activities; (b) a user friendly intuitive teaching interface integrating the skill capture system with mixed reality and voice control, (c) a holistic approach to capture and transfer manipulative skills of arm and hand as coordinated system; and (d) skill generalization mechanism for robots to perform new tasks without additional demonstration.
Collaborative (WS2) - to deliver an intelligent control system for cobots to achieve optimal human-robot cooperation, so that a human's flexibility and creativity can be efficiently integrated with a robot's accuracy and repeatability. This involves (a) a reliable and efficient gesture/posture/voice based communication channel for human co-workers to command the robots easily; (b) improved cobot cooperation skills by embedding human intent perception into robot's control actions, (c) learning strategies to capture individual human co-worker's motion/force pattern for a cobot to provide customized support, and (d) validation in commercially available cobots such as KUKA iiwa and UR5-CBR, together with 3-finger Robotiq gripper and 5-finger Wessling Robotic Hand.
The collective outcome will be innovative, user friendly, technology that permits existing members of the workforce to train robots to undertake new tasks - reducing the cost of outsourcing to one fifth and enabling reprogramming to be completed at a rate that is approximately ten times faster than previous methods. This will have notable economic benefits for distributors of the software and companies as end-users within the manufacturing sector. Not only will existing production lines be more cost effective and profitable but new markets (i.e. customisation and the delivery of new products) will be accessible because of the ability to swiftly reprogram robots for new tasks. Therefore, corporate expansion will be facilitated, via the adoption of digital technology (a priority area for the UK Government), ultimately bolstering the UK Economy.

Planned Impact

Digital technology, especially Robotics and Artificial Intelligence Systems, can orchestrate a transformational effect on industry. Robots are able to provide manufacturers with a plethora of benefits; guaranteeing consistency in repeated tasks, high throughput rates, operating in environments and conditions that may be considered unsafe for humans, performing with accuracy, minus the necessity for breaks, holidays and compliance with health and safety regulations in the workplace.

However, automation can be both costly and disruptive. The initial outlay for a robotic unit is further exacerbated by the cost of reprogramming and operational challenges. An industrial robot will require specialist programming and integration for every new task (for instance the control system of an industrial robotic arm will take >200 hours to reprogram) in order to meet the emergent customisation and flexibility of production requirements. This is usually outsourced or requires the recruitment of a full-time specialist which is both time consuming and expensive (particularly for SMEs). Furthermore, employees are required to adapt to the new technology and difficulties with user experiences notably underpin costly delays in production.

This research, undertaken as part of the Fellowship, will target these challenges, resulting in commercially available software that will lead to significant economic benefits for distributors of the software and, on a wider scale, financial sustainability and expansion opportunities for the companies within the Manufacturing Sector. Moreover, long-term, it will facilitate the UK's economic development through the uptake and application of transformational technology.

Commercial transformation of the research outputs and the knowledge gained through further industrial participation will follow the IP protection of commercially relevant results. This is expected to yield substantive revenue for the prospective co-developers . The software will be sold to end-users, organisations within the manufacturing domain, generating wealth creation through the availability of a new product.

Organisations within the Manufacturing Sector and their employees will benefit from reduced costs, increased productivity and by gaining access to new markets through customisation and flexibility of production. As opposed to outsourcing, the software and hardware toolkits will enable existing employees to reprogram the robots to undertake new tasks. This will not only be cost effective but will motivate potentially upskill members of the workforce. Additionally, the new technology will allow robotic reprogramming to be accomplished at a markedly faster rate, approximately ten times faster than existing methods, avoiding costly delays in production and potentially facilitating increased revenue via entry into new markets (i.e the production of new products, customisation). Notably, this would previously have been challenging for many SMEs, given the amount of time and expense to teach robots on the production line to undertake new tasks. Finally, the ability to customise individual user experiences will ultimately increase efficiency.

The UK economy can be strengthened from expansion in the manufacturing sector with the ability to produce ever-more customised products, reversing the cost advantage of low-wage economies, incentivizing companies to re-shore activities to the UK, relocating closer to their customer base. This has already been the focus of key industrial nations in the EU, China, Japan and the USA to bolster their economies. The research offers a pathway for organisations within the Manufacturing Sector, who have already automated, to increase profits and expand production. It will also serve as an incentive for organisations who have yet to purchase robotics, by mitigating concerns about additional costs and maintenance and highlighting a route to increased productivity.

Publications

10 25 50
 
Description 1. We developed a novel adaptive control methodology based on the admittance model for multiple manipulators transporting a rigid object cooperatively along a predefined desired trajectory.
2. Based on a hierarchical operational space formulation of a seven-degree-of-freedom redundant robot, we developed an improved human-robot collaborative control scheme.
3. Due to unskilful operation and muscle physiological tremor of the human operator, a virtual fixture method is developed to ensure accuracy of teleoperation and to reduce the operation pressure on the human operator.
4. To save computational memory, we developed an innovative neural network (NN) based robot control using a dimension compressed radial basis function NN for a class of n-degree of freedom (DOF) robot with full-state constraints.
5. The ability of robot grasp planning for novel objects is necessary for a robot to autonomously perform grasps under unknown environments. We consider the task of grasp planning for a parallel gripper to grasp a novel object, given an RGB image and its corresponding depth image taken from a single view. We adopted fuzzy Gaussian mixture models (GMMs) for novel objects' shape approximation. With the obtained GMM, we decompose the object into several ellipses, while each ellipse is corresponding to a grasping rectangle.
Exploitation Route The robot grasping method could potentially be used by Wessling Robotics to develop a software kit, as an accessories of their robot hand product.
Sectors Digital/Communication/Information Technologies (including Software)

Electronics

Manufacturing

including Industrial Biotechology

 
Description The outcomes of the research sponsored by this fellowship has been well disseminated not only to the academic communities but also to the industrial and general public communities, through organzation of workshops, conferences and journal special issues. As a Program Co-chair, I have successfully co-organized the the 11th Fuman Friendly-Robotics Conference, Shenzhen, China, 13-14 Nov, 2018. As General chair, I have successfully organized the 27th IEEE International Conference on Automation and Computing, 1-3 September 2022, Bristol, UK. Partial outcomes of this fellowship project have bee reported in this workshop to international audience of these conferences. The fellow's collaboration with the German company of Wessling Robotics has been strengthened through collaborative research for this project. The object shape approximation algorithm we co-designed for robot grasping has been very successful and is under further test to be potentially embedded into a software kit as an accessory of their project Dexterous Hand.
First Year Of Impact 2018
Sector Digital/Communication/Information Technologies (including Software),Manufacturing, including Industrial Biotechology
Impact Types Cultural

 
Description International Organizing Committee Chair of The 12th International Conference on Intelligent Robotics and Applications (ICIRA) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact As ab International Organizing Committee Chair, I am co-organizing The 12th International Conference on Intelligent Robotics and Applications (ICIRA) to be held in Shenyang, China, on, August 8-11, 2019. This is an international conference with over ten years history and has a large number of audience. My participation will further enhance collaboration between roboticsts in UK and China.
Year(s) Of Engagement Activity 2019
URL https://www.icira2019.org
 
Description Program Co-chair of the Human Friendly-Robotics Conference (HFR) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact As a program co-chair, I co-organized the 11th Fuman Friendly-Robotics Conference, Shenzhen, China, 13-14 Nov, 2018. Participants come from universitties, institutes, industry and the general public, and the workshop served as an platform for international robotists to exchange ideas and report their recent progress.
Year(s) Of Engagement Activity 2018
URL http://www.hfr2018.org/
 
Description Program co-chair of The 19th UK Workshop on Computational Intelligence (UKCI) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact As a program co-chair, I am co-organizing the 19th UK Workshop on Computational Intelligence (UKCI), to be held in Portsmouth, UK, on September 4-6, 2019. Attendees will include academics, postgraduate students and industrial delegates. Recent progress on computational intelligence will be reported and discussed in the workshop.
Year(s) Of Engagement Activity 2019
URL http://www.ukci2019.port.ac.uk
 
Description Publiction chair of The 15th International Conference on Intelligent Unmanned Systems (ICIUS) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact As a publiction chair, I am co-organizing The 15th International Conference on Intelligent Unmanned Systems (ICIUS) to be held in Beijing, China, on August 27-29, 2019. Around 100 participants will attend the conference to exchange ideas on robots and other unmanned systems.
Year(s) Of Engagement Activity 2019
URL http://www.icius2019.org