Digital twin-based Bilateral Teleautonomous System for Nuclear Remote Operation

Lead Research Organisation: University of Manchester
Department Name: Electrical and Electronic Engineering

Abstract

The final goal has been incubated from the bottom line truth about the deployment telerobotic technology for real-world nuclear applications; even after half a century of innovation, there is no tangible telerobotic technology that is adequate for deployment for real-world nuclear applications such as decontamination and dismantling of nuclear facilities. Bilateral teleoperators and other sensor-based telerobotic systems are too complex and fragile, and unilateral teleoperators are too inefficient. What is needed to this end is a new telerobotic system that can utilize simple and robust slave hardware and yet perform dexterous manipulation via (potentially degraded) low-bandwidth communication. To this end, a new teleoperation method, namely 'digital twin-based tele-autonomy', which incorporates a digital replica of the world (i.e., digital twin) and local autonomy is proposed.
The overall research objective of this project is to implement a new teleoperation method, namely digital twin-based bilateral tele-autonomy, and demonstrate its applicability for civil nuclear applications. By integration of simulation in the digital twin and local autonomous behaviours, the performance goal is to achieve dexterous teleoperation with high precision and efficiency by extracting more performance from simple and robust robotic equipment.

Planned Impact

This project has been outlined with achievable pathways to impact in mind from the outset. VR is a common thread through all this work and hence each collaboration is an impact route. In particular, this work aligns with the intentions of Robotics and AI in Nuclear hub (RAIN) to strengthen national networks (e.g. nuclear: NDA, SL, AWE, EDFE, UKAEA; non-nuclear RASSIG, UKRAS, Robotics Growth Partnership) and form new international collaborations. Existing projects include: ITER (EU, US, Russia, ROK, China, Japan India), F4E and EUROfusion, NDF and TEPCO (Japan). New collaborations include CEA, IIT, Tokyo Univ., UoTexas as well as KAERI which once developed will place RACE within a global robotics web.
RACE - This project will be one of more than 20 within RACE (robotics team is now >>130 making RACE one of the largest nuclear robotics labs globally) with more than 100 industrial partners and >10 universities. The research will also deliver societal impact through events and social media. RACE has more than 1000 visitors per annum and supports the UKAEA's engagement with schools visits (>>1000 pa), apprenticeships (growing to 80 pa with Oxford Advanced Skills opening in Sept 2019), graduate training (~10 in RACE alone).
MAN - The Robotics for Extreme Environment group at the University of Manchester has strong links with Nuclear Industry, where the project will have a direct impact. Results will be demonstrated at REEL (Cumbria) for Sellafield Ltd and National Nuclear Laboratory (NLL). However, we are carrying out projects with other industry sectors. The results in this project will be disseminated to other partners in manufacturing and health care.

Publications

10 25 50
 
Description This grant has achieved the following results:

1/ Reinforcement learning-based control has been developed for the motion planning of robotic arms and collision avoidance of mobile robots. Virtual marker is also designed to represent the pose of the end effector of robotic arm and inverse kinematics algorithm is applied to convert the movement of the virtual marker in the virtual reality world into the joint control of the real robot arm.

2/ A 3D vision-guided pick-and-place method is developed to control Kuka iiwa robot arm to pick randomly placed objects. The point cloud of each object needs to be pre-scanned, segmented, and registered. Iterative closest point algorithm is implemented to recognise the pose of the randomly placed object, the object pose under the camera coordinate is then converted to the robot arm coordinate, and an inverse kinematics algorithm is applied to control the robotic arm to the target position to pick the object. Moreover, an object clustering and segmentation strategy were integrated to prioritise the visualisation of important objects on the scene. Using KD-trees techniques it was possible to differentiate clusters of objects located on the scene.

3/ Development of several approaches for point cloud compression strategies for 3D visualisation of the remote environment. This allows the operator to visualise in real-time the tasks conducted on the remote system. This is particularly important to avoid undesirable behaviour of the manipulator and to avoid collisions. The compression approaches focused on single and multiple level-of-detail compression and transmission aimed at prioritising the visualisation of specific objects being manipulated by the robot based on the bandwidth limitations. Furthermore, saliency- and gaze-based approaches were also developed. The visualisation system was developed using the game engine Unity, which allows us to generate realistic rendering environments. Also, Unity is compatible with multiple COTS VR and XR headsets that enhance the user experience.
Exploitation Route Our reinforcement learning code has been made open-access, and it is being widely used. The paper was published in Dec 2020 and it has already almost 100 cites in google scholar.
Sectors Digital/Communication/Information Technologies (including Software),Other

 
Description SBRI - Digital Technologies for Robotic Nuclear Decommissioning: Modelling and Control of a Long Flexible Manipulator - C/2064385
Amount £149,975 (GBP)
Funding ID C/2064385 
Organisation UK Atomic Energy Authority 
Sector Public
Country United Kingdom
Start 07/2022 
End 08/2023
 
Description SBRI - Digital Technologies for Robotic Nuclear Decommissioning: Teleoperation with digital twins - C/2064382
Amount £149,926 (GBP)
Funding ID C/2064382 
Organisation UK Atomic Energy Authority 
Sector Public
Country United Kingdom
Start 07/2022 
End 08/2023
 
Description School Visit (The Premier Academy) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact A talk on a self-balanced robot and a haptic glove was delivered for 32 school children (18 girls, 14 boys) at The Premier Academy, Saffron Street, Bletchley, MK2 3AH
Year(s) Of Engagement Activity 2020
 
Description VR-enabled teleoperation of robotic arm between University of Manchester and University of Edinburgh. An initial approach 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Open webinar for researchers and general public about digital tissue technologies. Attended by 60 people. Discussion on required training of the operator, VR tools for teleoperating a remote manipulator, and commercial available solutions.
Year(s) Of Engagement Activity 2020
URL https://orcahub.org/engagement/iscf-robots-for-a-safer-world/iscf-cross-hub-webinars