Digital twin-based Bilateral Teleautonomous System for Nuclear Remote Operation
Lead Research Organisation:
University of Manchester
Department Name: Electrical and Electronic Engineering
Abstract
The final goal has been incubated from the bottom line truth about the deployment telerobotic technology for real-world nuclear applications; even after half a century of innovation, there is no tangible telerobotic technology that is adequate for deployment for real-world nuclear applications such as decontamination and dismantling of nuclear facilities. Bilateral teleoperators and other sensor-based telerobotic systems are too complex and fragile, and unilateral teleoperators are too inefficient. What is needed to this end is a new telerobotic system that can utilize simple and robust slave hardware and yet perform dexterous manipulation via (potentially degraded) low-bandwidth communication. To this end, a new teleoperation method, namely 'digital twin-based tele-autonomy', which incorporates a digital replica of the world (i.e., digital twin) and local autonomy is proposed.
The overall research objective of this project is to implement a new teleoperation method, namely digital twin-based bilateral tele-autonomy, and demonstrate its applicability for civil nuclear applications. By integration of simulation in the digital twin and local autonomous behaviours, the performance goal is to achieve dexterous teleoperation with high precision and efficiency by extracting more performance from simple and robust robotic equipment.
The overall research objective of this project is to implement a new teleoperation method, namely digital twin-based bilateral tele-autonomy, and demonstrate its applicability for civil nuclear applications. By integration of simulation in the digital twin and local autonomous behaviours, the performance goal is to achieve dexterous teleoperation with high precision and efficiency by extracting more performance from simple and robust robotic equipment.
Planned Impact
This project has been outlined with achievable pathways to impact in mind from the outset. VR is a common thread through all this work and hence each collaboration is an impact route. In particular, this work aligns with the intentions of Robotics and AI in Nuclear hub (RAIN) to strengthen national networks (e.g. nuclear: NDA, SL, AWE, EDFE, UKAEA; non-nuclear RASSIG, UKRAS, Robotics Growth Partnership) and form new international collaborations. Existing projects include: ITER (EU, US, Russia, ROK, China, Japan India), F4E and EUROfusion, NDF and TEPCO (Japan). New collaborations include CEA, IIT, Tokyo Univ., UoTexas as well as KAERI which once developed will place RACE within a global robotics web.
RACE - This project will be one of more than 20 within RACE (robotics team is now >>130 making RACE one of the largest nuclear robotics labs globally) with more than 100 industrial partners and >10 universities. The research will also deliver societal impact through events and social media. RACE has more than 1000 visitors per annum and supports the UKAEA's engagement with schools visits (>>1000 pa), apprenticeships (growing to 80 pa with Oxford Advanced Skills opening in Sept 2019), graduate training (~10 in RACE alone).
MAN - The Robotics for Extreme Environment group at the University of Manchester has strong links with Nuclear Industry, where the project will have a direct impact. Results will be demonstrated at REEL (Cumbria) for Sellafield Ltd and National Nuclear Laboratory (NLL). However, we are carrying out projects with other industry sectors. The results in this project will be disseminated to other partners in manufacturing and health care.
RACE - This project will be one of more than 20 within RACE (robotics team is now >>130 making RACE one of the largest nuclear robotics labs globally) with more than 100 industrial partners and >10 universities. The research will also deliver societal impact through events and social media. RACE has more than 1000 visitors per annum and supports the UKAEA's engagement with schools visits (>>1000 pa), apprenticeships (growing to 80 pa with Oxford Advanced Skills opening in Sept 2019), graduate training (~10 in RACE alone).
MAN - The Robotics for Extreme Environment group at the University of Manchester has strong links with Nuclear Industry, where the project will have a direct impact. Results will be demonstrated at REEL (Cumbria) for Sellafield Ltd and National Nuclear Laboratory (NLL). However, we are carrying out projects with other industry sectors. The results in this project will be disseminated to other partners in manufacturing and health care.
Organisations
Publications
Zhang J
(2020)
Zames-Falb multipliers for convergence rate: motivating example and convex searches
in International Journal of Control
Hu J
(2020)
Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning
in IEEE Transactions on Vehicular Technology
Description | This grant has achieved the following results: 1/ Reinforcement learning-based control has been developed for the motion planning of robotic arms and collision avoidance of mobile robots. Virtual marker is also designed to represent the pose of the end effector of robotic arm and inverse kinematics algorithm is applied to convert the movement of the virtual marker in the virtual reality world into the joint control of the real robot arm. 2/ A 3D vision-guided pick-and-place method is developed to control Kuka iiwa robot arm to pick randomly placed objects. The point cloud of each object needs to be pre-scanned, segmented, and registered. Iterative closest point algorithm is implemented to recognise the pose of the randomly placed object, the object pose under the camera coordinate is then converted to the robot arm coordinate, and an inverse kinematics algorithm is applied to control the robotic arm to the target position to pick the object. Moreover, an object clustering and segmentation strategy were integrated to prioritise the visualisation of important objects on the scene. Using KD-trees techniques it was possible to differentiate clusters of objects located on the scene. 3/ Development of several approaches for point cloud compression strategies for 3D visualisation of the remote environment. This allows the operator to visualise in real-time the tasks conducted on the remote system. This is particularly important to avoid undesirable behaviour of the manipulator and to avoid collisions. The compression approaches focused on single and multiple level-of-detail compression and transmission aimed at prioritising the visualisation of specific objects being manipulated by the robot based on the bandwidth limitations. Furthermore, saliency- and gaze-based approaches were also developed. The visualisation system was developed using the game engine Unity, which allows us to generate realistic rendering environments. Also, Unity is compatible with multiple COTS VR and XR headsets that enhance the user experience. |
Exploitation Route | Our reinforcement learning code has been made open-access, and it is being widely used. The paper was published in Dec 2020 and it has already almost 100 cites in google scholar. |
Sectors | Digital/Communication/Information Technologies (including Software),Other |
Description | SBRI - Digital Technologies for Robotic Nuclear Decommissioning: Modelling and Control of a Long Flexible Manipulator - C/2064385 |
Amount | £149,975 (GBP) |
Funding ID | C/2064385 |
Organisation | UK Atomic Energy Authority |
Sector | Public |
Country | United Kingdom |
Start | 07/2022 |
End | 08/2023 |
Description | SBRI - Digital Technologies for Robotic Nuclear Decommissioning: Teleoperation with digital twins - C/2064382 |
Amount | £149,926 (GBP) |
Funding ID | C/2064382 |
Organisation | UK Atomic Energy Authority |
Sector | Public |
Country | United Kingdom |
Start | 07/2022 |
End | 08/2023 |
Description | School Visit (The Premier Academy) |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Schools |
Results and Impact | A talk on a self-balanced robot and a haptic glove was delivered for 32 school children (18 girls, 14 boys) at The Premier Academy, Saffron Street, Bletchley, MK2 3AH |
Year(s) Of Engagement Activity | 2020 |
Description | VR-enabled teleoperation of robotic arm between University of Manchester and University of Edinburgh. An initial approach |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | Open webinar for researchers and general public about digital tissue technologies. Attended by 60 people. Discussion on required training of the operator, VR tools for teleoperating a remote manipulator, and commercial available solutions. |
Year(s) Of Engagement Activity | 2020 |
URL | https://orcahub.org/engagement/iscf-robots-for-a-safer-world/iscf-cross-hub-webinars |