Digital-twin-based Bilateral Tele-autonomous System for Nuclear Remote Operation
Lead Research Organisation:
CCFE/UKAEA
Department Name: Culham Centre for Fusion Energy
Abstract
Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
Publications
Liu, W
(2021)
A Model-free Deep Reinforcement Learning Approach for Robotic Manipulators Path Planning
in 2021 21st International Conference on Control, Automation and Systems (ICCAS)
Pacheco-Gutierrez S
(2021)
A Multiple Level-of-Detail 3D Data Transmission Approach for Low-Latency Remote Visualisation in Teleoperation Tasks
in Robotics
Pacheco-Gutierrez S
(2021)
Point Cloud Compression and Transmission for Remote Handling Applications
in Journal of Software
Su L
(2023)
On the necessity and sufficiency of discrete-time O'Shea-Zames-Falb multipliers
in Automatica
| Description | We have been able to provide real time 3D visualisation information remotely. This has allowed to tele-operate a remote manipulator safely. Outcomes: • Development of a deep reinforcement learning-based motion planning algorithm for manipulators to carry out pick and place activities. o The algorithm was tested using simulated and real robots o A performance comparison against standard path planning was carried out demonstrating a smoother trajectory generation while keeping joint angles within work range. • Learning collision avoidance from human operator. o Allows remote operators to carry out tasks safely by integrating dinamyc collision avoidance based on 3D data captured from the remote side. • Object 3D model reconstruction from partial views scans o 6DoF pose estimation using deep learning techniques for partially scanned highly symmetrical assets. o Thorough evaluation of capabilities and limitations • Synthethic views o Development of pipeline for 3D remote data transmission that allows up to 15fps data transmission in locations geographically distant ~300 miles. o Deveopment of a geometrical- and deep-learning-based approach to segment remote 3D information to prioritise visualisation and increase transmission speed. |
| Exploitation Route | We are currently testing system with the robotics set up at the University of Manchester. We are looking for integration with the glove boxes located at Sellafield and at RACE as well. |
| Sectors | Digital/Communication/Information Technologies (including Software) Energy Retail |
| URL | https://race.ukaea.uk |
| Description | Innovation with regards to fast point cloud data transmission to support remote viewing has been demonstrated internally and externally through conferences and seminars. This work has been used to define requirements and expectations of other research programs such as LongOps and RAICo. |
| First Year Of Impact | 2020 |
| Sector | Digital/Communication/Information Technologies (including Software),Energy |
| Impact Types | Cultural |
| Description | Vision system integration on VR-aided remote manipulator |
| Organisation | University of Manchester |
| Department | School of Electrical and Electronic Engineering |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | The University of Manchester contributed with a VR-aided remote manipulation system as test scenario for our 3D vision system. Our 3D vision system consists on a real-time 3D data compression and transmission system which enhances the remote side visually. |
| Collaborator Contribution | Provided a test scenario consisting of a robotic manipulator controlled using a hand-tracking device. |
| Impact | A demonstration setup of a 3D-enhanced vision system for remote operations. Data to assess the 3D data transmission time. |
| Start Year | 2020 |
| Title | Point cloud compression ROS (Robot Operating System) package |
| Description | This package allows the compression of 3D and colour data (point cloud). The package can compress all or specific objects on the scene at multiple levels of resolution. Colour can also be dismissed accordingly. This allows to transfer 3D information remotely at high rates. |
| Type Of Technology | New/Improved Technique/Technology |
| Year Produced | 2021 |
| Impact | Improve the visualisation of remote environments. Provide 3D feedback to remote operators reliably. General improvement of remote handling tasks. |
| Description | Book on Robotics and Remote Operations |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Book discussion. |
| Year(s) Of Engagement Activity | 2021 |
| Description | Civil Nuclear Fusion & Supply Chain HPO Workshop - 4 November 2020 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | Civil Nuclear Fusion & Supply Chain HPO Workshop - 4 November 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | Cortex Demo Day - 11 September 2020 |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | Demo Day |
| Year(s) Of Engagement Activity | 2020 |
| Description | European Nuclear Young Generation Forum 2021 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | • Pacheco Gutierrez, S., Caliskanelli, I., Skilton, R., "The Importance of Synthetic Viewing for Teleoperation Tasks in Hazardous Environments" the European Nuclear Young Generation Forum 2021 (ENYGF21), Tarragona, Spain, September 27 30, 2021 |
| Year(s) Of Engagement Activity | 2021 |
| Description | ICRND - regular meetings in 2020 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | ICRND - regular meetings throughout 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | ICRND Virtual Meeting - 13 November |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | ICRND Virtual Meeting |
| Year(s) Of Engagement Activity | 2020 |
| Description | Lakes College Walkaround - 22 September 2020 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | Meeting |
| Year(s) Of Engagement Activity | 2020 |
| Description | Meetings - Boston Dynamics SPOT Nuclear Deployment Discussion / SPOT Webinar - 18 November 2020 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | Meetings - Boston Dynamics SPOT Nuclear Deployment Discussion / SPOT Webinar - 18 November 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | Nuclear Innovation Developments for Legacy Cost Reduction 2020 meeting - 17 November 2020 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Industry/Business |
| Results and Impact | Nuclear Innovation Developments for Legacy Cost Reduction 2020 meeting - 17 November 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | Presentation of remote visualisation and teleoperation system during a RAIN seminar |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | Participation in the live demonstration event organised by RAIN as part of the Remote Handling Working Group. We presented a real-time demonstration of the teleoperation system and the real-time 3D visualisation system operating from two remote locations: Manchester and Oxford. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://youtu.be/eAIQN0wCUzs?t=3712 |
| Description | RACE - Boston Dynamics Meeting - 24 November 2020 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | RACE - Boston Dynamics Meeting |
| Year(s) Of Engagement Activity | 2020 |
| Description | RAI Meeting - 9 November 2020 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Industry/Business |
| Results and Impact | RAI Meeting |
| Year(s) Of Engagement Activity | 2020 |
| Description | RAICo Statement of Need |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Submission of Statement of Need to UKRI re: Robotics and AI Collaboration Laboratory proposal on behalf of NDA, University of Manchester, Sellafield Ltd, National Nuclear Laboratory and UKAEA. |
| Year(s) Of Engagement Activity | 2021 |
| Description | TEPCO Workshop - 26 November 2020 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Supporters |
| Results and Impact | TEPCO Workshop |
| Year(s) Of Engagement Activity | 2020 |
| Description | Tepco Discussion - 5 November 2020 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Industry/Business |
| Results and Impact | Tepco discussion |
| Year(s) Of Engagement Activity | 2020 |
| Description | The 19th International Conference on Ubiquitous Robots (UR), Jeju, South Korea, 2022. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Industry/Business |
| Results and Impact | • Pacheco Gutierrez, S., Caliskanelli, I., Skilton, R., "Saliency and Perception Techniques for Scene Prioritisation in Remote Viewing for Nuclear", The 19th International Conference on Ubiquitous Robots (UR), Jeju, South Korea, 2022 |
| Year(s) Of Engagement Activity | 2021 |