Semi Autonomous Teleoperation System (SATS)
Lead Research Organisation:
University College London
Department Name: Computer Science
Abstract
Advanced robots should be used for difficult, dirty, dangerous and dull tasks, rather than humans. At present, the capabilities of these systems are hindered by their ability to control complex robotic functions such as performing a range of grasping tasks with tools that requires coordinating multiple end-effectors at the same time. To overcome these limitations, UCL will work with UK SMEs Shadow Robot Company, OC Robotics and Cambrian Intelligence to develop and test the feasibility of a novel semi-autonomous teleoperation system (SAT) to demonstrate new paradigm-shifting approaches. By leveraging unique testbed facilities at UCL HereEast for teleoperation technologies, the goal of the project is to replace tedious low level control operations with higher level tasks; allowing the operator to concentrate on the overall task to be conducted, rather than needing to micromanage a robot system through low-level operations like maintaining gripper contact with objects, or path planning through cluttered spaces. This system is expected to significantly ease the cognitive burden on the operator, making difficult tasks easier and all tasks more reliable and robust, whilst at the same time enabling operators to need less training in "using the machine", and speeding up task performance. Across industries using teleoperation in regular use, such as nuclear operations, fusion engineering, and subsea maintenance and inspection we expect semi-autonomous teleoperation technologies to produce a step-change in task productivity and remote decision making with concomitant economic impact.
The project builds on key UK activities in snake-arm robots, dexterous hands, teleoperation platforms, connectivity and research environments to deliver a step-change in capability. Within this project, the primary objective for the UCL team is to support the development, performance testing, analysis and dissemination of results of the proposed demonstrator systems. Leveraging a UCL EPSRC capital award >£2.4M in "Robotic Teleoperation for Multiple Scales: Enabling Exploration, Manipulation and Assembly Tasks in New Worlds Beyond Human Capabilities" (EP/K005030/1), we will provide the SME partners access to relevant robotic equipment and teleoperation platforms to develop a series of SAT systems. We will also exploit our existing background research to define suitable benchmarks to assist the measurement of end-user and technical performance of the system. These activities will include:
1) Providing space to mock-up different testing environments for demonstration
2) Help define suitable benchmarking frameworks to analyse the technical feasibility of the developed SAT systems such as stability, transparency and robustness measures
3) Work with SME partners to establish end user requirements and optimise developed demonstrator to potential first customer needs
4) Give access to UCL's existing industrial collaboration network to help broker follow on investment
5) Enhance international dissemination activities by presenting usability and performance results at leading events such as ICRA and IROS, in addition to industry focused forums
For UCL, this project will provide essential knowledge to help develop future sensors, actuators and components for creating the next generation of robotic technologies.
The project builds on key UK activities in snake-arm robots, dexterous hands, teleoperation platforms, connectivity and research environments to deliver a step-change in capability. Within this project, the primary objective for the UCL team is to support the development, performance testing, analysis and dissemination of results of the proposed demonstrator systems. Leveraging a UCL EPSRC capital award >£2.4M in "Robotic Teleoperation for Multiple Scales: Enabling Exploration, Manipulation and Assembly Tasks in New Worlds Beyond Human Capabilities" (EP/K005030/1), we will provide the SME partners access to relevant robotic equipment and teleoperation platforms to develop a series of SAT systems. We will also exploit our existing background research to define suitable benchmarks to assist the measurement of end-user and technical performance of the system. These activities will include:
1) Providing space to mock-up different testing environments for demonstration
2) Help define suitable benchmarking frameworks to analyse the technical feasibility of the developed SAT systems such as stability, transparency and robustness measures
3) Work with SME partners to establish end user requirements and optimise developed demonstrator to potential first customer needs
4) Give access to UCL's existing industrial collaboration network to help broker follow on investment
5) Enhance international dissemination activities by presenting usability and performance results at leading events such as ICRA and IROS, in addition to industry focused forums
For UCL, this project will provide essential knowledge to help develop future sensors, actuators and components for creating the next generation of robotic technologies.
Planned Impact
Continued technological revolutions in Robotics and Autonomous Systems (RAS) is set to drive UK productivity into the future. Current UCL examples of the diverse economic, academic and societal impact include autonomous additive manufacture of buildings in-situ using distributed robots (EP/N018494/1); helping to reduce construction time, material and transport costs, improve worker safety and environmental impact. Within domains such as aerospace, our collaborations with Airbus and the AMRC (Innovate UK/57459-256136) will develop step changing technologies in both volume and individualised agile manufacture where each part can be a one-off and tracked throughout its product lifecycle. Within agriculture, working with the Rothamsted Research will develop robotic systems that use machine learning to adapt to changing environmental conditions and grow plants with limited use of pesticides, making production more environmentally friendly. Altogether, these outputs align with the Government's 'Manufacturing the Future' and EPSRC's objectives to for a 'Productive, Resilient and Healthy nation'.
Build upon existing connected initiatives, this project will generate ambitious and achievable impact. Industrial impact will be delivered through UCL's network of industrial partners and potential first customers of teleoperation technologies: Airbus, Reinshaw, Ultimaker, Skanska, Dyson, Arup, Burohappold, BBC, Microsoft, Electronic Arts, National Physical Laboratory (NPL), Cullum Centre for Fusion Energy (CCFE), PriceWaterhouseCoopers, Siemens, Philips, High-Value Manufacturing (HVM) and Digital Catapults, High Speed Sustainable Manufacturing Institute (HSSMI), GKN, and Intuitive Surgical Inc. We will also exploit membership on external networks such as the EPSRC UK-RAS Network, and Innovate UK's HVM Network. Academic impact will leverage the high-profile of each lead applicant ensuring dissemination of scientific knowledge through conferences, workshops, journals, exhibitions and demonstrator projects. Utilising the space afforded at UCL HereEast, public activities will benefit from capacity to host 2000 people at any one time to showcase community and high impact events. These activities will also be supported by established relationships with media outlets including the BBC, The Guardian, TimeOut and Wired Magazine. Regular interaction with schools and a dedicated YouTube channel will be used to inform, inspire and educate the wider public about the potential of future manufacturing technologies.
Planned initiatives to support Industrial, Academic and Societal impact include:
1) Real-world industrial impact: Regular meetings with network of industrial members. Includes scoping exercises, technology roadmap workshops and calls for feasibility studies. We will also extend existing Industrial Advisory Board to have a clear role in the project's governance. Boosting the visibility of underpinning technologies through new testing facilities at UCL HereEast will also aid industrial translation
2) Clear IP & legal frameworks: Work with UCL Business (UCLB) to create a clear framework to accelerate innovation
3) Supporting the academic community: Target for publication at major conferences, organised sessions and workshops such as IROS, ICRA, TAROS, SIGCHI, and industry tradeshows to involve the wider community
4) Bridging the gap to local communities: Commitment to yearly showcase and enterprise initiatives at UCL HereEast with local practices to address fragmentation and understand the non-standard demands on. Examples include the Bartlett Summer Show and planned Exhibition at the Venice Architecture Biennale in 2018. Past exhibits include demonstrations during the Science Museum Lates to help disseminate results to a wider audience
5) Enhancing national capability: Provide case studies of UCL's capabilities that combine design with high end RAS research that fills a gap in provision and serves a variety of engineering demands
Build upon existing connected initiatives, this project will generate ambitious and achievable impact. Industrial impact will be delivered through UCL's network of industrial partners and potential first customers of teleoperation technologies: Airbus, Reinshaw, Ultimaker, Skanska, Dyson, Arup, Burohappold, BBC, Microsoft, Electronic Arts, National Physical Laboratory (NPL), Cullum Centre for Fusion Energy (CCFE), PriceWaterhouseCoopers, Siemens, Philips, High-Value Manufacturing (HVM) and Digital Catapults, High Speed Sustainable Manufacturing Institute (HSSMI), GKN, and Intuitive Surgical Inc. We will also exploit membership on external networks such as the EPSRC UK-RAS Network, and Innovate UK's HVM Network. Academic impact will leverage the high-profile of each lead applicant ensuring dissemination of scientific knowledge through conferences, workshops, journals, exhibitions and demonstrator projects. Utilising the space afforded at UCL HereEast, public activities will benefit from capacity to host 2000 people at any one time to showcase community and high impact events. These activities will also be supported by established relationships with media outlets including the BBC, The Guardian, TimeOut and Wired Magazine. Regular interaction with schools and a dedicated YouTube channel will be used to inform, inspire and educate the wider public about the potential of future manufacturing technologies.
Planned initiatives to support Industrial, Academic and Societal impact include:
1) Real-world industrial impact: Regular meetings with network of industrial members. Includes scoping exercises, technology roadmap workshops and calls for feasibility studies. We will also extend existing Industrial Advisory Board to have a clear role in the project's governance. Boosting the visibility of underpinning technologies through new testing facilities at UCL HereEast will also aid industrial translation
2) Clear IP & legal frameworks: Work with UCL Business (UCLB) to create a clear framework to accelerate innovation
3) Supporting the academic community: Target for publication at major conferences, organised sessions and workshops such as IROS, ICRA, TAROS, SIGCHI, and industry tradeshows to involve the wider community
4) Bridging the gap to local communities: Commitment to yearly showcase and enterprise initiatives at UCL HereEast with local practices to address fragmentation and understand the non-standard demands on. Examples include the Bartlett Summer Show and planned Exhibition at the Venice Architecture Biennale in 2018. Past exhibits include demonstrations during the Science Museum Lates to help disseminate results to a wider audience
5) Enhancing national capability: Provide case studies of UCL's capabilities that combine design with high end RAS research that fills a gap in provision and serves a variety of engineering demands
Publications
Sustarevas J
(2018)
MAP - A Mobile Agile Printer Robot for on-site Construction
Pawar VM
(2017)
Toward autonomous architecture: The convergence of digital design, robotics, and the built environment.
in Science robotics
Description | Key results developed through the supported research is our understanding on how to develop robust multi-functional robot systems within a coherent framework. This included being able to test the performance of a range of consumer and software libraries that cover the following technology domains: consumer RGB-D cameras for environmental mapping, vision based calibration systems, motion planning systems and virtual reality interfaces. By understanding how these technologies can work together, the supported research has improved our knowledge and developed in house technical expertise to create robotic systems that can perceive and safely interact with their surroundings (useful for a wide range of industrial applications). The development of 2 demonstrator systems will also aid the growth of two research themes at UCL: 1) adaptive grasping using machine learning and 2) the autonomous deployment of large scale metrology networks using real-time dexterous robotic systems. |
Exploitation Route | These findings has helped UCL develop a common set of capabilities that can be deployed across a wide range of robotic systems available at UCL HereEast. We intend to extend this work further to improve our ability to quickly develop multi-functional robotic demonstrator systems and have the flexibility to address a larger variety of industrial use cases. This method of working will be integrated with our technology roadmap as a way of generating results and become more competitive in supporting new activities that explore the use of robotics within their problem domains. |
Sectors | Aerospace Defence and Marine Construction Digital/Communication/Information Technologies (including Software) Education Manufacturing including Industrial Biotechology |
Description | Through collaboration with Shadow Robot Company and OC Robotics (GE Aviation), we were introduced to industry use cases of the SAT technology. Through this effort, non-academic outputs include: 1) the development of 2 industry focused demonstrator systems and 2) learning associated with using UCL's new Robotics facility for supporting large scale innovation projects. This has led to changes in the UCL technology roadmap towards common frameworks that enable flexible collaboration to speed up the delivery of demonstration systems. Through the associated networking activities of the project, this has also benefited UCL brand in support innovation in robotics and resulted in new EPSRC iCASE studentship sponsorship with ARUP and UKAEA. The project has also created the foundation for new projects that utilise the dexterous operation of the OC snake-robot and ability to deploy agile sensor networks over a large volume. This is informing a set of manufacturing industry-focused demonstrator outputs linking large-volume tracking to component assembly tolerances suitable for the aerospace industry. |
First Year Of Impact | 2020 |
Sector | Aerospace, Defence and Marine,Construction,Digital/Communication/Information Technologies (including Software) |
Impact Types | Economic |
Description | Robotic and Autonomous Systems for Resilient Infrastructure |
Geographic Reach | National |
Policy Influence Type | Implementation circular/rapid advice/letter to e.g. Ministry of Health |
URL | http://hamlyn.doc.ic.ac.uk/uk-ras/sites/default/files/UK_RAS_wp_infrastructure_final_web_retina.pdf |
Description | EPSRC iCASE with ARUP |
Amount | £110,000 (GBP) |
Organisation | Arup Group |
Sector | Private |
Country | United Kingdom |
Start | 09/2017 |
End | 10/2021 |
Description | EPSRC iCASE with UKAEA |
Amount | £110,000 (GBP) |
Organisation | Culham Centre for Fusion Energy |
Sector | Academic/University |
Country | United Kingdom |
Start | 09/2017 |
End | 10/2021 |
Description | OC Robotics |
Organisation | GE Aviation Systems |
Country | United States |
Sector | Private |
PI Contribution | 1) Development of a large scale metrology guided aerospace demonstrator system 2) Defining the need for a ROS compatible joint interface to control the robot system 3) Defining the need for a motion planning system to support autonomous motions |
Collaborator Contribution | 1) Introduction to industry relevant use cases for future development 2) Technical support in integrating and setting up the metrology guided aerospace demonstrator system |
Impact | Through co-development of the demonstrator system, we have explored aerospace focused use cases for future exploitation. In particular, we have identified an opportunity that incorporates UCL research in 3D printing and controlled deposition that would provide new capabilities to the robot. In doing so, this project has highlighted the need for future work that includes 1) a ROS compatible joint interface and 2) robust motion planner. |
Start Year | 2017 |
Description | Shadow Robot Company |
Organisation | Shadow Robot Company |
Country | United Kingdom |
Sector | Private |
PI Contribution | 1) Co-development of a dexterous grasping platform to investigate research in multi-modal autonomous manipulation 2) Alignment of software framework to enable future collaborative work and easy integration of other UCL research outputs and devices 3) Development of demonstrator system for showcasing SAT project outputs |
Collaborator Contribution | 1) Loan of a robotic grasping system for testing and evaluation 2) Introduction to industry relevant use cases for grasping and SAT technologies for future development 3) Introduction to industry relevant software platforms and technologies to consider using within UCL research frameworks |
Impact | Discovery of new research problems: 1) fixed and freely moving visual-robot calibration 2) autonomous visual mapping to overcome occlusions to reinforce grasping and subsequent task planning 3) robust robot motion planner 4) database system of registering recognised objects that maybe dynamic. Understanding these challenges required a multi-disciplinary engineering approach in computer science, mathematics and robotics. Pilot results achieved through this project to tackle these issues include a software platform that enables future research and potential publications (part of which has already contributed to an IROS '18 paper submission [under review]). |
Start Year | 2017 |
Title | ROS Compatible Fast Robot Interface for a Kuka IIWA |
Description | This is a software stack that enables operation of the a Kuka IIWA using its Fast Robot Interface via ROS. |
Type Of Technology | Software |
Year Produced | 2018 |
Open Source License? | Yes |
Impact | A software stack that enables access to high performance capabilities of the robot. This is now being used to support research more widely within UCL and its student base |
Description | Construction Site Robotics Symposium (ARUP, London) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Organised by Arup, event brought together a range of leading thinkers to explore opportunities and implications of responding to a future of construction site robotics. Between 50-100 industry and academic participants attending the event generating following collaboration leads. This includes an EPSRC iCASE with ARUP. |
Year(s) Of Engagement Activity | 2018 |
Description | Industry Talk at Haidenhain Corporation |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Invited talk to discuss UCL research results and relevance to future manufacturing challenges. This was done to provide decision makers at the Haidenhain Corporation to engage with UCL in a collaboration agreement |
Year(s) Of Engagement Activity | 2017 |
Description | Invited talk at Indian Institute of Technology Madras |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Presentation of UCL and project results. This included discussions for a formal collaboration and co-development of technologies. |
Year(s) Of Engagement Activity | 2018 |
Description | Invited talk at Queensland Brain Institute |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Presentation of UCL results and outputs from the project. This included discussions to set up a collaboration between institutions and exchange of expertise. |
Year(s) Of Engagement Activity | 2017 |
Description | Invited talk at Tufts University School of Medicine |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Introduction to UCL results and outputs from the project related to potential use cases in the healthcare domain |
Year(s) Of Engagement Activity | 2017 |
Description | Robotics and Autonomous Systems in UK Manufacturing |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | The aim work of this workshop was to address the knowledge gap between RAS researchers, policy and the manufacturing communities. The specific objectives were to: -Identify to the state of the art in RAS and how these technologies can address real-world manufacturing challenges. -Identify areas of added value for RAS that can have long term impact on UK manufacturing. -Support new collaborations between technology and application communities. -Develop a strategy for RAS to support UK manufacturing. |
Year(s) Of Engagement Activity | 2016 |
URL | http://hamlyn.doc.ic.ac.uk/uk-ras/events/manufacturing-workshop-AMRC |
Description | UCL-Shadow Robot Company Dognguan Visit |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | We hosted a Chinese delegation at UCL to discuss the future of Robotic and Autonomous Systems technologies in transforming manufacturing and factory environments. We introduced the SAT concept and gave a live demo of the Shadow Robot Hand. In total between 40 - 50 people attended the event organised in partnership with the Foreign and Commonwealth Office. Follow on outputs include an invitation to visit Dognguan from its Major and to meet local industry with the opportunity to work with them in the future. |
Year(s) Of Engagement Activity | 2017 |