Semi Autonomous Teleoperation System (SATS)

Lead Research Organisation: University College London
Department Name: Computer Science

Abstract

Advanced robots should be used for difficult, dirty, dangerous and dull tasks, rather than humans. At present, the capabilities of these systems are hindered by their ability to control complex robotic functions such as performing a range of grasping tasks with tools that requires coordinating multiple end-effectors at the same time. To overcome these limitations, UCL will work with UK SMEs Shadow Robot Company, OC Robotics and Cambrian Intelligence to develop and test the feasibility of a novel semi-autonomous teleoperation system (SAT) to demonstrate new paradigm-shifting approaches. By leveraging unique testbed facilities at UCL HereEast for teleoperation technologies, the goal of the project is to replace tedious low level control operations with higher level tasks; allowing the operator to concentrate on the overall task to be conducted, rather than needing to micromanage a robot system through low-level operations like maintaining gripper contact with objects, or path planning through cluttered spaces. This system is expected to significantly ease the cognitive burden on the operator, making difficult tasks easier and all tasks more reliable and robust, whilst at the same time enabling operators to need less training in "using the machine", and speeding up task performance. Across industries using teleoperation in regular use, such as nuclear operations, fusion engineering, and subsea maintenance and inspection we expect semi-autonomous teleoperation technologies to produce a step-change in task productivity and remote decision making with concomitant economic impact.

The project builds on key UK activities in snake-arm robots, dexterous hands, teleoperation platforms, connectivity and research environments to deliver a step-change in capability. Within this project, the primary objective for the UCL team is to support the development, performance testing, analysis and dissemination of results of the proposed demonstrator systems. Leveraging a UCL EPSRC capital award >£2.4M in "Robotic Teleoperation for Multiple Scales: Enabling Exploration, Manipulation and Assembly Tasks in New Worlds Beyond Human Capabilities" (EP/K005030/1), we will provide the SME partners access to relevant robotic equipment and teleoperation platforms to develop a series of SAT systems. We will also exploit our existing background research to define suitable benchmarks to assist the measurement of end-user and technical performance of the system. These activities will include:

1) Providing space to mock-up different testing environments for demonstration
2) Help define suitable benchmarking frameworks to analyse the technical feasibility of the developed SAT systems such as stability, transparency and robustness measures
3) Work with SME partners to establish end user requirements and optimise developed demonstrator to potential first customer needs
4) Give access to UCL's existing industrial collaboration network to help broker follow on investment
5) Enhance international dissemination activities by presenting usability and performance results at leading events such as ICRA and IROS, in addition to industry focused forums

For UCL, this project will provide essential knowledge to help develop future sensors, actuators and components for creating the next generation of robotic technologies.

Planned Impact

Continued technological revolutions in Robotics and Autonomous Systems (RAS) is set to drive UK productivity into the future. Current UCL examples of the diverse economic, academic and societal impact include autonomous additive manufacture of buildings in-situ using distributed robots (EP/N018494/1); helping to reduce construction time, material and transport costs, improve worker safety and environmental impact. Within domains such as aerospace, our collaborations with Airbus and the AMRC (Innovate UK/57459-256136) will develop step changing technologies in both volume and individualised agile manufacture where each part can be a one-off and tracked throughout its product lifecycle. Within agriculture, working with the Rothamsted Research will develop robotic systems that use machine learning to adapt to changing environmental conditions and grow plants with limited use of pesticides, making production more environmentally friendly. Altogether, these outputs align with the Government's 'Manufacturing the Future' and EPSRC's objectives to for a 'Productive, Resilient and Healthy nation'.

Build upon existing connected initiatives, this project will generate ambitious and achievable impact. Industrial impact will be delivered through UCL's network of industrial partners and potential first customers of teleoperation technologies: Airbus, Reinshaw, Ultimaker, Skanska, Dyson, Arup, Burohappold, BBC, Microsoft, Electronic Arts, National Physical Laboratory (NPL), Cullum Centre for Fusion Energy (CCFE), PriceWaterhouseCoopers, Siemens, Philips, High-Value Manufacturing (HVM) and Digital Catapults, High Speed Sustainable Manufacturing Institute (HSSMI), GKN, and Intuitive Surgical Inc. We will also exploit membership on external networks such as the EPSRC UK-RAS Network, and Innovate UK's HVM Network. Academic impact will leverage the high-profile of each lead applicant ensuring dissemination of scientific knowledge through conferences, workshops, journals, exhibitions and demonstrator projects. Utilising the space afforded at UCL HereEast, public activities will benefit from capacity to host 2000 people at any one time to showcase community and high impact events. These activities will also be supported by established relationships with media outlets including the BBC, The Guardian, TimeOut and Wired Magazine. Regular interaction with schools and a dedicated YouTube channel will be used to inform, inspire and educate the wider public about the potential of future manufacturing technologies.

Planned initiatives to support Industrial, Academic and Societal impact include:
1) Real-world industrial impact: Regular meetings with network of industrial members. Includes scoping exercises, technology roadmap workshops and calls for feasibility studies. We will also extend existing Industrial Advisory Board to have a clear role in the project's governance. Boosting the visibility of underpinning technologies through new testing facilities at UCL HereEast will also aid industrial translation
2) Clear IP & legal frameworks: Work with UCL Business (UCLB) to create a clear framework to accelerate innovation
3) Supporting the academic community: Target for publication at major conferences, organised sessions and workshops such as IROS, ICRA, TAROS, SIGCHI, and industry tradeshows to involve the wider community
4) Bridging the gap to local communities: Commitment to yearly showcase and enterprise initiatives at UCL HereEast with local practices to address fragmentation and understand the non-standard demands on. Examples include the Bartlett Summer Show and planned Exhibition at the Venice Architecture Biennale in 2018. Past exhibits include demonstrations during the Science Museum Lates to help disseminate results to a wider audience
5) Enhancing national capability: Provide case studies of UCL's capabilities that combine design with high end RAS research that fills a gap in provision and serves a variety of engineering demands
 
Description Key results developed through the supported research is our understanding on how to develop robust multi-functional robot systems within a coherent framework. This included being able to test the performance of a range of consumer and software libraries that cover the following technology domains: consumer RGB-D cameras for environmental mapping, vision based calibration systems, motion planning systems and virtual reality interfaces. By understanding how these technologies can work together, the supported research has improved our knowledge and developed in house technical expertise to create robotic systems that can perceive and safely interact with their surroundings (useful for a wide range of industrial applications). The development of 2 demonstrator systems will also aid the growth of two research themes at UCL: 1) adaptive grasping using machine learning and 2) the autonomous deployment of large scale metrology networks using real-time dexterous robotic systems.
Exploitation Route These findings has helped UCL develop a common set of capabilities that can be deployed across a wide range of robotic systems available at UCL HereEast. We intend to extend this work further to improve our ability to quickly develop multi-functional robotic demonstrator systems and have the flexibility to address a larger variety of industrial use cases. This method of working will be integrated with our technology roadmap as a way of generating results and become more competitive in supporting new activities that explore the use of robotics within their problem domains.
Sectors Aerospace, Defence and Marine,Construction,Digital/Communication/Information Technologies (including Software),Education,Manufacturing, including Industrial Biotechology

 
Description Through collaboration with Shadow Robot Company and OC Robotics (GE Aviation), we were introduced to industry use cases of the SAT technology. Through this effort, non-academic outputs include: 1) the development of 2 industry focused demonstrator systems and 2) learning associated with using UCL's new Robotics facility for supporting large scale innovation projects. This has led to changes in the UCL technology roadmap towards common frameworks that enable flexible collaboration to speed up the delivery of demonstration systems. Through the associated networking activities of the project, this has also benefited UCL brand in support innovation in robotics and resulted in new EPSRC iCASE studentship sponsorship with ARUP and UKAEA.
First Year Of Impact 2018
Sector Aerospace, Defence and Marine,Construction,Digital/Communication/Information Technologies (including Software)
Impact Types Economic

 
Description Robotic and Autonomous Systems for Resilient Infrastructure
Geographic Reach National 
Policy Influence Type Implementation circular/rapid advice/letter to e.g. Ministry of Health
URL http://hamlyn.doc.ic.ac.uk/uk-ras/sites/default/files/UK_RAS_wp_infrastructure_final_web_retina.pdf
 
Description EPSRC iCASE with ARUP
Amount £110,000 (GBP)
Organisation Arup Group 
Sector Private
Country United Kingdom
Start 10/2017 
End 10/2021
 
Description EPSRC iCASE with UKAEA
Amount £110,000 (GBP)
Organisation Culham Centre for Fusion Energy 
Sector Academic/University
Country United Kingdom
Start 10/2017 
End 10/2021
 
Description OC Robotics 
Organisation GE Aviation Systems
Country United States 
Sector Private 
PI Contribution 1) Development of a large scale metrology guided aerospace demonstrator system 2) Defining the need for a ROS compatible joint interface to control the robot system 3) Defining the need for a motion planning system to support autonomous motions
Collaborator Contribution 1) Introduction to industry relevant use cases for future development 2) Technical support in integrating and setting up the metrology guided aerospace demonstrator system
Impact Through co-development of the demonstrator system, we have explored aerospace focused use cases for future exploitation. In particular, we have identified an opportunity that incorporates UCL research in 3D printing and controlled deposition that would provide new capabilities to the robot. In doing so, this project has highlighted the need for future work that includes 1) a ROS compatible joint interface and 2) robust motion planner.
Start Year 2017
 
Description Shadow Robot Company 
Organisation Shadow Robot Company
Country United Kingdom 
Sector Private 
PI Contribution 1) Co-development of a dexterous grasping platform to investigate research in multi-modal autonomous manipulation 2) Alignment of software framework to enable future collaborative work and easy integration of other UCL research outputs and devices 3) Development of demonstrator system for showcasing SAT project outputs
Collaborator Contribution 1) Loan of a robotic grasping system for testing and evaluation 2) Introduction to industry relevant use cases for grasping and SAT technologies for future development 3) Introduction to industry relevant software platforms and technologies to consider using within UCL research frameworks
Impact Discovery of new research problems: 1) fixed and freely moving visual-robot calibration 2) autonomous visual mapping to overcome occlusions to reinforce grasping and subsequent task planning 3) robust robot motion planner 4) database system of registering recognised objects that maybe dynamic. Understanding these challenges required a multi-disciplinary engineering approach in computer science, mathematics and robotics. Pilot results achieved through this project to tackle these issues include a software platform that enables future research and potential publications (part of which has already contributed to an IROS '18 paper submission [under review]).
Start Year 2017
 
Title ROS Compatible Fast Robot Interface for a Kuka IIWA 
Description This is a software stack that enables operation of the a Kuka IIWA using its Fast Robot Interface via ROS. 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact A software stack that enables access to high performance capabilities of the robot. This is now being used to support research more widely within UCL and its student base 
 
Description Construction Site Robotics Symposium (ARUP, London) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Organised by Arup, event brought together a range of leading thinkers to explore opportunities and implications of responding to a future of construction site robotics. Between 50-100 industry and academic participants attending the event generating following collaboration leads. This includes an EPSRC iCASE with ARUP.
Year(s) Of Engagement Activity 2018
 
Description Industry Talk at Haidenhain Corporation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited talk to discuss UCL research results and relevance to future manufacturing challenges. This was done to provide decision makers at the Haidenhain Corporation to engage with UCL in a collaboration agreement
Year(s) Of Engagement Activity 2017
 
Description Invited talk at Indian Institute of Technology Madras 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation of UCL and project results. This included discussions for a formal collaboration and co-development of technologies.
Year(s) Of Engagement Activity 2018
 
Description Invited talk at Queensland Brain Institute 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation of UCL results and outputs from the project. This included discussions to set up a collaboration between institutions and exchange of expertise.
Year(s) Of Engagement Activity 2017
 
Description Invited talk at Tufts University School of Medicine 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Introduction to UCL results and outputs from the project related to potential use cases in the healthcare domain
Year(s) Of Engagement Activity 2017
 
Description Robotics and Autonomous Systems in UK Manufacturing 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact The aim work of this workshop was to address the knowledge gap between RAS researchers, policy and the manufacturing communities. The specific objectives were to:

-Identify to the state of the art in RAS and how these technologies can address real-world manufacturing challenges.
-Identify areas of added value for RAS that can have long term impact on UK manufacturing.
-Support new collaborations between technology and application communities.
-Develop a strategy for RAS to support UK manufacturing.
Year(s) Of Engagement Activity 2016
URL http://hamlyn.doc.ic.ac.uk/uk-ras/events/manufacturing-workshop-AMRC
 
Description UCL-Shadow Robot Company Dognguan Visit 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact We hosted a Chinese delegation at UCL to discuss the future of Robotic and Autonomous Systems technologies in transforming manufacturing and factory environments. We introduced the SAT concept and gave a live demo of the Shadow Robot Hand. In total between 40 - 50 people attended the event organised in partnership with the Foreign and Commonwealth Office. Follow on outputs include an invitation to visit Dognguan from its Major and to meet local industry with the opportunity to work with them in the future.
Year(s) Of Engagement Activity 2017