Improving Robotic Teleoperation through Human-Robot Interaction and Robotic Autonomy

Teleoperation's overall goal is to give the operator situational awareness (i.e., a sense of being there) of the remote environment and the ability to interact conveniently. Operators of telemanipulators in nuclear plants perform inspections and maintenance tasks, including complex manipulations. Several studies reveal that the unstructured nature of the environment, task complexity, and the conservative nature of the nuclear industry cause teleoperators to prefer to be in control and fully aware of the multiple systems' current and future behaviours at all times, limiting the deployment autonomous teleoperation or robotics. As a result, the operators of these mechanical systems interact with several mediums, such as various displays, robotic telemanipulation systems, information systems, and other colleagues, at once. These interactions impose overwhelming levels of information processing on the operator, leading to a high cognitive workload, which impacts performance and safety in teleoperation.

This PhD research addresses the automation challenge in teleoperation, focusing on enhancing operator performance and safety. It acknowledges the preference for operator control in complex, unstructured environments, hindering the deployment of autonomous systems. It proposes integrating adaptive robotic assistance and responsive interfaces that dynamically respond to an operator's mental workload (MWL) demands, task complexity and structure. The following objectives will be met to achieve this aim:

To identify the objective performance and mental workload metrics for teleoperation.
To determine the sources of physical and mental workload using task analysis.
To develop methods for assessment of performance and mental and physical workloads in teleoperation.
To design robotic assistance schemes based on physical and mental demands to guide operators through teleoperation task completions.
Several pieces of literature have demonstrated how haptic guidance based on the sense of touch can be deployed for robotic assistance. For example, haptic shared control and haptic traded control, which the operators and the system share and take turns in the control task, respectively. This work will explore how these assistance schemes could be deployed to respond to an operator's physical and cognitive demands within structured environments for repetitive and predictable tasks. It will combine some physiological, subjective and performance measures to give an improved assessment and evaluation of the operator's mental workload to understand its relationships with teleoperation tasks. With these improved systems, operators can perform their work more conveniently, faster, and accurately, increasing safety and reducing operating costs.

We will collaborate with over 40 partners drawn from across FMCG and Food; Creative Industries; Health and Wellbeing; Smart Mobility; Finance; Enabling technologies; and Policy, Law and Society. These will benefit from engagement with our CDT through the following established mechanisms:

- Training multi-disciplinary leaders. Our partners will benefit from being able to recruit highly skilled individuals who are able to work across technologies, methods and sectors and in multi-disciplinary teams. We will deliver at least 65 skilled PhD graduates into the Digital Economy.

- Internships. Each Horizon student undertakes at least one industry internship or exchange at an external partner. These internships have a benefit to the student in developing their appreciation of the relevance of their PhD to the external societal and industrial context, and have a benefit to the external partner through engagement with our students and their multidisciplinary skill sets combined with an ability to help innovate new ideas and approaches with minimal long-term risk. Internships are a compulsory part of our programme, taking place in the summer of the first year. We will deliver at least 65 internships with partners.

- Industry-led challenge projects. Each student participates in an industry-led group project in their second year. Our partners benefit from being able to commission focused research projects to help them answer a challenge that they could not normally fund from their core resources. We will deliver at least 15 such projects (3 a year) throughout the lifetime of the CDT.

- Industry-relevant PhD projects. Each student delivers a PhD thesis project in collaboration with at least one external partner who benefits from being able to engage in longer-term and deeper research that they would not normally be able to undertake, especially for those who do not have their own dedicated R&D labs. We will deliver at least 65 such PhDs over the lifetime of this CDT renewal.

- Public engagement. All students receive training in public engagement and learn to communicate their findings through press releases, media coverage.

This proposal introduces two new impact channels in order to further the impact of our students' work and help widen our network of partners.

- The Horizon Impact Fund. Final year students can apply for support to undertake short impact projects. This benefits industry partners, public and third sector partners, academic partners and the wider public benefit from targeted activities that deepen the impact of individual students' PhD work. This will support activities such as developing plans for spin-outs and commercialization; establishing an IP position; preparing and documenting open-source software or datasets; and developing tourable public experiences.

- ORBIT as an impact partner for RRI. Students will embed findings and methods for Responsible Research Innovation into the national training programme that is delivered by ORBIT, the Observatory for Responsible Research and Innovation in ICT (www.orbit-rri.org). Through our direct partnership with ORBIT all Horizon CDT students will be encouraged to write up their experience of RRI as contributions to ORBIT so as to ensure that their PhD research will not only gain visibility but also inform future RRI training and education. PhD projects that are predominantly in the area of RRI are expected to contribute to new training modules, online tools or other ORBIT services.