REINS

Lead Research Organisation: King's College London
Department Name: Engineering

Abstract

The REINS project is to design and investigate haptic communicational interfaces (reins) between a human agent and a mobile robot guide. The reins will facilitate joint navigation and inspection of a space under conditions of low visibility (occurring frequently in fire fighting). The focus is on haptic and tactile human robot cooperation as it has been found that a limited visual field and obscured cameras adds to the distress of humans working under pressure. Humans naturally interact with animals using tactile feedback in scenarios such as working with guide dogs and horse riding; the REINS project aims to extend this practice to human robot interaction. Expertise from a number of different disciplines - design, engineering, robotics, and communication - will be brought to bear on the problem of designing a communicational interface which will be both sufficiently robust for the relevant physical environment and sufficiently flexible to allow for the on-the-spot exercise of human judgement and creativity.Inspired by the use of a harness for a guide dog and also the rein to ride or drive a horse, the REINS project will investigate and experiment with haptic interfaces for human-robot cooperation. The low/no visibility constraint ensures the focus is on the tactile and haptic aspects only. Currently, robots do not sufficiently enhance human confidence. In human-robot cooperation, the human (by nature) will try to 'read' the situation, and anticipate the movements of the robot companion. The robot is provided with an impedance filter and the rein enables the human to feel the robot's movements and behaviour. Experiences with remotely controlling a robot which is not directly visible show that 'operators spent significantly more time gathering information about the state of the robot and the state of the environment than they did navigating the robot'.The REINS project aims to map the communicational landscape in which humans (fire fighters, but also the visually impaired) might be working with robots, with the emphasis on tactile and haptic interaction. We adapt a semi-autonomous mobile robot for navigation in front of a human. The robot provides rich sensory data and is enabled to try the mechanical impedances of the objects it encounters. We also design and implement a soft rein (rope), a wireless rein and a stiff rein (inspired by the lead for guide dog) enabling the human to use the robot to actively probe objects. The project thus creates the means to explore the haptic Human-Robot Interaction landscape. We will work from an integrationist perspective in which the communicator is not a mere user of pre-existing signs but a sign-maker; the signs emerge in the ongoing coordination and integration of activities adapted to the particular circumstances. We review the communicational landscape occurring within a team of (human) fire fighters and in addition review literature on working guide dogs and horse riding. A research question is whether the information should be explicitly encoded as messages or can remain implicit.In the initial phase of the project the robot is adapted and the first prototypes of the reins are implemented; the emphasis in this phase is on providing rich data to the human. The second phase is dedicated to surveying the communicational landscape. The human-robot team will navigate a known environment with low visibility where unknown obstacles may occur. At least two different types of reins are applied: one requires that messages are explicitly coded, while the other propagates the information implicitly. Based on experiences in the first trials the reins might be adapted to improve their usability. Professional fire fighters will be the first group of subjects to try the reins, later on also volunteers experienced with guide dogs may join the experimentations.
 
Description 1. Human-human demonstrations of guiding showed that the guider uses on average, a 3rd order predictive policy to take current guiding actions based on predicted errors of the follower.

2. The follower uses on average, a 2nd order reactive policy to make current movements based on past commands given by the guider.

3. When the follower's movements in response to tug forces generated by the guider is modeled using a virtual damped inertial model (virtual model because, followers movements largely comes from voluntary forces than due to tug forces of the guider), the variability of the damping coefficient reliably reflects the variability of the follower's trust in the guider.

4. When control policy identified from human demonstration experiments was implemented on a planar 1-DoF robotic arm to perturb the blindfolded subjects' most dominant arm in leftward/rightward directions, it was found that naive subjects elicit a 2nd order reactive behaviour similar to human demonstration experiments. However, trained subjects developed 2nd order predictive following behaviour.

5. Naive and trained subjects' arm muscle activation is significantly different in leftward/rightward arm perturbation.

6. When humans are trained with primitive vibroactuator array patterns given in a sleeve worn in the arm, they can accurately recognize linear combinations of those patterns.


The low/no visibility constraint ensures the focus is on the tactile and haptic aspects only. The research aims to map the communicational landscape in which humans (fire fighters, but also the visually impaired) might be working with robots, with the emphasis on tactile and haptic interaction.



In the initial phase, we conducted human-human interaction experiments to find the salient features where one person with limited auditory and visual perception of the environment ( human), is guided by an agent with full perceptual capabilities (assume as robot/machine), via a hard rein along a given path.



Key findings:



1. We defined the state as the relative orientation difference between the human guider (a guider) and the human follower (a follower), the angle of the rein relative to the guider as an action. Numerical novel auto regressive model of linear discrete control policy of the guider and the follower was proposed. The results show the guider learns 3rd order predictive control policy to guide the human follower, while the follower learns 2nd order reactive model to follow the guider.



2. While learning the control policy, we tried to understand gradual change in muscle activation of the guider across the trials to see optimality of total effort to generate actions in the actuation space by the guider. The results show that the total muscle activation increases to a maximum around the mid of the trials (20 trials, 10 pairs) and then decreases in last 10 trials. This suggests that effort optimization is a non-monotonic process. During the first 10 trials, participants may have given priority to order selection than optimization in the actuation space.



3. To study human (the follower) confidence towards the guiding agent, a confidence model is proposed (to study the virtual damping coefficient and the virtual mass with pushing/pulling force). Experiments were conducted on three different paths (straight / 60 degree turn / 90 degree turn). Our experimental results of human participants show that the variability of the virtual damping coefficients correlates more with the complexity of the path. Furthermore, When the follower drops confidence in 60 degree turn and 90 degree turn, the guider has to exert a higher tug force to take following agent into desired trajectory that leads to higher average values for the virtual mass and virtual damping coefficient.
Exploitation Route At present we are planning to carry out an impact acceleration project in collaboration with the Fire Service College, Gloucestershire. This involves testing an outdoor version of the robot in a simulated fire-fighting scenario.
Sectors Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Electronics

URL http://thrish.org/research-team/anuradha-ranasinghe
 
Description Cultural impact: Contributed to collaboration with King's Cultural Institute to develop a human-robot dancing systems and event.
First Year Of Impact 2015
Sector Creative Economy,Education
Impact Types Cultural

 
Description "The brain is lazy" highlighting the clever ways it takes to condition the body to simplify autonomous control 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact UK/US Bio-inspired Unmanned Autonomous Systems (BioUAS)

Plenary talk by Thrishantha Nanayakkara
Year(s) Of Engagement Activity 2019
 
Description Active search for haptic information in the impedance domain 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact IEEE ICRA2018,

Invited workshop talk by Thrishantha Nanayakkara at IEEE ICRA2018

on "Active search for haptic information in the impedance domain",

Brisbane, Australia
Year(s) Of Engagement Activity 2018
 
Description Brain is the last resort 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact IEEE ICRA2018

Plenary workshop talk

by Thrishantha Nanayakkara at IEEE ICRA2018 on

"Brain is the last resort"

Brisbane, Australia
Year(s) Of Engagement Activity 2018
 
Description Embodied computation of perception and action for fast uncertainty reduction 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Robotics Science and Systems (RSS) 2019,

Plenary workshop talk by Thrishantha Nanayakkara on "Embodied computation of perception and action for fast uncertainty reduction", Freiburg, Germany
Year(s) Of Engagement Activity 2019
 
Description Haptic information gain in the impedance domain 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact IEEE RoboSoft 2018,

Plenary workshop talk by Thrishantha Nanayakkara at IEEE RoboSoft 2018 on

"Haptic information gain in the impedance domain"

Levorno, Italy
Year(s) Of Engagement Activity 2018
 
Description If robots were to survive like living beings in uncertain environments 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact IMechE Christmas event,

Keynote talk by Thrishantha Nanayakkara at IMechE Christmas event -

"Tinkering" on "If robots were to survive like living beings in uncertain environments",

London, UK
Year(s) Of Engagement Activity 2017
 
Description Morphological Computation Through Physical Adaptation of Soft Robots 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact IEEE RoboSoft 2019

Workshop organizer and workshop talk by Thrishantha Nanayakkara

at IEEE RoboSoft 2019 on "Morphological Computation Through Physical Adaptation of Soft Robots"

Seoul, South Korea
Year(s) Of Engagement Activity 2019
 
Description Morphological computation - The role of the physical circuits in the body in the computation of perception and action 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Jilin University,

Keynote talk by Thrishantha Nanayakkara at International Workshop

on Bioinspiration on "Morphological computation - The role of the physical circuits in the body in the computation of perception and action",

Jilin, China
Year(s) Of Engagement Activity 2018
 
Description Soft Robotic Patients for Medical Education 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact RCGP-2018,

Workshop keynote talk by Thrishantha Nanayakkara

at Royal College of General Practitioners annual conference

on "Soft Robotic Patients for Medical Education",

Glasgow, UK
Year(s) Of Engagement Activity 2018
 
Description Soft Robotics for Rehabilitation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Xuzhuo Central Hospital,

Keynote talk by Thrishantha Nanayakkara

at Xuzhou Central Hospital on

"Soft Robotics for Rehabilitation",

Xuzhou, China
Year(s) Of Engagement Activity 2018
 
Description Soft robotics for sustainable rainforest communities 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other audiences
Results and Impact LIPI workshop,

Keynote talk by Thrishantha Nanayakkara

at Indonesia Institute of Science and technology

on "Soft robotics for sustainable rainforest communities",

Bandung, Indonesia
Year(s) Of Engagement Activity 2017
 
Description Survival in uncertain environments with slow communication pathways - evidence from morphological computation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact IEEE PIMRC,

Plenary keynote talk by Thrishantha Nanayakkara

at IEEE PIMRC on "Survival in uncertain environments with slow communication pathways - evidence from morphological computation",

Montreal, Canada
Year(s) Of Engagement Activity 2017
 
Description Surviving in an uncertain world with slow communication pathways 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact University of Leeds,

Keynote by Thrishantha Nanayakkara

on "Surviving in an uncertain world with slow communication pathways",

Leeds, UK
Year(s) Of Engagement Activity 2018
 
Description Surviving with slow communication pathways 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact 2nd UK manipulation workshop,

Invited talk by Thrishantha Nanayakkara on

"Surviving with slow communication pathways",

London, UK
Year(s) Of Engagement Activity 2017
 
Description Towards the nature of information processing in soft machines 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact IEEE RoboSoft 2019

Plenary workshop talk by Thrishantha Nanayakkara at IEEE RoboSoft 2019 on "Towards the nature of information processing in soft machines"

Seoul, South Korea
Year(s) Of Engagement Activity 2019
 
Description Trespass public event 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Sponsored by the King's Cultural Institute, I worked with a choreographer to understand the parallels between programming the human body and programming robots to explore new meaning and opportunities in both fields. A public exposition was held on 31st March 2013, titled "translocations" (http://www.shobanajeyasingh.co.uk/works/translocations/ ), where my views were subjected to public discussion. This led to a wider collaboration including Bartlett School Architecture at UCL, and the above cultural partners to showcase human-robot dancing in a public exhibition titled "Trespass" (https://vimeo.com/133691929). I participated in a panel moderated by New Scientist journalist Simon Ings.
Year(s) Of Engagement Activity 2015
URL https://vimeo.com/133691929
 
Description Workshop on "Haptics for Healthcare" 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact ERU-2017,

Co-organizer and speaker in the Workshop on "Haptics for Healthcare",

Edinburgh, UK
Year(s) Of Engagement Activity 2017