REINS

Lead Research Organisation: Sheffield Hallam University
Department Name: Faculty of Arts Computing Eng and Sci

Abstract

The REINS project is to design and investigate haptic communicational interfaces (reins) between a human agent and a mobile robot guide. The reins will facilitate joint navigation and inspection of a space under conditions of low visibility (occurring frequently in fire fighting). The focus is on haptic and tactile human robot cooperation as it has been found that a limited visual field and obscured cameras adds to the distress of humans working under pressure. Humans naturally interact with animals using tactile feedback in scenarios such as working with guide dogs and horse riding; the REINS project aims to extend this practice to human robot interaction. Expertise from a number of different disciplines - design, engineering, robotics, and communication - will be brought to bear on the problem of designing a communicational interface which will be both sufficiently robust for the relevant physical environment and sufficiently flexible to allow for the on-the-spot exercise of human judgement and creativity.Inspired by the use of a harness for a guide dog and also the rein to ride or drive a horse, the REINS project will investigate and experiment with haptic interfaces for human-robot cooperation. The low/no visibility constraint ensures the focus is on the tactile and haptic aspects only. Currently, robots do not sufficiently enhance human confidence. In human-robot cooperation, the human (by nature) will try to 'read' the situation, and anticipate the movements of the robot companion. The robot is provided with an impedance filter and the rein enables the human to feel the robot's movements and behaviour. Experiences with remotely controlling a robot which is not directly visible show that 'operators spent significantly more time gathering information about the state of the robot and the state of the environment than they did navigating the robot'.The REINS project aims to map the communicational landscape in which humans (fire fighters, but also the visually impaired) might be working with robots, with the emphasis on tactile and haptic interaction. We adapt a semi-autonomous mobile robot for navigation in front of a human. The robot provides rich sensory data and is enabled to try the mechanical impedances of the objects it encounters. We also design and implement a soft rein (rope), a wireless rein and a stiff rein (inspired by the lead for guide dog) enabling the human to use the robot to actively probe objects. The project thus creates the means to explore the haptic Human-Robot Interaction landscape. We will work from an integrationist perspective in which the communicator is not a mere user of pre-existing signs but a sign-maker; the signs emerge in the ongoing coordination and integration of activities adapted to the particular circumstances. We review the communicational landscape occurring within a team of (human) fire fighters and in addition review literature on working guide dogs and horse riding. A research question is whether the information should be explicitly encoded as messages or can remain implicit.In the initial phase of the project the robot is adapted and the first prototypes of the reins are implemented; the emphasis in this phase is on providing rich data to the human. The second phase is dedicated to surveying the communicational landscape. The human-robot team will navigate a known environment with low visibility where unknown obstacles may occur. At least two different types of reins are applied: one requires that messages are explicitly coded, while the other propagates the information implicitly. Based on experiences in the first trials the reins might be adapted to improve their usability. Professional fire fighters will be the first group of subjects to try the reins, later on also volunteers experienced with guide dogs may join the experimentations.

Planned Impact

The REINS project studies tactile and haptic interaction in low visibility conditions. The project focuses on the emergence of the navigational abilities of a human robot team, much of it will depend on the confidence the human has in the robot. WHO: South Yorkshire Fire and Rescue (letter of support) are involved in the project. The project studies fire fighting practice and performs experiments targeted at a fire fighting environment. A direct impact is that fire fighters become better aware of their (implicit) practices. In case the project succeeds and robots are indeed applied, they will reduce the risks for fire fighters and thus enhance and improve the rescue of citizens and thus safety of society. Fire brigades within the UK cooperate on a very amicable level, colleagues meet at the national Fire Service College, also regularly regional and national events are organised and informally colleagues exchange their latest experiences. Thus possibly applicable knowledge obtained in REINS will propagate to a national level. The consortium engages with the Guide Dogs for the Blind Association (letter of support). When the technology has matured (and REINS will contribute to that) robots will become useful to assist people with handicaps in their daily routines. Moreover, robots could generally support patients and carers or even takeover certain caring tasks. The consortium engages with Thales NL/UK, manufactures in electronic equipment (letter of support). Direct human robot interaction is still in its infancy. Thales is looking for new ways for interfacing people and machines; new technologies 'for humans-in-the-loop of complex systems-of-systems'. The human factors research of the Thales group is based in the Netherlands, however when usable the knowledge passes on to products and devices (including robots) developed by Thales UK. Short term South Yorkshire Fire and Rescue (SYFR) are very happy to facilitate and assist in the experimental work of the Reins project. SYFR have cooperated with Sheffield Hallam in the Guardians and ViewFinder projects. Reflecting on this they stated: 'As a Fire Brigade we do not have the means to be at the forefront of science and technology developments. However, we are looking at technology to help us improve our service. Being involved in these projects has made our officers better aware of available and up-coming technologies and directly helps us when evaluation new FireFighting technology'. When and if appropriate the Guide Dogs for the Blind Association has offered to arrange volunteers to test a robotic solution as an alternative for a guide dog. Long term In fire fighting, risk assessments is extremely important but very time consuming. Robots with a mobile detection unit would certainly speed this up also more risk can be taken and contamination and in particular decontamination would not be as big an issue as it is related to human beings. The application of robots in Fire and Rescue operations will improve data collection and thus speed-up and improve decision making and therefore adds to saving lives but also to saving property. Appropriate human robot interfaces are essential for introducing robots into daily human life. once this issue is reasonably solved robots can be applied for (partially) handicapped people and also for the elderly. The Guide Dogs for the Blind Association is keen to evaluate the developed solutions for this purpose. Staff development The PDRAs and PhD students will work together with professional fire fighters as their end-users. Thus they will have to listen to and try to understand the viewpoints of people who are non-professionals in the scientific/technical field of the PDRAs/PhDs. Thus they develop skills and experiences applicable in any situation where knowledge and high tech are to be transferred to society.
 
Description The aim of the project is to understand how a human is guided by another intelligent agent when the vision is impaired. In that scenario, haptics becomes a natural solution. Therefore KCL presents identification of abstracted dynamics of human control policies and human responses in haptic perceptions.
The SHU team aims at exploring how a human and a semi-autonomous robot can develop cooperation using a haptic interface in low visibility conditions.

1) Human-human guidance (KCL)

Follower-Model Generation
First, Experiments were conducted to extract the control policies in guiding and following when a blindfolded human is guided by another human by using a hard rein. The key findings from human demonstration experiments are,
? By modelling how state maps into action by a Nth order state dependent discrete linear controller, it was found that the guider gave more emphasis on 3rd order predictive model and the follower gave more emphasis on 2nd order reactive model. The model order and reactive/predictive nature will give an insight of human behaviour in guiding and following when the vision is impaired.
? Studying of how the guider may modulate the pulling force in response to the confidence level of the follower in order to study how the above control policy interact with the follower in an arbitrary path tracking task is necessary to understand the follower's trust on the guider. Modelling the voluntary movements of the follower as a damped inertial system. It was found that damping coefficient is most sensitive to explain the confidence level of the follower.

Follower-Model Validation
Secondly, extracted control policies were implemented on planner 1-DoF robotic arm to test its validity and stability. Moreover the human response patterns and behavioural characteristic were studied when the blindfolded human's most dominant arm was perturbed by the robotic arm to guide them in to a desired angular position. They key findings from human robot demonstration experiments are,
? Studying the human movements in leftward and rightward directions convey how humans understand the perceiving information to move another point when the robotic arm uses to generate a tug force to perturb the humans' most dominant arm in low visibility conditions. The asymmetry was noticed in human behavioural metrics in leftward/rightward movements. The extracted behavioural metrics present characteristics and limitations of human movements when the haptic perturbation is given from different directions.
? Haptic perception of external perturbations depends on involuntary muscle recruitment patterns to stabilize the hand.

2) Robot-Human Guidance (SHU)

Inspired by how a visually impaired person is collaborating with a guide dog, we distinguish between the navigation task and the task of guiding.
The guide dog just guides while the human being makes the navigation decisions (which direction to go to reach the goal). In addition, exploration of the (unknown) environment is an additional task in search and rescue.

The focus of the REINS project is on locomotion guidance - the (autonomous) robot leading the human along a safe path - and the human using the robot for exploring the close by environment.

The project has experimented with 1) a wireless rein, 2) a flexible rein (rope) and 3) a stiff rein.
A basic requirement for guidance as well as exploration is that the human being 'knows' where the robot is. A stiff rein provides that information implicitly. If a flexible rein is tense it also provides a good feeling (implicit information) of direction however when slack occurs there is no feedback. A wireless rein does not provide any implicit information; all feedback (including the position of the robot) has to be made explicit.

The project continues to investigate using a fixed rein, which not necessarily requires a set of a-priory fixed codes. We aim for so-called non-performative feedback; we are not looking for coded messages, a coded message would be for example: 'two taps mean go right'.
Exploitation Route Most promising is to use and apply our findings for devices that are being used in normal conditions (visibility), for instance to assist elderly people.

We gave some demonstrations for people diagnosed with dementia; they turned out to be very receptive to the idea of allowing a robot in their domestic/residential space. (Our initial expectation was that they would be reluctant and negative.)

Potential for collaboration with China on the dementia theme.
Sectors Healthcare,Security and Diplomacy

 
Description Apply similar technology for supporting patients with dementia (see added publication). Potential for collaboration with China.
First Year Of Impact 2019
Sector Agriculture, Food and Drink,Healthcare
Impact Types Societal

 
Description Action for Involvement, talk and expert for discussion 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? Yes
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Lively discussions on whether people wanted to be assisted by a robot

Organiser intend to organise a more assistive robotics for the elderly event
Year(s) Of Engagement Activity 2014
 
Description Invited talk at the RE.WORK Technology Summit 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Several lengthy discussions with participants

No direct impacts yet
Year(s) Of Engagement Activity 2014
 
Description Invited talk, British Computing Society, Yorkshire branch 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Some lively discussion afterwards.

No direct impacts
Year(s) Of Engagement Activity 2014
 
Description Invited talk: 'Haptic Interfaces for a Robot Guide' for Science & Technology Group of the Sheffield University of the Third Age 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact About 30 mostly retired scientists,



One lady (not retired) came back with a request for discussing a new project.
Year(s) Of Engagement Activity 2014
 
Description Search and Rescue Robotics @ Sheffield Hallam University 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Participants in your research or patient groups
Results and Impact Presentation to a Japanese robotics delegation organised by Industrial Maths KTN.
Year(s) Of Engagement Activity 2012
 
Description Visit to primary School, group 4 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact 85 pupils attended the talk (5 minutes) and the demonstrations.


The children were very interested in the robots as long as they did not get too close to them.
Year(s) Of Engagement Activity 2012