iSeat - Towards an intelligent driver seat for autonomous cars

Lead Research Organisation: University College London
Department Name: Civil Environmental and Geomatic Eng

Abstract

According to the most optimistic predictions, the first commercially available fully-autonomous cars are expected in 2040 offering the consumer a full end-to-end journey. These self-driving vehicles will be equipped with technology allowing autonomy Level 5 in which there is no interference required by the human. The concept of autonomy levels was first published by the international Society of Automotive Engineers in 2014. The report defines six levels of autonomy that automakers would need to achieve on their way to building the no-steering-wheel self-driving bubble pods of the future reaching from the fully-manual Level 0 to the fully-autonomous Level 5.

In the race towards the first commercially available fully-autonomous car, the majority of cars on UK roads will be equipped with technology that allows Level 3 or 4 autonomy over the next two decades. Drivers will be provided with increasingly sophisticated features such as lane-keep and steering assist. These semi-autonomous cars might be able to transport the driver autonomously on sections of a journey. However, the driver is required to take control occasionally between different levels of autonomy when required to complete an end-to-end journey. These transitions between autonomy levels cause safety concerns, as the driver might not be fully aware of the surrounding situation and the enabled autonomy features instantly.

This project proposes a new interface design for semi-autonomous cars called iSeat. This system is fundamentally different compared to current systems (such as Tesla's Autopilot or DistronicPlus by Mercedes) using visual or auditory indications which might be mentally overloading and distracting for the driver. iSeat is an intelligent driver seat acting as a co-pilot measuring the current mental and physical engagement of the driver and allowing safe, coordinated and timely transitions between different levels of autonomy. Of particular significance is the driver seat made of robotic structures serving the feedback purpose as well as providing monitoring capabilities through direct contact with the human during any level of autonomy: Tactile sensation can be fed back to the driver, the seat ergonomics and comfort can be changed and the robotic structures can measure the pressure distribution of the driver's weight. iSeat sensing information will be fused with multi-modal sensing data from electrical activity produced by skeletal muscles (Electromyography (EMG) signals) and in the driver's brain (Electroencephalography (EEG) signals), and input from vision cameras regarding the driver's posture and the point of gaze (i.e. where the driver is looking). This real-time knowledge will be classified through machine learning and affective interaction techniques in terms of the awareness state of the driver. Personalised feedback will be provided (i.e. tactile sensation, stiffness feedback, change of the driver seat ergonomics/comfort, visual/auditory feedback) to support the driver so that safe, timely, effective and intuitive transitions between different autonomy levels can be completed.

The iSeat system builds upon a complete re-think of the manner in which humans interact with autonomous cars. The smart combination of sensor systems, machine learning, affective computing, human factors, haptics and robotics will result in a bi-directional human-machine cooperation that is safe, intuitive, effective, and personalised.

Planned Impact

Who will benefit from the project?

This outcomes of this project will benefit (i) the general public, i.e. people who will travel in autonomous cars that cannot offer full end-to-end journey, (ii) the UK industry, i.e. carmakers, aiming at producing autonomous cars, and the technology industry, that supplies carmakers with systems that will be integrated into cars and (iii) academics in the area of human factors, robotics and transport engineering. Other major beneficiaries will be policy makers who will investigate ways to standardise procedures in autonomous cars to ensure safety.

How will they benefit from the project?

The general public - Autonomous vehicles will transform the transportation industry beyond the automotive and technology industries and affect delivery services and public transport even in remote areas and dispersed communities. An autonomous vehicle will certainly be, for the majority of people, the first "autonomous robot" that they are interacting with on a daily basis and that might make decisions through AI that can be life-threatening. To overcome any lack of trust and confidence in these autonomous robots, which might lead to any constraints of public acceptance, iSeat addresses human-machine cooperation risk factors and introduces intelligent sensing and intuitive (haptic) feedback technologies into the driving environment.

The UK industry - The annual economic benefits of autonomous cars is predicted to grow to £51 and £121 billion by 2030 and 2040 respectively. Reliability and safety concerns can be extremely damaging for carmakers. For instance, after Tesla's Autopilot crash in 2016, new tighter regulations were released on testing automation features. Tighter regulation could slow the development and introduction of new technology. In worse cases, new developments might be aborted due to high-risk commercialisation factors. Further, autonomous cars are transformative for the automotive industry at large - acting as a catalyst for new products and technology. The introduction of autonomous cars will revolutionise safety. iSeat aims at reducing safety risks involved in the process of taking back control from a car or vice versa, which will lead to direct economic impacts to various road industries such as the automotive, coach, trucking, ground shipping, as well as, other transportation sectors such as aviation, maritime and rail. This will help the UK to become world-leading in the development of technological products and to attract experts for human-machine cooperation of autonomous cars.

Academics - Due to the high-risk nature of the proposed work, this project will primarily produce new knowledge and understanding on how various feedback modalities can support drivers to gain the right level of engagement/vigilance, optimally complete procedural steps between different levels of autonomy, and efficiently get back into the control loop in various driving situations. The knowledge gained throughout this project will be beneficial for other application areas such as the integration of collaborative robots that work closely together with the human in industrial settings.

Publications

10 25 50
publication icon
Gandarias J (2020) Open-Loop Position Control in Collaborative, Modular Variable-Stiffness-Link (VSL) Robots in IEEE Robotics and Automation Letters

 
Description This project effectively started on 2 Jan 2019 after 2 rounds of recruitment (interview process). So far, we have developed a reconfigurable structure as a haptic feedback interface in a driver seat to guide drivers during transition between different levels of autonomy. We have made a review on existing situation awareness and detection systems for transition between autonomy level in (highly) automated vehicles. We have also collected and analysed data on situational awareness and currently we are testing multiple models to establish a relationship between signals received from a human brain and situational awareness.
Exploitation Route We are currently exploring a number of robotic origami systems that can be used as haptic feedback actuators informing the driver of transitions between different levels of autonomy. This haptic feedback system may also be used to inform drivers of the current autonomy level of the vehicle.

We have analysed data about driver Situation Awareness (SA) detection systems, classified equipment and approaches to measure SA, and reviewed all accidents of highly-automated vehicles in California, USA. The outcomes will define scenarios that will need to be designed to validate new technologies for highly-automated vehicles. Also, it will guide research and development in other sectors such as manufacturing industry where SA is critical.
Sectors Digital/Communication/Information Technologies (including Software),Education,Healthcare,Transport

 
Description Expert for new funding programme for the Federal Ministry of Education and Research (Germany)
Geographic Reach National 
Policy Influence Type Participation in a advisory committee
Impact n/a
 
Description Capital Award in Support of Early Career Researchers at UCL
Amount £425,000 (GBP)
Funding ID EP/S01800X/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2018 
End 03/2020
 
Description DfT Transport-Technology Research Innovation (T-TRIG) Grant - GAIN
Amount £2,995,859 (GBP)
Organisation Department of Transport 
Sector Public
Country United Kingdom
Start 01/2020 
End 06/2020
 
Description EIT Urban Mobility - AI-TraWell
Amount € 401,463 (EUR)
Funding ID 128547 
Organisation European Institute of Innovation and Technology (EIT) 
Sector Public
Country Hungary
Start 01/2020 
End 12/2020
 
Description Performing Place - Trellis programme - EPSRC Impact Acceleration Account - The Dancing Brain
Amount £8,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2020 
End 05/2020
 
Description Two Departmental PhD funding
Amount £140,000 (GBP)
Organisation University College London 
Sector Academic/University
Country United Kingdom
Start 03/2020 
 
Description In-kind contribution of a car 
Organisation Honda Of The UK Manufacturing Ltd
Country United Kingdom 
Sector Private 
PI Contribution ideas/knowledge
Collaborator Contribution An in-kind contribution of a Honda HRV to UCL
Impact - Building a stationary Driver-in-the-Loop car simulator (Sigma series) at UCL in collaboration with Ansible Motion Ltd (Collaboration between Mechanical Eng. and Transport Eng.) - Successful funding application: Securing another research grant from DfT Search Transport-Technology Research Innovation Grants ((Collaboration between Mechanical Eng., Transport Eng. and Human factors psychology, Computer Science)
Start Year 2019
 
Description Knowledge exchange with Ansible Motion Ltd 
Organisation Ansible Motion
Country United Kingdom 
Sector Private 
PI Contribution Ideas and knowledge exchange especially on: - Building a simulation environment using Unity - A robotic seat to provide haptic feedback to drivers
Collaborator Contribution Ideas and knowledge on the current practice and future avenues of Driver-in-the Loop simulators around the world
Impact Collaboration Successful funding application
Start Year 2018
 
Description Public engagement in collaboration with Akamedi 
Organisation Akademi
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Knowledge, ideas and equipment
Collaborator Contribution Contacts, knowledge and ideas
Impact Public engagement and outreach Successful funding application: The Dancing Brain funded by EPSRC Impact Acceleration Account
Start Year 2019
 
Description Research collaboration with MIRA Ltd 
Organisation MIRA Ltd
Country United Kingdom 
Sector Private 
PI Contribution Use of equipment and models
Collaborator Contribution Knowledge and collaborative research ideas
Impact Potential collaboration: - Identified potential joint PhD studies - Joint proposal writing
Start Year 2019
 
Description Five Invited Talks 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact 1.
2. Overview of the Development of Technology of CAV and Drones (University of Oxford)
3. Implementation and Impact of CAV Drones for Cities (University of Oxford)
4. Intuitive Haptic Feedback for Take Over Tasks in AVs (UCL Energy Institute)
5. Intelligent Robots and Systems (IROS 2018 Workshop on Shape Changing Robotic Structures and Interfaces)
Year(s) Of Engagement Activity 2018,2019,2020
 
Description Four Invited Panels 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact 1. Accelerating CAV deployment - Test-beds, Pilots and Demonstrators (IDEALondon/Capital Enterprise)
2. Digital Cities Networking Event - The Drive to Sustainability (EIT Digital)
3. Workshop on the UK Connected and Automated Roadmap to 2030 (Zenzic (formerly known as Meridian))
4. The London Automotive Forum (KTN & TfL)
Year(s) Of Engagement Activity 2019,2020
 
Description Invited talk - Inherently Soft - Inherently Safe: Robotics Research at UCL, University of Toronto Robotics Institute, Canada 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact n/a
Year(s) Of Engagement Activity 2019
 
Description Invited talk - Morphing between Levels of Situational Awareness in Highly-automated Vehicles, ICRA 2019 Workshop on Soft Haptic Interaction: Modelling, Design and Application, Montreal, Canada 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact n/a
Year(s) Of Engagement Activity 2019
 
Description Invited talk - Soft Material Robotics: Inherently Soft - Inherently Safe, Brunel Robotics Engineering Society, Brunel University, London, UK 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact n/a
Year(s) Of Engagement Activity 2019
 
Description Keynote Talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Trends in Urbanization, Smart Cities and Transport (BMW Summer School)
Year(s) Of Engagement Activity 2019