COMMOTIONS: Computational Models of Traffic Interactions for Testing of Automated Vehicles

Lead Research Organisation: University of Leeds
Department Name: Institute for Transport Studies

Abstract

As automated vehicles (AVs) are being developed for driving in increasingly complex and diverse traffic environments, it becomes increasingly difficult to comprehensively test that the AVs always behave in ways that are safe and acceptable to human road users. There is wide consensus that a key part of the solution to this problem will be the use of virtual traffic simulations, where simulated versions of an AV under development can meet simulated surrounding traffic. Such simulations could in theory cover vast ranges of possible scenarios, including both routine and more safety-critical interactions. However, the current understanding and models of human road user behaviour is not good enough to permit realistic simulations of traffic interactions at the level of detail needed for such testing to be meaningful. This fellowship aims to develop the missing simulation models of human behaviour, to ensure that development of the future automated transport system can be carried out in a responsible, human-centric way.

Behaviour of car drivers and pedestrians will be observed both in real traffic as well as in controlled studies in driving and pedestrian simulators, in some cases complementing behavioural data with neurophysiological (EEG) data, since several candidate component models make specific predictions about brain activity. The fellowship will then build on existing models of driver and pedestrian behaviour in routine and safety-critical situations, and extend these with state of the art neuroscientific models of specific phenomena like perceptual judgments, beliefs about others' intentions, and communication, to create an integrated cognitive modelling framework allowing simulations of traffic interactions across a variety of targeted scenarios.

Such cognitive interaction models, based on well-understood underlying mechanisms, will be one main contribution from the fellowship. Some researchers have suggested the use of another type of model altogether, instead obtained directly by applying machine learning (ML) methods to large data sets of human road user behaviour, i.e., without an ambition to correctly model underlying mechanisms. This fellowship hypothesises that to achieve reliable virtual testing of AVs, both types of modelling approaches will be needed, and methods for combining them will be researched. Not least, due to their "black box" nature, ML models need to be investigated and benchmarked, to for example determine their ability to generalise to rare, safety-critical events.

The multi-disciplinary research, building on and extending on the fellow's past experience in vehicle engineering, cognitive neuroscience, and machine learning, will be carried out at the Institute for Transport Studies, University of Leeds, with support also from the Schools of Psychology and Computing. The fellowship has direct support from industry, both in advisory capacities and as project partners actively sharing data and methods as well as providing first proof-of-concept uptake of the developed models into industrial environments for simulated testing.

Planned Impact

The primary, long-term impact that the project envisions, and which it will actively work towards, is the development and successful deployment of safe and acceptable automated vehicles (AVs). There are large hoped-for economical and societal benefits from transport automation. Long-term, the global economy for AVs and AV-enabled services is projected to be worth trillions of pounds per year, and the UK government is targeting a leading role for the UK in this economy, with an estimated £51 billion annual benefit for the UK economy and 300,000 new jobs by 2030, as well as reductions in road traffic injuries and death, improved inclusive mobility, reduced congestion, and increased productivity.

However, a primarily technology-driven approach to automation, without proper consideration of human behaviour, risks resulting in AVs that behave in ways that are unappreciated by, and potentially unsafe to human road users. If AVs for example cause traffic jams because they are overly cautious, or misinterpret human road user behaviour in ways that lead to crashes, public acceptance and market penetration will suffer, which could in turn severely limit the abovementioned potential benefits. The human behaviour models and virtual testing simulations developed by this project will help mitigate against these risks, by providing a direct means of supporting human-centred, responsible innovation on vehicle automation, to develop AV technology that puts human behaviour, capabilities and well-being first. The project therefore holds promise of impact both at the level of the individual UK citizen, in terms of a safer and more desirable urban road traffic environment, as well as on the national level, giving the UK industry (vehicle manufacturers, suppliers, simulation tool developers, ...) and economy an edge over competitors, and as a result a greater share for the UK of the global market for automation.

Early impacts are expected already while the fellowship is active (2019-2023), in terms of first proofs of concept of the developed models in industrial simulation tools, as well as a raised awareness of the need for proper consideration of human road user behaviours in testing of AVs, among industry, general public, and policy makers. In the first years after the fellowship, this can in turn help drive policy-making on AV testing requirements, while in parallel the models should start to see actual use as part of industrial development processes. This will in turn support larger-scale deployment of safe and acceptable AVs in urban traffic in the UK and elsewhere, by current estimates circa 2025-2035.

Given that the project touches on a wide range of applied disciplines beyond just road vehicle automation, there are many more potential industrial and societal impacts. For example, transport planners can make use of improved traffic simulation tools to make better decisions on public spending on road traffic infrastructure. Furthermore, better models of human interactive locomotion and human situational awareness can be useful also outside the road traffic context, for example in the design of robots locomoting among humans, and of safety-critical environments like aircraft or nuclear power plants.
 
Description One main achievement so far has been the definition and publication of a conceptual framework -- a structured way of talking about -- interactions betweens humans, and between humans and automated vehicles, in road traffic. This framework has seen substantial adoption by researchers in this field, and has already been useful for this specific project's goal of developing mathematical models of human road user behaviour.

Another key finding is that human behaviour and brain responses during detection of collision threats (such for example in traffic) can be described as being determined by evidence accumulation - a type of decision-making mechanism that has previously been studied mainly for more abstract laboratory-based tasks. This provides a further solid foundation for the models being developed in this project, and opens many new research directions both within road traffic research, and within human decision-making research in general.

We have also found that evidence accumulation models work well for describing human decision-making also in specific interactive situations in road traffic, and these models are currently being extended and developed into a more complete modelling framework for addressing a wider range of interaction scenarios.
Exploitation Route The mentioned conceptual framework provides theoretical structure for (1) academic or industry investigations into how humans interact with automated vehicles, and for (2) formulation of clear and appropriate requirements on the interactive capabilities of automated vehicles. The mentioned models of collision threat detection are useful to traffic safety research and development in industry and academia, as well as to traffic accident litigation (our findings refute a previously dominant account, which has been leveraged by expert witnesses and similar). The developed mathematical models of interactions can be adopted by industry and academia, for example to generate simulated testing environments for automated vehicles. We are pursuing such collaborations with several partners.
Sectors Digital/Communication/Information Technologies (including Software),Manufacturing, including Industrial Biotechology,Transport,Other

 
Description So far, we have been able to leverage the project to gain access to various standardisation fora where discussions are currently happening that are important for how automated vehicles will be tested going forward, and into which the project's outputs can fit nicely, later on. Both in these fora, and in discussions with industry stakeholders regarding the project, we have been able to raise awareness of why there is a need for the type of human behaviour models that the project is targeting. We have also provided concrete input into standardisation documents in development.
First Year Of Impact 2019
Sector Digital/Communication/Information Technologies (including Software),Manufacturing, including Industrial Biotechology,Transport,Other
Impact Types Societal

 
Description Scientific advisor to automotive industry (Volvo, Nissan, Waymo) in recurring meetings on road user behaviour modelling
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a advisory committee
 
Description Shaping ISO Technical Specification on simulation-based safety testing
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a advisory committee
 
Description Society of Automotive Engineers Automated Driving Simulation Task Force
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a advisory committee
 
Description COVID 19 Grant Extension Allocation
Amount £23,821 (GBP)
Organisation United Kingdom Research and Innovation 
Sector Public
Country United Kingdom
Start 01/2021 
End 09/2021
 
Description Two PhD studentships sponsored by Nissan
Amount £170,000 (GBP)
Organisation Nissan Motor Manufacturing Ltd 
Sector Private
Country United Kingdom
Start 01/2020 
End 12/2023
 
Title Collision threat detection dataset 
Description Primary research data (behavioural responses and electroencephalography), from the collision threat detection study described in this paper: Markkula G, Uludag Z, Wilkie R M, Billington J. 2020. Accumulation of continuously time-varying sensory evidence constrains neural and behavioral responses in human collision threat detection. PsyArXiv preprint: https://doi.org/10.31234/osf.io/ca3h9 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact No impacts yet, but the findings have applied and societal impacts in terms of refuting a widely used assumption about human collision threat detection, which has been used in traffic safety research and development, as well as in traffic accident litigation. 
URL https://osf.io/ku3h4/
 
Title Model of human collision threat detection 
Description The collision threat detection model described in this paper: Markkula G, Uludag Z, Wilkie R M, Billington J. 2020. Accumulation of continuously time-varying sensory evidence constrains neural and behavioral responses in human collision threat detection. PsyArXiv preprint: https://doi.org/10.31234/osf.io/ca3h9 
Type Of Material Computer model/algorithm 
Year Produced 2020 
Provided To Others? Yes  
Impact No known impacts yet, but the model has applied and societal impacts in terms of replacing a previously widely used assumption about human collision threat detection, which has been used in traffic safety research and development, as well as in traffic accident litigation. 
URL https://github.com/gmarkkula/LoomingDetectionStudy
 
Description Collaboration with TU Delft AiTech 
Organisation Delft University of Technology (TU Delft)
Country Netherlands 
Sector Academic/University 
PI Contribution Researchers in the TU Delft project AiTech are pursuing similar objectives to COMMOTIONS. We have organised several information workshops with them between our respective groups, are planning research visits (when the pandemic permits) etc. We are also working toward co-authored publications.
Collaborator Contribution Input to discussion and collaboration as mentioned above.
Impact See Engagement Activities: A talk or presentation - Presentation at TU Delft AiTech Agora: "Modeling human-AV interactions for safety and acceptance of automated vehicles" Our first joint paper is also available as a preprint: https://psyarxiv.com/p8dxn/
Start Year 2019
 
Description Collaboration with US project "Modeling driver behavior during platooning failures" 
Organisation Texas A&M University
Country United States 
Sector Academic/University 
PI Contribution Advising this SAFE-D-funded project on driver behaviour modelling.
Collaborator Contribution Wider application of the types of models researched in COMMOTIONS, thus increasing our understanding of these models.
Impact Paper DOI: 10.1016/j.aap.2021.106055 See also the project report: https://trid.trb.org/view/1765408
Start Year 2019
 
Description Collaboration with US project "Modeling driver behavior during platooning failures" 
Organisation Virginia Tech
Department Transportation Institution
Country United States 
Sector Academic/University 
PI Contribution Advising this SAFE-D-funded project on driver behaviour modelling.
Collaborator Contribution Wider application of the types of models researched in COMMOTIONS, thus increasing our understanding of these models.
Impact Paper DOI: 10.1016/j.aap.2021.106055 See also the project report: https://trid.trb.org/view/1765408
Start Year 2019
 
Description Circulation and discussion of project-produced "green paper" 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact The "green paper" produced by the project (see Publications) was circulated via e-mail lists, Twitter, and at conferences, to raise awareness of the project and elicit feedback on the outlined project approach. These objectives were both achieved.
Year(s) Of Engagement Activity 2019
URL https://osf.io/vbcaz
 
Description Presentation at MathPsych 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Presentation at the annual meeting of the Mathematical Psychology Society, Montreal, Canada: "Mathematical psychology in the wild - why and how?Insights from applying basic modelling concepts to applied problems in traffic safety and self-driving cars"
Year(s) Of Engagement Activity 2019
 
Description Presentation at SHIFT Mobility: "Self-Driving priorities: Building robots or understanding humans?" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Hybrid/online event as part of IFA Berlin.
Year(s) Of Engagement Activity 2020
URL https://xtended.ifa-berlin.com/eventgrid/stage/7/110
 
Description Presentation at TU Delft AiTech Agora: "Modeling human-AV interactions for safety and acceptance of automated vehicles" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation and discussion at the TU Delft AiTech Agora (https://www.tudelft.nl/aitech/agora). Led on to a couple of follow-on discussions with researchers, currently developing into more substantial collaborations.
Year(s) Of Engagement Activity 2020
URL https://www.youtube.com/watch?v=nRCbKFK2b2A