Theme 3: Driving Simulation

Lead Research Organisation: University of Leeds
Department Name: Institute for Transport Studies

Abstract

Comparable to other vehicle manufacturers, at the initial prototype stage Jaguar Land Rover (JLR) undertakes some 30,000 individual assessments to evaluate the robustness of its vehicle design through the Design Verification Programme (DVP). The Programme for Simulation Innovation (PSI) is fundamentally targeted at improving capability to support such a DVP through testing in a virtual environment. The development of advanced simulation capability along with corresponding virtual processes, methods and tools can support the delivery of new products and concepts to market more quickly and efficiently through the provision of feedback as early as possible into the vehicle design process. If the vision of zero physical prototypes is to be realised, then the process of digital design and verification needs to accurately represent not just the physical dimensions of component and system function, but also the perceptual experience from the driver's and passenger's perspective. Furthermore, the digital design process must be underpinned by a robust development and validation programme. These ambitions drive the academic focus and industrial impact of this Theme in the development of driving simulation as an integral part of PSI.

The Theme will develop a modular simulator environment to support virtual prototyping, in which performance characteristics and the interfaces between the vehicle and its occupants can be manipulated. Simulation enjoys many benefits over naturalistic studies with its main advantage being considerable versatility in the configuration of virtual scenarios that exactly match the requirement of a particular verification test. Environmental conditions such as day/night operation, weather conditions and state of the road surface can be changed at will. The parameters of the driven vehicle can be altered: for example suspension design, tyre construction and steering characteristics can be matched to an existing or prototype vehicle. Vehicle interiors, driver support systems and ergonomic design can be evaluated in an inherently safe and repeatable test environment.

A well designed, flexible driving simulator with a modularised architecture is integral to the development of a virtual DVP. Theme 3 will provide such a simulation environment within PSI. The University of Leeds Driving Simulator, an existing major research facility made possible through HEFCE's Science Research Investment Fund, and whose functionality is completely under the control of the research team, will form the test-bed for the development. The research team's existing expertise in human perception and behaviour in a virtual driving environment, coordinated with parallel PSI Themes in vehicle analysis, multi-physics and visualisation, will be combined with JLR's existing knowledge-base in Computer Aided Engineering. The goal is to develop a coordinated evaluation tool that has the potential to support agile, cost-effective and efficient manufacturing processes in virtual vehicle design.

Planned Impact

The ability to fully validate the human aspects of the vehicle design through virtual methods will significantly reduce the considerable expense of building physical prototypes for this purpose, and reduce time to market. The ability to subject virtual components, systems, and the whole vehicle to realistic driving and environmental inputs will also significantly increase the breadth of testing and validation over and above that which could be achieved through physical testing alone. Reduced physical testing and the development of more agile, cost-effective manufacturing is one of the five key strategic themes reflecting business opportunities in the Technology Strategy Board's current landscape for High Value Manufacturing. Furthermore, the objective of the proposed work programme to expand the coherence and compatibility of JLR's use of simulation in the vehicle design process meets another TSB strategic theme "increasing the global competitiveness of UK manufacturing technologies by creating more efficient and effective manufacturing systems".

UK involvement in the cutting edge of research in emerging automotive system simulation is currently rather slim, as evidenced by the generally low level of UK participation in large-scale European research programmes within the automotive and technology areas. PSI represents a concerted partnership by major players in the UK automotive and research sectors to address this imbalance. Given the rapid changes in vehicle technology, it is vital to have the UK at the forefront of new developments, particularly involving the under-developed areas of vehicle simulation and virtual prototyping. This is especially pertinent given the recent resurgence of the UK-based automotive industry which exported a record-breaking 80% of production in 2011 (Society of Motor Manufacturers and Traders, 2012).

This project also has significant potential to improve the efficiency of the UK vehicle manufacturing industry outside of JLR (e.g. Aston-Martin, Bentley, Lotus) who share the same business drivers of the faster delivery of new products and concepts to market. The scientific impact of the work programme to facilitate these improvements fall under the challenges set by EPSRC's Manufacturing the Future vision, particularly in the Manufacturing Informatics research areas of Engineering Design and Graphics and Visualisation.
 
Description This grant has carried out research to improve knowledge about and capability for the use of driving simulators in industrial development of vehicles, as a cost-efficient alternative to testing with physical prototypes. One main overarching research question has been: "What type of simulator is needed to permit reliable testing of vehicle attribute X?"

From a directly applied perspective, we have carried out controlled studies on vehicle testing both in physical vehicles and simulators, and have obtained the following main results:

(1) In testing of in-vehicle human-machine interfaces (HMI), the needed simulator varies greatly with the type of question being asked, and in many cases very simple and cheap simulators can be enough.
(2) In testing of the "driven attributes" of a vehicle (e.g., "steering feel", "handling", "ride"), non-blind testing of alternative vehicle designs by test drivers carries risk of confirmation bias.
(3) In testing of stability control systems (SCS), driver behaviour and subjective ranking of alternative SCS configurations are well preserved in the simulator. This suggests that current high costs for SCS testing in remote, cold-weather locations, could potentially be reduced by an increased and earlier use of simulator-based testing.

To permit these applied outcomes, a number of novel, reusable methods have been developed, especially:

(1) Methods for quantitatively comparing observed driver behaviour between real vehicles and simulators, including metrics based on driver models.
(2) Methods for comparing subjective ratings of alternative vehicle or system designs, both between individual test drivers and between testing in real vehicles and simulators.
(3) Methods for optimising, for a given driving task, the motion system of motion-enabled driving simulators, as well as the algorithms controlling the motion system.

The work has relied on concurrent foundational research on driver behaviour, also carried out in this grant, leading to formulation of novel theoretical models of human multisensory integration and control in real and simulated driving.
Exploitation Route The applied outcomes mentioned above can provide guidance to anyone carrying out, or aiming to carry out, simulator-based testing of HMI, SCS, or driven attributes. The novel methods mentioned above can be put to use in both academia and industry, to quantify and compare driver behaviour and subjective ratings between different contexts, not least between real and simulated vehicles, also for driving tasks or tests not addressed here, for example to determine whether simulator-based testing is appropriate for some given application. The developed driver models have a large number of potential uses, including improved algorithms for simulator motion cueing and completely virtual, model-based approaches to vehicle testing.
Sectors Digital/Communication/Information Technologies (including Software),Manufacturing, including Industrial Biotechology,Transport,Other

 
Description The project's work on developing and assessing methods for virtual vehicle testing in driving simulators involved considerable interaction between University of Leeds researchers and Jaguar Land Rover (JLR) personnel, allowing for a strong knowledge transfer between the two parties, and permitted an acceleration of simulator-based work within JLR. In the specific types of vehicle testing being researched - low-friction vehicle stability testing, testing of human-machine interfaces, and driven attributes testing - there was uptake of new concrete knowledge about what types of simulator capabilities are required for valid virtual testing. Also, the project raised the general awareness within JLR at many different levels, from test drivers via engineers to decision makers, of what can be achieved with simulator-based testing and what its benefits are. As of 2022, JLR are reporting that they are using driving simulators more extensively than ever before to help develop their products faster and more robustly. This activity occurs from the earliest concept selection stage right through to the final vehicle sign off and validation. JLR have also openly stated that they have matched Formula 1 levels of simulator usage in developing the recently revealed 2022 Range Rover model.
First Year Of Impact 2016
Sector Manufacturing, including Industrial Biotechology,Transport
 
Description Training of JLR users in PSi tools and methods
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description COMMOTIONS: Computational Models of Traffic Interactions for Testing of Automated Vehicles
Amount £1,170,743 (GBP)
Funding ID EP/S005056/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2019 
End 06/2023
 
Description PhD topic: Shared Haptic Control in Vehicle Automation
Amount £66,425 (GBP)
Organisation Jaguar Land Rover Automotive PLC 
Department Jaguar Land Rover
Sector Private
Country United Kingdom
Start 10/2014 
End 04/2018
 
Description VeriCAV: Verification of Connected and Automated Vehicles
Amount £2,383,099 (GBP)
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 11/2018 
End 10/2020
 
Title Integration capabilities for running third-party functionality in the University of Leeds Driving Simulator (UoLDS) 
Description The technical architecture of the UoLDS has been improved to allow interfacing of third party functionality. So far, this has been used to integrate (1) industry grade multi-body vehicle dynamics models, (2) custom-made motion cueing algorithms under research in the project, (3) a software-in-the-loop version of a Jaguar Land Rover stability control system. 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact Each specific integration enables new lines of research that would not have otherwise been feasible. Item (1) above underpins much of the virtual vehicle testing research in the project; see other entries in the report for this award. Research based on items (2) and (3) are still work in progress. 
 
Title Methods for Bayesian analysis of subjective ranking data 
Description A method for statistically analysing data from humans providing subjective rankings of several options (e.g., subjectively perceived stability performance of alternative vehicle designs), to quantify degree of consistency between individual participants, and especially to quantify to what extent rankings are preserved between different contexts (e.g., between a real vehicle and a simulated vehicle). 
Type Of Material Model of mechanisms or symptoms - human 
Year Produced 2017 
Provided To Others? Yes  
Impact Has allowed us to show in a clear way how some of JLR's current test practices do not seem to provide reliable data, whereas others do, and crucially also to show that some testing can be reliably carried out in a simulator instead of in a physical vehicle prototype. 
 
Title Virtual Gaydon Emissions Circuit 
Description A virtual replica of the "Emissions Circuit" proving ground of Jaguar Land Rover's Gaydon site. 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact Has allowed stringent testing of the validity of HMI testing in the simulator, as compared to testing in a physical vehicle. 
 
Title Virtual Gosport Lane 
Description A geospecific simulation database, including high-fidelity road surface data from LIDAR scans, replicating a stretch of rural road routinely used by Jaguar Land Rover for vehicle testing. 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact Allows stringent testing of the extent to which the vehicle testing done on the real road can be replaced/complemented with virtual testing in the simulator. 
 
Title Virtual Sweden proving ground 
Description A virtual replica in the driving simulator of parts of proving grounds in Northern Sweden, used by JLR and several other vehicle manufacturers for winter testing. This tool allows comparison of driver and vehicle behaviour between real world and driving simulation, with the ultimate goal of replacing as much as possible of costly off-site winter testing with virtual tests. Development of this tool has required extensive gathering of data from the actual driving environment in Sweden, as well as interfacing of industry-grade multi-body vehicle dynamics software into the real-time environment of the driving simulator. 
Type Of Material Improvements to research infrastructure 
Year Produced 2016 
Provided To Others? Yes  
Impact Positive indications that the simulator can be used to replace/complement certain parts of cold-climate testing with physical vehicle prototypes. This offers potential for large cost-savings for industry. 
 
Title Closed-loop self-learning driver model 
Description A driver model, based on state of the art machine learning algorithms, capable of learning to drive a vehicle through a test track. 
Type Of Material Computer model/algorithm 
Year Produced 2017 
Provided To Others? No  
Impact None so far. 
 
Title Driven attributes data set 
Description A collection of objective and subjective data from 6 professional test drivers evaluating driving attributes of a vehicle under a number of different vehicle configurations, on a rural road. Currently being complemented with a corresponding data set of the same type of testing in the simulator. 
Type Of Material Database/Collection of data 
Provided To Others? No  
Impact None so far. 
 
Title Method for interpreting driver control behaviour as intermittent control 
Description A method for interpreting driver control behaviour as intermittent control, reconstructing a continuous control signal as a sequence of discrete adjustments. This method allows a more detailed understanding of driver control behaviour. 
Type Of Material Data analysis technique 
Year Produced 2016 
Provided To Others? Yes  
Impact The method has been used both for applied purposes, allowing better comparisons of driver behaviour between real vehicle and simulator, and for more basic scientific purposes, as part of research to understand human sensorimotor control both in driving and in other tasks. 
 
Title Real/Virtual HMI testing data set. 
Description A collection of data from a total of 23 drivers, performing human-machine interaction tasks both on a real test track in a real vehicle, and in a virtual replica of the same experiment, with different simulator motion capabilities. 
Type Of Material Database/Collection of data 
Provided To Others? No  
Impact Indications as to what type of simulator is needed for valid HMI testing. Under integration in Jaguar Land Rover guidelines. 
 
Title Real/Virtual Sweden data set 
Description Objective and subjective data from 8 drivers performing low-friction vehicle stability testing both in test tracks in northern Sweden and in a virtual replica of the same test tracks. 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? Yes  
Impact First positive indications that virtual testing can be used to complement/replace part of this type of vehicle testing. This provides large potential for cost-savings for industry. 
URL https://osf.io/va5kr/
 
Title SCS evaluation data set 
Description A collection of data from eight drivers evaluating a number of different versions of a stability control system, in a real vehicle on low-friction test tracks in northern Sweden, as well as in a simulated replica of the same. 
Type Of Material Database/Collection of data 
Year Produced 2017 
Provided To Others? No  
Impact Has provided JLR with tangible evidence that their costly overseas stability control testing can be complemented with more cost-efficient simulator-based alternatives. 
 
Title Sensorimotor control modelling framework 
Description A computational framework for modelling sensorimotor control, based on motor primitives, evidence accumulation mechanisms, and prediction of sensory outcomes of motor actions. This has extended on earlier work, to specify the framework in more detail, and validate it further with human control data. 
Type Of Material Computer model/algorithm 
Year Produced 2016 
Provided To Others? Yes  
Impact This framework provides a potentially unifying account of a number of mechanisms that have previously been modelled separately, and could thus be of high scientific value. From an applied point of view, this framework is being applied elsewhere as part of methods for evaluating vehicle safety support functions and vehicle automation. 
 
Title Steering models for behaviour comparisons 
Description A selection of few-parameter driver steering control models. It has been shown that these can be meaningfully fitted to observed driver steering records, to provide a detailed understanding of driver steering behaviour. 
Type Of Material Computer model/algorithm 
Year Produced 2017 
Provided To Others? Yes  
Impact These models have been used to compare driver behaviour between real vehicles and simulator, providing a number of novel insights. 
 
Description Co-simulation work with PSi Theme 1 
Organisation University of Leeds
Department Faculty of Biological Sciences
Country United Kingdom 
Sector Academic/University 
PI Contribution We have supplied simulation databases and software to the School of Computing researchers, for their use as a test case for their research on co-simulation methodology.
Collaborator Contribution The School of Computing researchers have implemented our databases and software in their co-simulation environment.
Impact The School of Computing researchers have gained an improved understanding of requirements on co-simulation in a driving simulation context. This is important since they are pursuing a spin-out commercialisation of their solutions. There has also been a mutual sharing of ideas about co-simulation, to the benefit of both parties. This is a multi-disciplinary collaboration, combining especially mechanical/vehicle engineering with computer science.
Start Year 2016
 
Description Reduced-order vehicle handling models 
Organisation Loughborough University
Department Department of Aeronautical and Automotive Engineering
Country United Kingdom 
Sector Academic/University 
PI Contribution Theme 3 of PSI has developed a mechanism to utilise driver models in quantifying objectively how cues are perceived, integrated and used to produce corresponding vehicle control actions. Vehicle handling plays a fundamental role in this perception-action loop, in simulation itself affected by the quality of a vehicle handling model. Theme 2 is attempting to reduce the required complexity of such handling models, however lacks an objective assessment technique to do evaluate the speed/performance trade-off. We will offer the ability to do this by comparing the structure of the driver models against real-world data.
Collaborator Contribution Development of reduced-order vehicle models which will benefit real-time simulation applications such as driving simulation.
Impact The collaboration is mulch-disciplinary in that it combines cognitive ergonomics with automotive engineering.
Start Year 2014
 
Description Sharing of visual test-track models 
Organisation University of Warwick
Department Warwick Manufacturing Group
Country United Kingdom 
Sector Academic/University 
PI Contribution The sharing of a driving simulation platform to visualise virtual road environments
Collaborator Contribution The development of real-world test-track scenarios and advanced rendering techniques to maximise visual quality in virtual vehicle handling driving simulator scenrios.
Impact Improved visual fidelity in the University of Leeds simulation databases.
Start Year 2014
 
Description Standard methodology to evaluate virtual driving environments 
Organisation University of Warwick
Country United Kingdom 
Sector Academic/University 
PI Contribution The IDL team within the Warwick Manfacturing Group have a sub-project within PSI which shares common group to our Theme 3. This project (7.4 - "Realism in real-time"), is using selective rendering technqiues to create high-fidelity multisensory virtual environments in real time. The collaboration shares our data and methdologies developed within sub-project 3.0 to evalaute such virtual environments.
Collaborator Contribution The sharing of emerging visual rendering techniques to improve the visual capability of the driving simualtors within Theme 3.
Impact Volumetric texture maps used in 3.1 Virtual Sweden.
Start Year 2014
 
Title Methods/tools for rapid tuning of simulator motion cueing algorithms 
Description Optimisation methods and a software tool for rapid tuning of the parameters of the so-called "Classical" algorithm for simulator motion cueing. 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2016 
Impact Has allowed more cost-efficient (and stringent) tuning of motion cueing for the research in this project. Is currently being transferred to Jaguar Land Rover to improve cost efficiency in their simulator work. 
 
Title Software for behavioural fidelity analysis of driving simulators 
Description A reusable MATLAB implementation of our methods for behavioural fidelity assessment of driving simulators. 
Type Of Technology Software 
Year Produced 2017 
Impact Nothing beyond our own research so far, but the software has been handed over to JLR with documentation and training. 
 
Title Software for carrying out Bayesian analysis of subjective ranking data 
Description A software implementing our developed method for comparing subjective ranking data between individuals and between ranking contexts (e.g., between a real vehicle and a simulator). 
Type Of Technology Software 
Year Produced 2017 
Impact Nothing beyond our own research so far, but the software has been handed over to JLR together with documentation and training. 
 
Title Software for right-sizing of simulator motion bases 
Description A MATLAB program for finding the minimum size of a simulator motion base giving acceptable performance for a given driving task (under assumptions about driver perception). 
Type Of Technology Software 
Year Produced 2017 
Impact So far nothing beyond our own research results, to be reported in paper/thesis, but the software has been handed over to JLR with training. 
 
Description JLR PhD conference 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Presentation of the work of PhD students in the project to JLR engineers.
Year(s) Of Engagement Activity 2015
 
Description JLR ShareFair 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Presentations, posters, and demos for JLR engineers and decision-makers. New connections made with important stakeholders with JLR, followed up in later work.
Year(s) Of Engagement Activity 2016
 
Description JLR ShareFair 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Presentation of Theme 3's work to JLR engineers with the intention to stimulate capability in their use of driving simulation.

Additional tie-in with JLR engineering staff
Year(s) Of Engagement Activity 2014,2015
 
Description PSi information and Virtual Sweden demo at Institute building launch event 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Demonstration of Virtual Sweden testing environment. Several persons present, although experienced in driving simulation, expressed that they were impressed with the quality of the virtual environment, and appreciated that it could be of high value to industry.
Year(s) Of Engagement Activity 2017
 
Description Presentation at Vehicle Dynamics workshop, Cambridge 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Dissemination of project results to a UK national workshop gathering researchers in the area of vehicle dynamics.
Year(s) Of Engagement Activity 2017