Statistical inference and uncertainty quantification for complex process-based models using multiple data sets

Lead Research Organisation: University of Warwick


Making responsible decisions about landscapes is facilitated by the use of complex models able to represent multiple competing demands on land use. Decisions about land use require that trade-offs between competing demands be identified, and their consequences through time be characterised. Methods for representing consequences through time on maps generally take the form of complex models such as stochastic computer simulations. Such models are increasingly used to make realistic predictions about real world processes from socio-ecological systems involving land use to the effects of climate change. Because these models attempt to simulate all relevant aspects of a real physical system, they may involve many parameters, some of which will be difficult to set correctly. As the final objective of these models is to assess the possible consequences of management decisions, such as the placement of wind turbines, it is crucially important that the uncertainty introduced by calibrating parameters be understood.

Approximate Bayesian Computation, or ABC, is a promising technique for estimating parameter values together with their credible intervals, and this allows calculation of the uncertainty deriving from parameter calibration. The overarching aim of this proposal is to improve ABC, or related approaches, to make them sufficiently fast and accurate that they can be widely used for the evaluation and calibration of complex stochastic computer models, and to quantify the uncertainty attached to their predictions. This process is complicated by the fact that making decisions about land use involves taking into account multiple processes and multiple datasets: this proposal aims to develop methods specifically designed for this situation.

The end goal of the project is to be able to fit and evaluate the accuracy of complex models for real, challenging applications, and for this approach to be more widely used in practice. We will work with investigators in the landscape decision-making programme, and others involved in landscape decision modelling, to apply the methods we develop to their models. Our proposal develops and brings to bear cutting-edge mathematical and statistical methodologies to calibrate complex models, and to quantify the uncertainty in their predictions that derives from parameter calibration.

Planned Impact

The beneficiaries of our research essentially span all investigators who are calibrating complex models and using them to make predictions about the consequences of management decisions. We will work closely with all investigators who engage with our work to ensure the methods developed can meet their needs. We will engage directly with those who attended workshops held in conjunction with the Isaac Newton Institute (INI) programme on Mathematical and Statistical Challenges in Landscape Decision Making. Looking more widely the beneficiaries of our research will be all those using complex models to make predictions about the consequences of management decisions. These will include our project partners in the Centre for Environment, Fisheries and Aquaculture Science (Cefas), which is an executive agency of DEFRA, and other policy makers in DEFRA.

The beneficiaries will acquire tools with which to efficiently and accurately calibrate models by matching model outputs against data, and to track the uncertainty in model predictions that derives from parameter calibration.

The outcome for beneficiaries of our research will be that they can reliably show the uncertainty in model predictions of the consequences of specific management interventions. This will allow management decisions to be taken while being aware of uncertainty about the consequences of considered interventions.
Description This work focussed on two objectives:

1. Statistical and computational methodology for calibrating complex mathematical models, to ensure that their representation of the real world is as accurate as possible.
2. Implementing the new algorithms to enable their easy application.

Relating to object 1, this project made a number of original contributions. The most significant of these were new methods for allowing the calibration of models to high-dimensional data. A weakness of pre-existing methodology was the requirement of reducing observed data to a (usually incomplete) summary in order to the model to be calibrated within a realistic computational time. Calibrating the model based on the summary can result in inaccurate inference. This project contributed automatic methods that can produce accurate calibration within a realistic computational time, without the requirement to reduce the data to a summary.

The algorithms developed under the project have been made freely available in software, which is easily accessible to researchers who develop their own mathematical models. The design of the software has been motivated by discussions with members of the Landscape Decisions programme. This software has so far been successfully applied to models of fisheries and meteorological models, and has received follow-on funding to make it applicable to applications in genetics.
Exploitation Route The methods developed by this project are expected to be widely used by researchers who used mathematical models in a wide range of fields, including ecology, epidemiology, evolution and physics. Using the methods from the project, this calibration of mathematical models can be performed more accurately, enabling predictions or scenario modelling produced by the models to be more accurate. This impact is likely to be larger due to the development of widely-applicable software under the grant.
Sectors Aerospace, Defence and Marine,Agriculture, Food and Drink,Environment,Healthcare,Government, Democracy and Justice

Title ilike 
Description Software for Bayesian inference for intractable models. 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact This software allows researchers to access a number of state-of-the-art algorithms for Bayesian Computation.