Quantifying uncertainty in the predictions of complex process-based models
Lead Research Organisation:
University of Reading
Department Name: Sch of Biological Sciences
Abstract
Making responsible decisions about landscapes is facilitated by the use of complex models able to represent multiple competing demands on land use. Decisions about land use require that trade-offs between competing demands be identified, and their consequences through time be characterised. Methods for representing consequences through time on maps generally take the form of complex models such as stochastic computer simulations. Such models are increasingly used to make realistic predictions about real world processes from socio-ecological systems involving land use to the effects of climate change. Because these models attempt to simulate all relevant aspects of a real physical system, they may involve many parameters, some of which will be difficult to set correctly. As the final objective of these models is to assess the possible consequences of management decisions, such as the placement of wind turbines, it is crucially important that the uncertainty introduced by calibrating parameters be understood.
Approximate Bayesian Computation, or ABC, is a promising technique for estimating parameter values together with their credible intervals, and this allows calculation of the uncertainty deriving from parameter calibration. The overarching aim of this proposal is to improve ABC methods to make them sufficiently fast and accurate that they can be widely used for the evaluation and calibration of complex stochastic computer models, and to quantify the uncertainty attached to their predictions. The end goal of the project is to be able to fit and evaluate the accuracy of complex models for real, challenging applications, and for this approach to be more widely used in practice. We will work with investigators in the landscape decision-making programme, and others involved in landscape decision modelling, to apply the methods we develop to their models. Our proposal develops and brings to bear cutting-edge mathematical and statistical methodologies to calibrate complex models, and to quantify the uncertainty in their predictions that derives from parameter calibration.
Approximate Bayesian Computation, or ABC, is a promising technique for estimating parameter values together with their credible intervals, and this allows calculation of the uncertainty deriving from parameter calibration. The overarching aim of this proposal is to improve ABC methods to make them sufficiently fast and accurate that they can be widely used for the evaluation and calibration of complex stochastic computer models, and to quantify the uncertainty attached to their predictions. The end goal of the project is to be able to fit and evaluate the accuracy of complex models for real, challenging applications, and for this approach to be more widely used in practice. We will work with investigators in the landscape decision-making programme, and others involved in landscape decision modelling, to apply the methods we develop to their models. Our proposal develops and brings to bear cutting-edge mathematical and statistical methodologies to calibrate complex models, and to quantify the uncertainty in their predictions that derives from parameter calibration.
Planned Impact
The beneficiaries of our research essentially span all investigators who are calibrating complex models and using them to make predictions about the consequences of management decisions. We will work closely with all investigators who engage with our work to ensure the methods developed can meet their needs. We will engage directly with those attending workshops held in conjuction with the Isaac Newton Institute (INI) programme on Mathematical and Statistical Challenges in Landscape Decision Making. Looking more widely the beneficiaries of our research will be all those using complex models to make predictions about the consequences of management decisions. These will include our project partners in the Centre for Environment, Fisheries and Aquaculture Science (Cefas), which is an executive agency of DEFRA, and other policy makers in DEFRA.
The beneficiaries will acquire tools with which to efficiently and accurately calibrate models by matching model outputs against data, and to track the uncertainty in model predictions that derives from parameter calibration.
The outcome for beneficiaries of our research will be that they can reliably show the uncertainty in model predictions of the consequences of specific management interventions. This will allow management decisions to be taken while being aware of uncertainty about the consequences of considered interventions.
The beneficiaries will acquire tools with which to efficiently and accurately calibrate models by matching model outputs against data, and to track the uncertainty in model predictions that derives from parameter calibration.
The outcome for beneficiaries of our research will be that they can reliably show the uncertainty in model predictions of the consequences of specific management interventions. This will allow management decisions to be taken while being aware of uncertainty about the consequences of considered interventions.
Organisations
Publications
Watson J
(2022)
Incorporating environmental variability in a spatially-explicit individual-based model of European sea bass?
in Ecological Modelling
Description | New method of calibrating and evaluating agent-based models has been successfully applied to a complex model of sea bass. The new method runs in a couple of hours on a supercomputer compared to many weeks using a previous method. |
Exploitation Route | The new method will hopefully be widely applied to complex agent-based models for which sufficient calibration/validation data is available. |
Sectors | Aerospace Defence and Marine Agriculture Food and Drink Environment Healthcare Leisure Activities including Sports Recreation and Tourism |
Description | Webinar for the Landscape Decisions Programme on Fitting Models and Quantifying the Uncertainty of Predictions |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | 45 members of the Landscape Decisions Programme attended and several followed up with requests for help with their projects |
Year(s) Of Engagement Activity | 2020 |