Quantifying uncertainty in the predictions of complex process-based models

Making responsible decisions about landscapes is facilitated by the use of complex models able to represent multiple competing demands on land use. Decisions about land use require that trade-offs between competing demands be identified, and their consequences through time be characterised. Methods for representing consequences through time on maps generally take the form of complex models such as stochastic computer simulations. Such models are increasingly used to make realistic predictions about real world processes from socio-ecological systems involving land use to the effects of climate change. Because these models attempt to simulate all relevant aspects of a real physical system, they may involve many parameters, some of which will be difficult to set correctly. As the final objective of these models is to assess the possible consequences of management decisions, such as the placement of wind turbines, it is crucially important that the uncertainty introduced by calibrating parameters be understood.

Approximate Bayesian Computation, or ABC, is a promising technique for estimating parameter values together with their credible intervals, and this allows calculation of the uncertainty deriving from parameter calibration. The overarching aim of this proposal is to improve ABC methods to make them sufficiently fast and accurate that they can be widely used for the evaluation and calibration of complex stochastic computer models, and to quantify the uncertainty attached to their predictions. The end goal of the project is to be able to fit and evaluate the accuracy of complex models for real, challenging applications, and for this approach to be more widely used in practice. We will work with investigators in the landscape decision-making programme, and others involved in landscape decision modelling, to apply the methods we develop to their models. Our proposal develops and brings to bear cutting-edge mathematical and statistical methodologies to calibrate complex models, and to quantify the uncertainty in their predictions that derives from parameter calibration.

The beneficiaries of our research essentially span all investigators who are calibrating complex models and using them to make predictions about the consequences of management decisions. We will work closely with all investigators who engage with our work to ensure the methods developed can meet their needs. We will engage directly with those attending workshops held in conjuction with the Isaac Newton Institute (INI) programme on Mathematical and Statistical Challenges in Landscape Decision Making. Looking more widely the beneficiaries of our research will be all those using complex models to make predictions about the consequences of management decisions. These will include our project partners in the Centre for Environment, Fisheries and Aquaculture Science (Cefas), which is an executive agency of DEFRA, and other policy makers in DEFRA.

The beneficiaries will acquire tools with which to efficiently and accurately calibrate models by matching model outputs against data, and to track the uncertainty in model predictions that derives from parameter calibration.

The outcome for beneficiaries of our research will be that they can reliably show the uncertainty in model predictions of the consequences of specific management interventions. This will allow management decisions to be taken while being aware of uncertainty about the consequences of considered interventions.