Evaluation and parameterisation of individual-based models of animal populations

Lead Research Organisation: University of Reading
Department Name: Sch of Biological Sciences

Abstract

Ecosystems are populated by autonomous, adaptive individuals, each figuring out its own ways of achieving its goals. It is a widely shared hope that the general principles governing such complex systems will eventually be understood from analysis of computer simulations known collectively as individual-based models (IBMs). IBMs are dynamical systems containing many autonomous interacting agents which are used where, broadly, the factors influencing the behaviour of individual agents are known, but interest centres on what happens at the population level. Will the population increase or decrease? How fast will be the response? Where practical management of ecosystems is required, many consider this can only be realistically performed with IBMs. Examples include conservation management of nature reserves and shell fisheries, assessment of environmental impacts of building proposals including wind farms and highways, management of fish stocks and assessment of the effects on non-target organisms of new chemicals for the control of agricultural pests. Articles in scientific journals have suggested IBMs are the only realistic way forward in diverse fields including economic analysis where the recent global 'credit crunch' might have been avoided with the use of such models. Thus IBMs are the only practicable method of modelling many complex systems where prediction is of vital importance to all.
Despite the widely-appreciated importance of IBMs, the evaluation of these very complex systems still leaves much to be desired. Clearly, the purpose of a model is to explain the world that we see around us. From a statistical point of view we wish to 'fit' the model to data. How can we do this? Recent advances in statistical theory, known as Approximate Bayesian Computation, ABC, suggest how this might be done. Implementation of ABC requires development of practical methods that will allow users to fit their IBM models to real data in an efficient manner. This Bayesian approach should allow calculation of distributions of possible parameter values in IBMs, given observations, and evaluation of whether one model is better than another. In this project we devise practical methods that will allow all makers of IBMs to validate their models properly by reference to relevant data. Provision of such methods is crucial if we are to have robust and reliable bases for making crucial decisions about environmental impacts, nature conservation, and the licensing of new chemicals for the control of agricultural pests.

Planned Impact

All those making individual-based models (IBMs) and using them for planning purposes will benefit. This includes planners and managers who use IBMs for conservation management of nature reserves and shell fisheries, assessment of environmental impacts of building proposals including wind farms and highways, management of fish stocks and assessment of the effects on non-target organisms of new chemicals for the control of agricultural pests. Examples of organisations who use IBMs are the UK Chemicals Regulation Directorate, Syngenta, Bayer, Dow Agrochemicals, and the Environment Agency. Other areas may eventually benefit - over the past 30 years ecology has produced over a thousand IBMs. Individual-based modelling is the only modeling method that can be used where explicit spatial landscapes affect population dynamics. As suggested in recent commentaries in Nature there is a view that current economic modelling methods are unsuitable and that individual-based modelling is the way forward.
All IBM modellers need to validate their models to show that they are reliable. The principal obstacle to validation is lack of a sound statistical methodology. Currently modellers rely on Pattern-Oriented Modelling but this has many drawbacks (lack of standard errors, ambiguity about best model) and modellers would prefer to have methods of evaluation like those used in statistics, where statistical tests are available to guide choice between models, statistics such as R2 or deviance can be used to assess overall goodness of fit, and parameters can be calibrated from data and confidence intervals reported using least-squares or maximum-likelihood procedures.
The proposal has been designed to meet these needs. Our research programme addresses two key areas of concern in IBMs: parameterisation and model validation. By providing a method of solving these problems, ecology as a whole will benefit through more accurate modelling of ecosystems with potential for improved forecasting, management and environmental risk assessment. In the longer term the methods developed may also allow more accurate prediction of the future performance of the economy and geographyical systems. We will create and maintain a website detailing algorithms, scripts, programs and manuals to allow widespread implementation of our methodology by IBM modellers.
We hope the benefits described here will be fully realised within four years. Our research will contribute by implementing and demonstrating practical methods for parameterising and validating IBMs. This will facilitate the production of credible and trustworthy models for use throughout environmental planning. Environmental planning contributes to the nation's health by regulating the chemicals used in agriculture and to the nature's culture and quality of life through improved conservation and management of nature reserves and sites of special scientific interest.

Publications

10 25 50

publication icon
Van Der Vaart E (2018) Taking error into account when fitting models using Approximate Bayesian Computation. in Ecological applications : a publication of the Ecological Society of America

 
Description We have shown that a simple form of the statistical technique we are working on, ABC, can, in some circumstances, parameterise and evaluate complex computer models. Further work has discovered improvements that improve the calibration and evaluation. We have shown that the computed parameter values and credible intervals are accurate according to the best available diagnostic test.
Exploitation Route After further exploration and development of alternative techniques, ABC should be useful to all those using complex computer models, provided there is data available with which to evaluate how well model outputs match data.
Sectors Aerospace, Defence and Marine,Agriculture, Food and Drink,Environment,Financial Services, and Management Consultancy,Government, Democracy and Justice,Manufacturing, including Industrial Biotechology,Transport

URL https://ibmreading.wordpress.com/about/
 
Description Undertaking ABC on IBM for CEFAS (fish community dynamics model) 
Organisation Centre For Environment, Fisheries And Aquaculture Science
Country United Kingdom 
Sector Public 
PI Contribution Method of evaluating computer models
Collaborator Contribution Example of application of our methods
Impact None yet
Start Year 2014
 
Description Undertaking ABC on IBM for CEH (midge population dynamics 
Organisation Natural Environment Research Council
Department Centre for Ecology & Hydrology (CEH)
Country United Kingdom 
Sector Academic/University 
PI Contribution Methods for evaluation of computer model
Collaborator Contribution New example of application of our methods
Impact No impact yet
Start Year 2014
 
Description Undertaking ABC on simple mechanistic model for Syngenta (pesticide risk assessment) 
Organisation Syngenta International AG
Department Syngenta Ltd (Bracknell)
Country United Kingdom 
Sector Private 
PI Contribution Methods for evaluating computer models
Collaborator Contribution Example of use of our methods
Impact None yet
Start Year 2014
 
Description Created a website (9th July 2014; www.abc4ibm.com) to disseminate information on the method 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Provided information to practioners

none so far
Year(s) Of Engagement Activity 2014
URL http://www.reading.ac.uk/biologicalsciences/res/eeb/ecology/biosci-ibm.aspx
 
Description "ABC for IBM" Workshop (11th July 2014), Charles Darwin House, London 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Sparked interest and discussion

Induced others to try the methods we developed.
Year(s) Of Engagement Activity 2014
 
Description One-day workshop for the British Ecological Society Annual Meeting 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Demonstrated our methods of building, calibrating and evaluating Individual Based Models to 60 attendees of the British Ecological Society Annual Meeting in an all-day workshop. Much interest including subsequent email enquiries.
Year(s) Of Engagement Activity 2017
 
Description Presented ABC as part on NERC short course on Agent-based modelling organised by Bournemouth University Jan 28 2016 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Spread knowledge of ABC to postgrads who may use it in the PhD
Year(s) Of Engagement Activity 2016
 
Description Presented ABC for IBM to Centre for Ecology and Hydrology (4th Sept 2014) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Sparked request for further one-to-one tuition

led to collaborative research
Year(s) Of Engagement Activity 2014
 
Description Presented ABC for IBM to members of Rutherford Appleton Lab (21st August 2014) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Presentation sparked interest

none so far
Year(s) Of Engagement Activity 2014
 
Description Presented ABC for IBM to staff in Syngenta (2nd Sept 2014), 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Request for follow up information

Formal collaboration now started
Year(s) Of Engagement Activity 2014
 
Description Workshop at the British Ecological Conference 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact 80 researchers attended to discuss the construction and evaluation of Individual Based Models using Approximate Bayesian Computation and other techniques
Year(s) Of Engagement Activity 2016