Calibration and analysis of complex models: methodological development and application to explore the impact of HAART in AfricaMR/J014362/1

Mathematical models are used routinely in government, the public sector, industry and academia to help predict the future and make policy decisions, such as what is the impact of ocean currents on the climate and how should we best prepare for the next flu pandemic? The usefulness of models in helping to make decision relies on how well they are can be matched to real data (model calibration) and how well we can analyse the models to find out how confident we should be about the predictions they make. In recent years models have become increasingly complex, but we don't yet have good methods to calibrate and analyse these more complicated models. The lack of these methods is a major obstacle to improving our predictions, and the policy decisions that affect all of us in the UK.

HIV/AIDS remains the leading public health problem in much of sub-Saharan Africa. While increased access to treatment for HIV (highly active anti-retroviral therapy or HAART) has successfully reduced suffering and deaths among people living with HIV/AIDS, it is unknown how best HAART can be used to reduce the number of new HIV infections. Models would help us determine the best strategy, but we need improved methods of model calibration and analysis to do this.

In this project we will develop new methods to calibrate and analyse complex individual-based stochastic models (these models are very computer-intensive as every individual is explicitly tracked through time). First we will develop two new model calibration methods specifically for stochastic models and also develop new methods that allow us to calibrate multiple models outputs at the same time (eg the number of people infected with HIV and the number of deaths from AIDS). Then we will compare these new methods with existing methods to see how they perform. We will go on to develop hybrid calibration methods that combines the strengths of our new and existing methods, to get the best of both. Finally we will apply our new methods to calibrate a complex model to data on HIV/AIDS trends in Africa, and use our fitted models to predict the impact of HAART on HIV/AIDS in Africa.

The development and evaluation of robust methods for the calibration and analysis of complex models should result in the generation of more reliable scientific evidence and better health policy and funding decisions. In addition, an effective method of assessing the likely impact of HAART should lead to better informed policies and improved public health in Africa. If the models are accurate, the benefits may include fewer people dying and being infected with HIV, alleviating the substantial social and economic burden of the epidemic as well as reducing the burgeoning costs of HIV-related health care.

An international multidisciplinary team will carry out this research. Statistical expertise is available from investigators from Durham University, Sheffield University and Cambridge University; expertise in the development and application of complex HIV models is available from investigators in the London School of Hygiene and Tropical Medicine; and virologic expertise and uniquely detailed high quality empirical data is available from MRC/UVRI in Uganda.

Complex individual-based stochastic models are increasingly being used across science, engineering, medicine and public health. The utility of models for prediction & planning relies on how well they are calibrated to data and analysed for robustness of predictions. However, development of methods to calibrate and analyse complex stochastic models has greatly lagged behind their application, resulting in unreliable predictions and increasing the probability of incorrect policy and funding decisions

Aim
Develop and evaluate methods to calibrate and analyse complex individual-based stochastic models, and apply them to explore the impact of HAART on HIV/AIDS in Africa

Objectives
-Develop novel methods for Bayesian Emulation of stochastic models
-Comparative analysis of accuracy/efficiency of existing MCMC, ABC and novel Bayesian Emulation methods for the calibration and analysis of stochastic models
-Develop and analyse a novel hybrid model calibration strategy combining the strengths of ABC and Emulation methods
-Apply developed and analysed methods to predict the impact of HAART on HIV/AIDS in Africa

Methods
Development: Bayes linear and fully Bayesian emulation for calibration of stochastic models, and multivariate outputs
Hybrid ABC & Emulation methods for efficient generation of probabilistic statements about goodness of fit & joint estimation of posterior probability distributions
Experiments: Use new methods to estimate model parameters by calibrating model to simulated data with known 'true' parameter values, varying number and prior bias & variance of model parameters
Application: Use new methods to calibrate complex model to uniquely detailed HIV/AIDS data from African cohorts

Application/exploitation
Application within project to predict impact of HAART on HIV in Africa and engagement with international policy makers/funders to improve policy/funding decisions. Novel methods dissemination to other science areas/public sector/industry

This research should benefit society as a whole, primarily through improved understanding and prediction of complex models, and thereby improved guidance to policy makers.

Immediate beneficiaries will include the users and developers of complex models in government, the public sector, industry and academia. These users will have better methods with which to calibrate and analyse their complex models. Example beneficiaries include epidemiologists and public researchers who use models to compare different strategies to combat infection in the UK and elsewhere, and oil industry engineers who use models to provide information on where to drill exploratory wells. The former is an example where this research could have a relatively quick impact on public heath in the UK and elsewhere. The latter is an example where this research could have a direct and relatively quick impact on the UK economy. Investigators on the proposal have direct links to both groups of beneficiaries.

Direct beneficiaries also include national and international technical modelling committees who will be interested in these developments. An example includes WHO Quantitative Immunisation and Vaccines related Research Committee, on which one of the investigators sits.

This research should also benefit those who rely on information generated by models to make decisions. For example, UK governmental committees such as the UK Joint Committee on Vaccination and Immunisation (JCVI) and international body UNAIDS, both of which rely heavily on information generated by models in their decision making. These bodies would benefit from the higher quality information that could be provided if our novel methods are implemented. Investigators on the proposal are advisors to both these bodies.

The general public should benefit from the improved policy and funding decisions that can be made based on the higher quality information provided by our novel model calibration and analysis methods. They should also benefit from improved policy formation that should lead to improved health and well being, and improved resource allocation in public services.