Uncertainty quantification and design of experiments

Mechanistic and phenomenological models are used - often in the form of time-consuming computer simulations - across diverse areas of Science and Engineering to understand phenomena and make predictions and inferences, e.g. climate science, epidemiology, and radiological protection to name a few. Such models are informed by theory, but they usually also require data from experiments or observational studies on the real physical process in order to establish parameter values, identify the discrepancy between the model output and reality, and/or suggest potential model improvements. These experiments require careful design in order to yield maximum information about, or understanding of, the process.

The major questions this research project will seek to answer are:-
1) how best can we fit such models to the data given the uncertainty about the parameters and the discrepancy (or model error), and given that the models are often too computationally expensive for traditional methods to be used?
2) how best can we design experiments and studies to provide the most informative data to support this model fitting?

The scientific field of Uncertainty Quantification, on the border between Statistics and Applied Mathematics, has put forward several techniques for the first problem, one of the most popular being the method proposed by Kennedy and O'Hagan (2001) which uses Gaussian processes and Markov Chain Monte Carlo (MCMC) techniques to handle the various uncertainties in a Bayesian framework. Despite its popularity, this technique suffers from the problem of 'non-identifiability': essentially, it cannot tell which parts of the true phenomenon are due to the model and which are due to model error. We will develop advanced function-space MCMC techniques to enable a fully identifiable formulation to be established. We anticipate that this will both increase the accuracy and decrease the uncertainty of inferences made from the data and subsequent predictions from the model.

Design of Experiments, a subfield of Statistics, gives many methods for optimizing the settings applied in an experiment in order to maximize the amount of information gained about a model. However, most existing results for physical experiments apply only to empirical models or computationally-inexpensive phenomenological models, not the expensive models underpinning modern simulations. In contrast, the design methods that apply to computationally-expensive models usually apply only to the simulations themselves (so called in silico experiments, or 'computer experiments'), not the physical experiments used to inform establish parameter values and estimate the model discrepancy. There are only a few published works on the design of the physical experiment for calibration of computationally expensive models, and none of these fully addresses the goals of both prediction and understanding in a coherent Bayesian framework, nor do these methods apply to the most up to date calibration methodology. Our work will seek to establish more comprehensive and up to date methodology by developing novel optimality criteria, utility approximations, and optimization techniques, and reflecting the latest developments in model-fitting approaches.


The project will generate important original knowledge in Statistics, in the subfields of both Uncertainty Quantification and Design of Experiments, including new methods for inference and fitting of complex models and new methods for designing experiments to support this. Comparisons with existing methods will be developed to show the benefits of the new methodology. We expect the work developed will be publishable in top international Statistical journals such as Journal of the American Statistical Association and Technometrics.


References
Kennedy, M. C., & O'Hagan, A. (2001). Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B, 63, 425-464.