Multi-objective optimal design of experiments

An experiment differs from a purely observational study in that interventions are deliberately made to the system under study and the effects of these interventions observed. In almost all fields of science, engineering, medicine and business, experiments are the most robust and reliable way of drawing causal conclusions - if we want to know the effect of making a change, we have to make that change and observe the effect. Interpreting the results of all but the simplest experiments involves analysing data on responses (outputs) from the experiment, using statistical models whose complexity depends on the complexity of the experiment. The validity and robustness of conclusions that can be drawn from the experiment depend on how informative the data are with respect to the statistical models used, and how informative the data are depends on the way the experiment is designed.

The statistical design of experiments has developed over the last 100 years to deal with different structures of experiments and data collected from them. Historically there have been two different approaches. Optimal design involves defining a mathematical function, which depends on the particular sets of interventions (treatments) used in the experiment, and then choosing the treatments to optimise this function. This has the advantage of being easily understood to be directly related to the properties of the data analysis, e.g. choose a design to minimise the variance of the estimate of some important quantity. However, it has the disadvantage of oversimplifying the multiple objectives that experimenters actually have in practice. Classical design, on the other hand, chooses designs with attractive mathematical structures (usually based on symmetries) which can make the designs fairly good for many objectives. However, classical designs can be difficult or impossible to find for some experimental structures and there is no guarantee that they will be very good for the objectives of any particular experiment.

This project aims to develop and implement methods which will get the best of both optimal and classical designs, namely multi-objective optimal designs (MOODs). MOODs use the idea of optimising a mathematical function, but that function represents a compromise between the many different objectives that experimenters have in practice. Some of the objectives can be used to restrict the set of designs over which we search for an optimum, e.g. in some cases we might restrict the search to designs which allow us to obtain uncorrelated estimates of the main effects of factors. Other objectives will be combined in a compound optimality criterion, which defines a weighted geometric mean of several individual simple criteria. Since MOODs require a more complex optimisation than standard designs, we will derive theoretical results to allow simplification of the criterion, e.g. by showing that two objectives are actually complementary, so only one is needed. We will also develop algorithms for searching for optimal designs and implement them in programs that can be used by experimenters.

The focus in this project will be on four types of experiment: those with many treatment factors being varied simultaneously; those where the experiments are carried out on a network of subjects; those in which the measured response is a function (or curve); and those in which the treatment factors can be varied over time within the same experimental unit. The breadth of these structures should help other researchers adapt the methods to different types of experiment in the future.

Since so many areas of application use experiments, the methods developed here have the potential to be applied in many different fields, either directly or after further development for particular types of experiment. Experimenters will benefit from being able to get exactly the information required from their experiment as economically and as free from bias as possible.

Experimenters in any field who do complex experiments can benefit from the proposed research by being able to run experiments whose design is more directly targeted at answering their research questions efficiently. Some of the most obvious groups who can benefit are:

1) Experimenters in industrial research, such as the pharmaceuticals, chemicals and food industries, who regularly run multifactor designs, usually classical or simple optimal designs. Given the high cost of many of these experiments, using designs which target the specific questions of interest could allow large savings to be made by obtaining the same information with fewer runs than used by standard designs.

2) Social and market researchers who run experiments on networks of subjects, including online social networks. Current methods usually do not take account of the viral effects of treatments and so might be biased. The methods developed for MOODs on networks will help these researchers to carry out more valid and more economical experiments, by allowing for the viral effects in the design of the experiments and the data analysis.

3) Researchers in the public sector, who use experiments with functional responses can benefit by running their experiments more efficiently in a way which directly optimises the data collected for the specific objectives of the experiment.

4) Process engineers in industry who perform experiments on dynamic systems. Up to now there is little guidance on how to design these experiments and inputs are usually varied in a rather arbitrary way. The proposed research will allow these experimenters to carry out the experiments in a more systematic way to ensure the modelling is as informative as possible.

5) The general public will ultimately benefit as consumers, taxpayers and citizens from the efficiency savings made by the researchers listed above.