Reproducibility of statistical tests in pharmaceutical products development
Lead Research Organisation:
Durham University
Department Name: Mathematical Sciences
Abstract
Throughout the development process of new medicines, many statistical tests are used to support decisions. Statistical analysis of the reproducibility of such test results is important: would a repeat of the experiment lead to the same test result and decision? The student will investigate reproducibility of several tests at AstraZeneca, which will require further development of statistical methods.
The main research questions, so the objectives of the project, involve the quantification of test reproducibility. This is core statistical theory and methods, as such it sits firmly within the EPSRC's remit as a novel topic in Statistics, a key sub-area of Mathematical Sciences.
At Durham, we have developed a novel statistical approach called Nonparametric Predictive Inference (NPI). This is a frequentist approach based on few model assumptions, enabled by the use of imprecise probabilities (which generalize the classical probabilities by assigning intervals instead of single values to events). In the last few years, the predictive nature of NPI has proven to be attractive to answer the test reproducibility question, by explicitly considering a hypothetical repeat of the actual experiment in the future. This approach has been developed, and published, for some basic nonparametric tests, and is now ready to be developed to a wider range of practical tests scenarios. It is important here to emphasize that test reproducibility can be regarded as the third crucial property of a statistical test, in addition to level of significance and power. It has long been misunderstood although its practical relevance is evident.
In this collaboration with AstraZeneca, the first test scenario that the student will work on involves both pairwise Student t-tests, and their combination for testing between multiple treatments. Explicit questions are not only on the development of the NPI reproducibility theory for such tests, but also on the implementation with regard to computation. Whilst this theory will be developed in relation to a pharmaceutical application, the methodology will be applicable in all areas where such tests are performed. After this first project, we will identify another test scenario with new challenges for the development of the appropriate NPI reproducibility methodology and its implementation.
The main research questions, so the objectives of the project, involve the quantification of test reproducibility. This is core statistical theory and methods, as such it sits firmly within the EPSRC's remit as a novel topic in Statistics, a key sub-area of Mathematical Sciences.
At Durham, we have developed a novel statistical approach called Nonparametric Predictive Inference (NPI). This is a frequentist approach based on few model assumptions, enabled by the use of imprecise probabilities (which generalize the classical probabilities by assigning intervals instead of single values to events). In the last few years, the predictive nature of NPI has proven to be attractive to answer the test reproducibility question, by explicitly considering a hypothetical repeat of the actual experiment in the future. This approach has been developed, and published, for some basic nonparametric tests, and is now ready to be developed to a wider range of practical tests scenarios. It is important here to emphasize that test reproducibility can be regarded as the third crucial property of a statistical test, in addition to level of significance and power. It has long been misunderstood although its practical relevance is evident.
In this collaboration with AstraZeneca, the first test scenario that the student will work on involves both pairwise Student t-tests, and their combination for testing between multiple treatments. Explicit questions are not only on the development of the NPI reproducibility theory for such tests, but also on the implementation with regard to computation. Whilst this theory will be developed in relation to a pharmaceutical application, the methodology will be applicable in all areas where such tests are performed. After this first project, we will identify another test scenario with new challenges for the development of the appropriate NPI reproducibility methodology and its implementation.
People |
ORCID iD |
| Frank Coolen (Primary Supervisor) | |
| Andrea Simkus (Student) |