HCD: Novel approaches of multi-parameter evidence synthesis and decision modelling for efficient evaluation of diagnostic health technologies

Effective and accurate assessment of diagnostic tests is crucial from the point of view of patient care, development of new therapies and allocation of resources in health services such as NHS. Decision makers, such as the National Institute for Health and Care Excellence (NICE) in the UK make recommendations regarding the uptake of the new diagnostic tools. Such assessments are complex as they are based not only on an assessment of the accuracy of the diagnostic tests, but also on clinical outcomes in patients diagnosed using these tests and cost effectiveness of both the diagnostic tests and the treatments patients receive following the diagnosis.

Following recent advances in science, a multitude of novel diagnostic tools, as well as pharmacological therapies closely linked to these diagnostic technologies, have been developed. Some of the tests are genetic biomarkers, others are imaging techniques (such as magnetic resonance imaging). Many novel diagnostic tools can speed up patients' diagnosis, thus improving their chances for successful treatment. Other diagnostic tests (or biomarkers) are used to identify groups of patients who can benefit from novel therapies.

These advances, however, bring an additional layer of complexity when it comes to evaluation of new diagnostic health technologies, due to complexity of available evidence. In this project, we aim to develop statistical tools to help in synthesis of such complex data. For example, often multiple test combinations need to be used when patients are being diagnosed or their prognosis is assessed. We will develop methods which will take into account interdependencies between these tests to ensure that decisions about the uptake of such new diagnostic tools are based on all relevant evidence. Such evidence also includes outcomes of patients. When patients are diagnosed as positive for certain biomarkers, their outcomes are often evaluated in clinical trials and data from these trials need to be combined to make efficient assessments. Such clinical trials, however, differ in the way they are designed generating heterogeneous data, varying with respect to the types of patients being included, and are therefore difficult to combine. To overcome this challenge, we will develop methods for efficient synthesis of evidence on the effectiveness of prognostic biomarkers and related therapies from heterogeneous study designs.

Evidence from clinical trials or diagnostic test accuracy studies may be not only heterogeneous but also limited. We will investigate how the use of electronic health records, such as data from cohort studies or patient registries, can help to generate more robust evidence for assessment of diagnostic tools.

In the final part of our project we will investigate how the methods developed in earlier parts of the project can be used to effectively inform models evaluating cost-effectiveness of new diagnostic technologies, including prognostic biomarkers. The accuracy and cost-effectiveness of diagnostic tests cannot be evaluated in isolation, as a combination of data on their ability to correctly diagnose patients, their ability to predict treatment effects, and their impact on clinical decisions about treatment options should be taken into account. All these components that impact on the decision about the uptake of new diagnostic tests result from different parts of analysis of different sources of evidence and the estimates of these analyses come with certain level of uncertainty, because they are based on relatively small subsets of the population. Including information on the accuracy of potentially multiple diagnostic tests (and related treatments) in a decision modelling framework, whilst taking into account their dependencies and related uncertainty, is a very complex undertaking. We will explore optimal methods for combining all this information in a decision framework.

Recent advances in research have led to development of a multitude of diagnostic health technologies as well as methods for their assessment, generating heterogeneous data sources on accuracy of these technologies. We will develop statistical tools for synthesis of complex data on multiple diagnostic tests and their inclusion in a health technology assessment (HTA) decision framework. We will carry out methodological research in three work packages (WPs).

WP1: We will develop multivariate meta-analytic methods for synthesis of evidence on test accuracy measures of multiple diagnostic tests and/or thresholds, employing a range of copulas to account for the correlations between sensitivities and specificities of multiple tests most effectively. We will develop further methods for evaluating the utility of dynamic test thresholds to maximise diagnostic ability of sequential tests.

WP2: We will develop evidence synthesis methods combining data on effectiveness of biomarkers from heterogeneous study designs ("randomise-all", enrichment, single-arm, and biomarker-strategy trials), building on a range of methods developed for combining randomised trial evidence with real-world data at aggregate level (including meta-analytic methods with bias adjustment and hierarchical methods differentiating between study designs).

WP1 and WP2: We will develop methods for inclusion of individual patient data from electronic health records to enable the development of more robust evidence for HTA of diagnostic tools, including prognostic biomarkers, and for evaluation of surrogate markers which are predictors of relative treatment effect.

WP3: We will develop a comprehensive decision modelling framework taking into account all relevant evidence on diagnostic tests and related uncertainty to assess potential for improvement in decision making process when using the methods developed in WP1 and WP2.

The proposed research will impact multiple stakeholders involved in the development of diagnostic tools, clinical practice as well as the drug development process. The outputs of this research programme, first and foremost, will provide tools and guidance for analysts employed by a range of stakeholders and beneficiaries: academia, industry (pharmaceutical companies), small and medium-sized enterprises (SMEs), commercial clinical research organisations (CROs), health technology assessment (HTA) agencies, such as NICE in the UK and regulatory agencies such as MHRA, EMA and FDA.

Equipped with the analytical tools and software produced in this project, analysts will be able to efficiently assess accuracy of diagnostic technologies including companion biomarkers. By helping to assess accuracy of new diagnostic technologies in specific disease areas and patient populations, the proposed methodology will impact clinical practice, where reliable diagnostic tools are needed to diagnose patients early and accurately in order to identify suitable treatment options. When reliable diagnostic tools or companion biomarkers are identified, manufacturers will be able to evaluate new health technologies more effectively, potentially in reduced time and at reduced cost (by, for example, conducting trials of therapies targeted on genetic biomarkers in smaller population with shorter follow up). Similarly, academic/clinical researchers who conduct clinical trials in hospitals in the local or multinational populations will also be able to undertake their trials more efficiently.

The developed analytic tools will also enable analysts to make predictions of the long term clinical benefit from treatment effects measured using diagnostic tests as early markers of clinical benefit or to assess the value or companion biomarkers in identifying target populations and in turn the best treatment options in those populations. This increased analytical ability in utilising data on diagnostic tools will have impact on all stakeholders in clinical practice, conducting clinical trials, policy makers in regulatory and reimbursement agencies, and ultimately manufacturers (of diagnostic tests and related therapies) and patients.

Robust meta-analytical methods to assess new diagnostic technologies will lead to more efficient regulatory processes in agencies such as the MHRA, the EMA and the FDA, which make decisions about market access of new diagnostic technologies. For companion diagnostics (prognostic biomarkers), they will also facilitate more efficient synthesis of evidence and hence efficient decision making processes by the HTA agencies such as NICE. By providing a framework for more efficient regulatory and reimbursement decisions, this methodology research will have impact on the SMEs manufacturing the diagnostic tools and pharmaceutical industry whose products will be available to health services. In turn, patients will benefit from more accurate and timely diagnoses which will ensure effective and personalised treatment in a shorter time. A faster drug development and approval process will lead to a healthier society and a better economy.

The methods developed will be published within months of completion of each project component and can be implemented immediately in all relevant settings. Staff employed on the grant will develop statistical, analytical, computing and communication skills transferable to all employment sectors including academia, CROs, industry and government. The research will also contribute to the development of a training course for analysts employed by any of the stakeholders and development of the curriculum of the Masters and Bachelors courses in Medical Statistics, run by the University of Leicester, graduates of which are trained for employment in all the relevant sectors.