HOD1: Comparative Effectiveness Research using Observational Data:Methodological Developments and a Roadmap (CER-OBS)

There is increased availability of linked coded patient data that are collected in the course of clinical care and held electronically in formats and within safe environments that protect their anonymity. This creates unique opportunities for research into the benefits and harms of treatments when prescribed in "real-world" clinical practice ", as opposed to those identified (or not) when tested in clinical trials (which may involve limited numbers and/or types of patients). Evidence generated from routine data is however not immune from controversy, because of the challenges posed by being based on information that is not collected for research purposes. Indeed, errors and incompleteness often affect these data, with the timing and frequency of their collection also potentially biasing their analysis. Our research proposal attempts to address these potential biases by adopting and extending a novel approach to the analysis of routine data. It consists of emulating the ideal clinical trial for the efficacy of a treatment using routinely collected data (the "emulate the target trial" (ETT) approach). To make this happen, the patient population, treatment strategy, follow-up procedures and disease assessment need to be identified from the routine data, with the effectiveness measure, that the target trial would pursue, specified. To do this explicitly is not straightforward. Indeed, there are several examples where the design of studies based on routine data has introduced bias, mostly via incorrect definitions of the patient population or treatment received. Also, studying the effectiveness of treatments to be sustained over time requires estimation of effects that measure the impact of adherence to treatment, and not just its initiation. This poses analytical challenges, usually addressed by the methods that are not necessarily the most suitable, in particularly when data quality issues are also to be addressed.
For these reasons, we propose to:
A) Create a roadmap for the assessment and generation of evidence of comparative effectiveness (or harms) formulated within the ETT framework.
B) Provide easy access to the most flexible of the estimation approaches for comparative effectiveness, g-estimation, and extend it to address the challenges posed by data quality, including facilitating sensitivity analyses.
C) Use exemplars from linked UK databases to illustrate the application of ETT to CER. The treatment regimens being examined are: intensive versus less intensive cardiovascular disease prevention and glycaemic control in type 2 diabetes patients; antibiotics in infancy and subsequent asthma risk in childhood; and palivizumab, a monoclonal antibody, in high risk infants and later hospitalization due to bronchiolitis.

These exemplars will illustrate the advantages of implementing the ETT approach when comparing treatment regimens in terms of estimating their benefits and harms (e.g. in the first example preventing heart disease but potentially increasing hypoglycaemic episodes). In comparison to more traditional uses of routine clinical data, ETT has greater transparency of purpose (addressing the same question as the target trial), which acts as a guide for the study design and analysis. The ability to 'enrol' large numbers of patients who receive their treatment in the "real-world", as delivered by the NHS, and who are followed for many years, as opposed to a limited period, is the great advantage of this approach over equivalent (i.e. pragmatic) randomised clinical trials.

Overall this research will provide tools to aid users of research results (e.g. NICE) in assessing the quality of the available evidence from observational studies, and to guide applied researchers in the implementation of the ETT framework, including designing the study that corresponds to the clinical question, and explicitly dealing with data quality issues while adopting flexible and robust estimation approaches.

The potential gains for enhancing comparative effectiveness research (CER) by exploiting linked health-related databases are indisputable. However, the potential pitfalls that may arise from not properly accounting for the administrative nature of this information have fuelled a debate on its utility. A promising approach that addresses (some of) these concerns advocates the implementation of trial design principles when exploiting observational data in CER. It consists of emulating the trial that would be designed to study the effectiveness of a treatment by specifying the target population and target comparative measures, and then handling/analysing the observational data to replicate them. There are still several unmet challenges for a robust adoption of this approach, however:
Study design:
Erroneous decisions at any step of the construction of the emulated trial may affect the robustness of the reported evidence. Awareness of potential methodological pitfalls is essential for the interpretation and delivery of evidence.
Data quality:
The coarseness often affecting observational data and the likely dependence of the timing of the observations on factors related to the disease evolution, impact on the robustness of the adopted estimation approach.
Estimation:
Most CER involves treatments sustained over time and requires implementing g-methods for estimation. G-estimation of structural nested models is the most flexible, particularly for dealing with multiple confounders and/or time-points, but has not been fully exploited in applications, nor extended to deal with data coarseness and dependent follow-up.
Given these challenges, this application aims to:
A) Create a roadmap for the assessment and robust delivery of evidence that adopts the "emulate the target trial" approach.
B) Extend g-estimation to deal with data coarseness and dependent follow-up.
C) Use clinically relevant exemplars to illustrate both approach and methodological developments.

This research proposal, if awarded, would have several beneficiaries. These include:

(a) The broader research community
This research will develop formal tools for the assessment of comparative effectiveness studies arising from electronic health records, as well as develop new tools appropriate for the type of linked health data now increasingly available. This will enhance the ability of all researchers, particularly those involved in systemic reviews and bioinformatics, to generate, evaluate and compare research outputs from different sources. This project will also demonstrate the feasibility of close collaborations between experts in different academic disciplines (statistical methodologists, epidemiologists, health informaticians, and clinicians) and will create a network of users via the dedicated website and planned short courses. This environment will also enhance the professional development of future generations of data science researchers.

(b) Evaluators and policy makers
This project will improve the exploitation of routine health records for the evaluation of evidence on clinical effectiveness (and/or harms) as experienced in health and other services (the "real world"). One additional gain will be the ability to reconcile inconsistencies in the available evidence when this arises from differences in key features of the design and/or analysis of the observational data, as highlighted in the proposal. The results from the three exemplars will have implications for policymakers involved in the prevention of premature mortality and CHD risk in T2 diabetic patients, understanding wider effects of antibiotic use and the potential for preventing onset of asthma in children, and the benefits of ensuring adherence to effective treatment of high risk infants hospitalised with acute bronchiolitis. These exemplars will also act as catalysts for setting up investigations into the effectiveness and harms of other treatments in different populations.

(c) Clinicians and patients
The guidelines, methodological developments and tools we plan to provide will aid researchers to address questions regarding benefits and harms of treatments and interventions for which randomised controlled trials (RCTs) may be unfeasible, not yet available, or unethical. It will also allow investigation of more diverse and larger study populations than is usually achievable with RCTs and hence be relevant to the "the real world" and hence to clinicians and patients in the very near future. Our exemplars are targeting questions of clinical uncertainty and which are of interest to patients and the public as well as to clinicians, commissioners and decision makers.