Real-world treatment effectiveness in people with type 2 diabetes: Maximising the applicability of clinical trials
Lead Research Organisation:
University of Glasgow
Department Name: College of Medical, Veterinary, Life Sci
Abstract
The best way to know whether a new medicine works is to perform a clinical trial. In clinical trials, participants are selected to have different medicines at random (ie by chance). As a result, participants receiving different treatments are, on average, similar. Consequently, researchers can compare groups receiving different treatments, and decide which are the most effective.
However, trial participants are commonly younger and fitter than other patients. As a result, clinicians and others have expressed uncertainty as to whether results from trials are relevant to many patients in "real-world" settings.
To address this concern, some researchers have instead studied treatments using routine data. Unlike with trials, routine data can be collected from all patients receiving healthcare (eg from medical records) including many older frailer patients. However, when this kind of data is used to compare treatments it is very difficult to be sure (even with very sophisticated analyses) that any differences have been caused by the medication. This is because, unlike in trials, different treatments are offered to patients because of differences in their clinical features.
In this project, we propose to combine clinical trial and routine data, using the strengths of both. We will use routine data to "calibrate" trial results. When trial results are calibrated, findings from under-represented groups (eg older women) influence the overall results more than findings from other over-represented groups (eg younger men). After calibration we can be more confident that the trial results are relevant. Also, calibration does not "break" the randomness; calibrated results remain reliable.
Some older calibration methods required researchers to have access to very detailed results from every relevant trial (eg the result for every trial participant). In most situations, this meant calibration was unfeasible. However, we recently developed a method to perform calibration which does not require this level of detail for every trial. This calibration feasible for many more conditions and treatments. We now propose, for the first time, to use this new method to calibrate trials using routine data.
Specifically, we will perform the calibration to decide which of the newer diabetes medicines are most effective in real-world patients in Scotland and China. We will obtain routine data from a complete register of people with diabetes in Scotland and from two hospitals in China. Having identified a group of patients suitable for treatment with the newer medicines, we will calculate the likely benefits and harms of each of the newer medicines as if the original clinical trials had been conducted in China or in Scotland.
We will produce an overall summary result from all the trials, making this available to clinicians and people with diabetes. We will also feed the results into a health economic model to predict the likely costs, benefits and value for money. Such models are used by organisations such as NICE to inform guidelines and regulations about medicines.
To better communicate our findings about the effectiveness and value for money of each medicine, we will develop an interactive web app, designed to be used by researchers, clinicians and people with diabetes. It will allow users to compare results which have been obtained the conventional way, alongside results obtained using calibration.
If funded, this project will produce results about differences in the effectiveness of newer drugs for diabetes that are reliable, and that clinicians can confidently apply to patients with diabetes in real-world settings. As well producing tangible benefits for people with diabetes, this will also demonstrate, for the first time, that calibration can improve the relevance of trial results.
However, trial participants are commonly younger and fitter than other patients. As a result, clinicians and others have expressed uncertainty as to whether results from trials are relevant to many patients in "real-world" settings.
To address this concern, some researchers have instead studied treatments using routine data. Unlike with trials, routine data can be collected from all patients receiving healthcare (eg from medical records) including many older frailer patients. However, when this kind of data is used to compare treatments it is very difficult to be sure (even with very sophisticated analyses) that any differences have been caused by the medication. This is because, unlike in trials, different treatments are offered to patients because of differences in their clinical features.
In this project, we propose to combine clinical trial and routine data, using the strengths of both. We will use routine data to "calibrate" trial results. When trial results are calibrated, findings from under-represented groups (eg older women) influence the overall results more than findings from other over-represented groups (eg younger men). After calibration we can be more confident that the trial results are relevant. Also, calibration does not "break" the randomness; calibrated results remain reliable.
Some older calibration methods required researchers to have access to very detailed results from every relevant trial (eg the result for every trial participant). In most situations, this meant calibration was unfeasible. However, we recently developed a method to perform calibration which does not require this level of detail for every trial. This calibration feasible for many more conditions and treatments. We now propose, for the first time, to use this new method to calibrate trials using routine data.
Specifically, we will perform the calibration to decide which of the newer diabetes medicines are most effective in real-world patients in Scotland and China. We will obtain routine data from a complete register of people with diabetes in Scotland and from two hospitals in China. Having identified a group of patients suitable for treatment with the newer medicines, we will calculate the likely benefits and harms of each of the newer medicines as if the original clinical trials had been conducted in China or in Scotland.
We will produce an overall summary result from all the trials, making this available to clinicians and people with diabetes. We will also feed the results into a health economic model to predict the likely costs, benefits and value for money. Such models are used by organisations such as NICE to inform guidelines and regulations about medicines.
To better communicate our findings about the effectiveness and value for money of each medicine, we will develop an interactive web app, designed to be used by researchers, clinicians and people with diabetes. It will allow users to compare results which have been obtained the conventional way, alongside results obtained using calibration.
If funded, this project will produce results about differences in the effectiveness of newer drugs for diabetes that are reliable, and that clinicians can confidently apply to patients with diabetes in real-world settings. As well producing tangible benefits for people with diabetes, this will also demonstrate, for the first time, that calibration can improve the relevance of trial results.
Technical Summary
Randomised controlled trials are the gold standard for obtaining unbiased estimates of treatment effects. However, older patients and those with co-morbidity are frequently underrepresented. Consequently, for many "real-world" settings, the applicability of trial findings is uncertain.
We propose to address this uncertainty using calibration. Following calibration trial participants who are overrepresented (with respect to the 'target' population) contribute less to the overall result, while participants who are underrepresented contribute more. Calibration improves representativeness, while preserving randomisation and hence validity.
Hitherto, calibration required individual-level participant data (IPD) for all relevant trials, making it unfeasible in most settings. However, in an MRC-funded methodology grant we developed and validated a novel method of calibration, which can be used even where access to IPD is restricted to a subset of the relevant trials. Together with our recent experience in accessing and analysing IPD for industry-funded trials, this makes calibration a feasible tool for improving trial applicability.
NICE guidance on novel drugs for type 2 diabetes (DPP-4 and SGLT2-inhibitors and GLP-1 agonists) noted that the underpinning trial evidence was unrepresentative. Therefore, using IPD, sub-group results, and overall results with baseline characteristics, we will conduct a calibrated network meta-analysis of relevant trials to compare the efficacy of these drug classes, subsequently applying these estimates to an existing health economic model. We will calibrate to the Scotland-wide Diabetes Register, and to two cohorts of people with diabetes based in China.
We will produce estimates for people with type 2 diabetes that are both valid and applicable. We will also demonstrate, for the first time, that calibration can be used to improve the relevance of trial results in realistic settings, where IPD is only available for some trials.
We propose to address this uncertainty using calibration. Following calibration trial participants who are overrepresented (with respect to the 'target' population) contribute less to the overall result, while participants who are underrepresented contribute more. Calibration improves representativeness, while preserving randomisation and hence validity.
Hitherto, calibration required individual-level participant data (IPD) for all relevant trials, making it unfeasible in most settings. However, in an MRC-funded methodology grant we developed and validated a novel method of calibration, which can be used even where access to IPD is restricted to a subset of the relevant trials. Together with our recent experience in accessing and analysing IPD for industry-funded trials, this makes calibration a feasible tool for improving trial applicability.
NICE guidance on novel drugs for type 2 diabetes (DPP-4 and SGLT2-inhibitors and GLP-1 agonists) noted that the underpinning trial evidence was unrepresentative. Therefore, using IPD, sub-group results, and overall results with baseline characteristics, we will conduct a calibrated network meta-analysis of relevant trials to compare the efficacy of these drug classes, subsequently applying these estimates to an existing health economic model. We will calibrate to the Scotland-wide Diabetes Register, and to two cohorts of people with diabetes based in China.
We will produce estimates for people with type 2 diabetes that are both valid and applicable. We will also demonstrate, for the first time, that calibration can be used to improve the relevance of trial results in realistic settings, where IPD is only available for some trials.
Planned Impact
The main impact is described in the pathways to impact document.
Briefly, other than academic beneficiaries, the main beneficiaries of this work will be people with type 2 diabetes, clinicians working in type 2 diabetes, and guideline developers (eg the National Institute for Health and Care Excellence).
Briefly, other than academic beneficiaries, the main beneficiaries of this work will be people with type 2 diabetes, clinicians working in type 2 diabetes, and guideline developers (eg the National Institute for Health and Care Excellence).
Organisations
Publications
