Fair comparisons of interventions: extending the Prior Event Rate Ratio method
Lead Research Organisation:
Plymouth University
Department Name: Sch of Biomedical & Biological Sciences
Abstract
To assess the effects of a treatment we typically compare health outcomes in a group of patients who received the treatment ? the treated group ? with outcomes in a group who did not ? the untreated group. There are two main approaches to conducting such comparisons: randomized trials and observational studies.
Trials are experiments which involve randomly deciding which patients should be in the treated and untreated groups. Making the selection random ensures the groups are comparable prior to the start of treatment, with no reason to suppose one group will be older or younger, sicker or healthier, and so on. Following treatment, comparison of the two groups gives a fair assessment of the effects of treatment.
Observational studies differ from a randomized trial in that here the researchers do not play an active part in determining who gets the treatment. Information for such studies could come from routinely collected health data, such as the records of General Practitioners. The treated and untreated groups may differ, for example, in terms of severity of illness, or age, or gender. Such factors need to be taken into account when making treatment comparisons or incorrect conclusions may be reached.
Observational studies are often easier to conduct than trials but lack of information about factors influencing who received the treatment can make it hard to tell whether differences between the groups relate to the treatment or to something else. For this reason, randomized trials are usually considered superior ? but they are time-consuming, expensive, and cannot be carried out in some situations for ethical reasons.
The aim of this project is to extend a promising statistical method for using information from observational studies in a way that allows us to discount pre-existing differences between the treated and untreated groups, and focus our attention on the effects of treatment. We can do this by assessing differences between the groups before anyone receives the treatment. Making use of this information allows us to identify consequences relating specifically to the treatment.
Extending this method (the ?Prior Event Rate Ratio?) to a wider range of situations will offer opportunities to assess the effects of treatments when trials are not possible and allow us to identify situations in which trials should be carried out. One important application of this research would be to the development and monitoring of new treatments for chronic health conditions such as arthritis, asthma and diabetes.
Trials are experiments which involve randomly deciding which patients should be in the treated and untreated groups. Making the selection random ensures the groups are comparable prior to the start of treatment, with no reason to suppose one group will be older or younger, sicker or healthier, and so on. Following treatment, comparison of the two groups gives a fair assessment of the effects of treatment.
Observational studies differ from a randomized trial in that here the researchers do not play an active part in determining who gets the treatment. Information for such studies could come from routinely collected health data, such as the records of General Practitioners. The treated and untreated groups may differ, for example, in terms of severity of illness, or age, or gender. Such factors need to be taken into account when making treatment comparisons or incorrect conclusions may be reached.
Observational studies are often easier to conduct than trials but lack of information about factors influencing who received the treatment can make it hard to tell whether differences between the groups relate to the treatment or to something else. For this reason, randomized trials are usually considered superior ? but they are time-consuming, expensive, and cannot be carried out in some situations for ethical reasons.
The aim of this project is to extend a promising statistical method for using information from observational studies in a way that allows us to discount pre-existing differences between the treated and untreated groups, and focus our attention on the effects of treatment. We can do this by assessing differences between the groups before anyone receives the treatment. Making use of this information allows us to identify consequences relating specifically to the treatment.
Extending this method (the ?Prior Event Rate Ratio?) to a wider range of situations will offer opportunities to assess the effects of treatments when trials are not possible and allow us to identify situations in which trials should be carried out. One important application of this research would be to the development and monitoring of new treatments for chronic health conditions such as arthritis, asthma and diabetes.
Technical Summary
In many situations where it is not ethical or practical to perform randomized controlled trials, observational studies provide a rich source of information for assessing effectiveness of treatment interventions. Unfortunately, the modest effects of modern medical treatments can be obscured by the confounding effects of other, often unmeasured, factors. There is a growing literature on methods for addressing residual confounding but recent research highlights the limitations of existing techniques.
A recently proposed and very promising solution to this problem is the Prior Event Rate Ratio (PERR) method. This method is applicable to studies where a new intervention is introduced for the first time and is based on the assumption that differences in outcomes between treated and control groups before intervention reflect the effect of all confounders, independent of the intervention effect. Simple adjustments are then possible to correct estimates of intervention effects for the aggregate effects of confounders. Previous work has shown the potential of the PERR method to remove or reduce residual confounding when analyzing effectiveness of new drug treatments using routine clinical databases.
This project will broaden the scope of this method by investigating its suitability for analysis of intervention effects more generally, including hazardous exposures that would not be possible to test experimentally, in both cohort and routine clinical studies. An extensive simulation study will be performed to explore sensitivity of the method to different assumptions about issues encountered in practice including multiple unmeasured confounders, attrition bias, measurement error and differential mortality. An important contribution of our work will be the extension of the PERR approach to address the analysis of ordinal and interval outcomes, both central to research on chronic health conditions in mid and later life.
The PERR method can be seen as creating a form of ?natural experiment? and has similarities to the ?controlled before-and-after? study design used in psychology and economics. One of our primary objectives will be to investigate differences in model specification and develop an integrated framework for quasi-experimental approaches in epidemiological applications. Following a detailed literature review we will make comparisons with selected alternative approaches to controlling residual confounding, such as instrumental variables, propensity scoring and Bayesian bias analysis.
The proposed work will develop new tools to make more effective use of observational databases, and has the potential to make a substantial contribution to evidence-based medicine and the assessment of risks from environmental hazards not amenable to experimental investigation.
A recently proposed and very promising solution to this problem is the Prior Event Rate Ratio (PERR) method. This method is applicable to studies where a new intervention is introduced for the first time and is based on the assumption that differences in outcomes between treated and control groups before intervention reflect the effect of all confounders, independent of the intervention effect. Simple adjustments are then possible to correct estimates of intervention effects for the aggregate effects of confounders. Previous work has shown the potential of the PERR method to remove or reduce residual confounding when analyzing effectiveness of new drug treatments using routine clinical databases.
This project will broaden the scope of this method by investigating its suitability for analysis of intervention effects more generally, including hazardous exposures that would not be possible to test experimentally, in both cohort and routine clinical studies. An extensive simulation study will be performed to explore sensitivity of the method to different assumptions about issues encountered in practice including multiple unmeasured confounders, attrition bias, measurement error and differential mortality. An important contribution of our work will be the extension of the PERR approach to address the analysis of ordinal and interval outcomes, both central to research on chronic health conditions in mid and later life.
The PERR method can be seen as creating a form of ?natural experiment? and has similarities to the ?controlled before-and-after? study design used in psychology and economics. One of our primary objectives will be to investigate differences in model specification and develop an integrated framework for quasi-experimental approaches in epidemiological applications. Following a detailed literature review we will make comparisons with selected alternative approaches to controlling residual confounding, such as instrumental variables, propensity scoring and Bayesian bias analysis.
The proposed work will develop new tools to make more effective use of observational databases, and has the potential to make a substantial contribution to evidence-based medicine and the assessment of risks from environmental hazards not amenable to experimental investigation.