Modelling generic preference based outcome measures - development and comparison of methods.

Quality Adjusted Life Years (QALYs) are used in most assessments of the cost effectiveness of health interventions. However, there is often an evidence gap between clinical measures of effect that are available and the detailed preference-based information needed to construct QALY measures. Instruments like the EQ-5D have preference-based scoring systems and are favoured by organisations such as the National Institute for Health and Care Excellence (NICE) but are frequently absent from clinical studies of treatment effect.

Often this gap is bridged by "mapping" - estimating a relationship between observed clinical outcomes and preference based measures, using data from another dataset containing both types of information. Unfortunately, we know that the statistical methods routinely used for this mapping often do not work well in practice. They can give seriously biased estimates of the value of treatments, leading to underestimation of cost effectiveness and consequently distorted funding decisions. This has real effects on patients, clinicians, industry and the general public. The same issues arise when these measures are used to assess clinical performance which is increasingly the case. The NHS for example now routinely collects EQ-5D from patients undergoing several common types of surgery in order to identify cases of concern (or good practice).

This project will develop new statistical methods to address this problem, test them, make recommendations about when they should be used and produce computer code to allow them to be applied easily in practice, even by researchers that are not specialist statisticians. It will take as its starting point three relatively new classes of methods which have shown some promise in the area. These are referred to as i) bespoke mixture models, ii) beta regression and iii) multidimensional response mapping. We will develop these methods and generalise them so that they can be applicable to a range of preference based outcome measures.

The models will be compared both in a series of real life datasets and in an extensive simulation study. Based on the results, we will make recommendations to analysts that will guide them in their use of the appropriate methods. We envisage that there will be no single "better" method but that the different models will be more appropriate depending on the circumstances such as the preference based instrument in question and the severity of the patient population.

Computer code, accompanying guidance notes and training materials will be produced and will be made freely available to ensure analysts can easily implement these more advanced methods where appropriate.

Health related quality of life is often assessed using "preference based outcome measures" (PBMs). These are widely used in economic evaluation. EQ-5D is the preferred instrument of NICE and features significantly in the DoH PROMS programme.
Analysts often estimate PBM summary values as a function of other variables. However, all PBMs are limited in their range - at the top by full health and at the bottom by the worst health state described by the PBM. There tends to be a mass of observations at the upper limit of full health. The remainder tends to be non-normal, the degree of which varies by specific instrument and severity of the patient cohort. With EQ-5D, the distribution is tri-modal with a large gap in the range of feasible values between full health and any degree of impairment at 0.883. Here, simple statistical models are inappropriate and many studies have shown this. Such methods are biased: their continued use results in the misrepresentation of the health benefits from treatments and procedures.
Recently, three approaches have shown some promise: a) bespoke mixture models, b) beta regression and c) multidimensional response mapping. These form the starting point for development of new methods applicable to a wide range of PBMs. Developments will include a beta mixture model, a factor analytic approach to deal with measurement error and the incorporation of correlation across dimensions in the response mapping approach, where we will also explore a flexible but challenging approach based on copula representations of the joint distribution.
Comprehensive model comparisons will be made in a simulation study and a series of real life datasets. Eight definitive mapping functions will be produced. Findings will be used to provide nuanced guidance on model choice for analysts conditional on target PBM, severity of patient group and distributional features of the dataset. STATA .ado commands will be written making methods accessible to non-specialists.

There are two broad areas of research where the results and guidance from this project will have direct relevance: economic evaluation and the analysis of patient reported outcomes. And within these two areas there are several groups of beneficiaries.

For the first group, we can consider those that conduct cost effectiveness modelling, those that perform analyses that inform the parameter values for those models and the recipients of those model results. We know that currently almost a quarter of all economic models submitted to NICE rely on a statistical model to estimate the relationship between clinical outcomes and health related quality of life (Kearns et al 2012) and this is likely to be a similar figure for those analyses produced for other decision makers. Economic modellers will benefit from this research because it will provide the means for them to make unbiased estimates of the health benefits of the treatments they are charged with assessing in terms of cost per QALY. It will provide them with simple implementable code and guidance on how to incorporate the results of potentially more advanced statistical approaches into their decision analytic models and the associated uncertainty in the estimates.

These models are currently performed by analysts employed in a range of settings: the private sector including pharmaceutical and medical devices companies, consultancies as well as public and academic sectors. In the NICE context it is often an academic group that is responsible for interrogating a submission made by a sponsoring company. In this situation there are clear incentives for "gaming" by employing methods that result in the most favourable results for a product. This research will provide guidance as to the appropriateness of methods according to the setting, thereby reducing the scope for such behaviour.

In many situations, those responsible for incorporating results into a decision model are not those that produce statistical values that feed into it. Indeed, the Kearns et al review paper identified that most regression models used as inputs to a decision analysis are drawn from the published literature. Health economists, quality of life researchers and statisticians that conduct these analyses will be provided with detailed guidance and computer code that enables them to conduct appropriate analyses of quality of life data and report them in a transparent manner.

Such analyses need not be undertaken with economic evaluation in mind. The Department of Health PROMS initiative for example uses outcome measures like EQ-5D as performance indicators in their own right. Analyses of these data are performed by academics, including those embedded within the various DoH Policy Research Units, staff of the DoH and private sector consultancies.

In all of these situations such analyses are undertaken to inform decisions. That may be around whether to recommend a drug or other health care technology or to assess the efficiency and productivity of hospitals or other treatment units. The use of statistical methods that are not appropriate leads to poor decisions that ultimately impact real patients and practising clinicians. Decision makers require confidence in the analyses presented to them. Whilst this research proposal is methodological in nature, and involves the development of some methods that may be seen as relatively complex, it will retain the focus on informing decision makers at its heart (we have a CI from NICE to help ensure this focus is maintained). In this way methodological advance can lead to better decisions and greater confidence in those decisions.