Multivariate meta-analysis of multiple correlated outcomes: development and application of methods, with empirical investigation of clinical impact

'Meta-analysis' is the statistical approach for combining results from multiple studies examining the same clinical question, such as whether a treatment is effective or not. The aim of a meta-analysis is to combine the results from all studies, to produce an overall summary of the findings (e.g. whether the treatment is effective) which helps doctors and healthcare professionals make patient-related decisions based on all the evidence available.

Reliable data to be combined in a meta-analysis are expensive and hard to come by; for example, they often come from clinical trials that have recruited hundreds of patients and followed them up for several years. It is thus important that when researchers undertake a meta-analysis they use all the available data in the most efficient way, to get the most of it. Yet, unfortunately in current practice most researchers are ignoring valuable information that is contained in the available meta-analysis data. This is because many healthcare studies have more than outcome of interest, such as time to recurrence of disease (called 'disease-free survival') and time to death (called 'overall survival'), and researchers currently meta-analyse each outcome separately. However, such multiple outcomes are often related to each other, i.e. they are correlated. For example, a patient's time to recurrence of disease is generally associated with their time of death, as when disease returns the time to death will often follow shortly after. So disease-free survival and overall survival results are correlated, but by meta-analysing each outcome independently, researcher ignores this correlation and thus lose potentially valuable information. In particular, if a study reports overall survival results, but not disease-free survival results, this study can still provide information about disease-free survival by recognising the correlation this outcome has with the available overall survival information. Yet at the moment most researchers simply 'throw away' this study when meta-analysing the disease-free survival results, and thus waste available data.

We believe that a better approach is to meta-analyse correlated outcomes together and utilise their correlated information to get the most out of the available data. The way to do this is a 'multivariate meta-analysis', which is a statistical approach for meta-analysing all the outcomes simultaneously whilst recognising and accounting for their correlation. This allows meta-analysis results for, say, outcome A to be informed by evidence from studies reporting outcome A and also from studies reporting other correlated outcomes B, C and D. As this approach uses more of the information available, it can potentially lead to more reliable results for doctors and healthcare professionals.

Our proposed research project aims to facilitate, and encourage more widespread use of, multivariate meta-analysis in current practice. We will firstly perform an empirical investigation as to how the use of multivariate meta-analysis would change existing clinical conclusions with Reviews of the Cochrane Pregnancy and Childbirth Group, to promote why the approach is important and to update clinical conclusions for pregnancy and childbirth based on more information. We will then develop statistical measures that 'flag' when multivariate meta-analysis is beneficial, so to directly inform researchers when they should use the multivariate approach. We will then develop and extend methods to estimating the correlation between outcomes, which are needed to apply the multivariate approach but are often missing. Finally, we will develop multivariate methods for meta-analysing multiple adverse outcomes. Current methods deal with more common outcomes, but adverse outcomes are rare and so more appropriate methods are needed for such situations. We will also produce a multivariate meta-analysis website, suitable statistical software, and a training course for researchers.

Following a systematic review, multiple meta-analyses are often performed because multiple outcomes are of interest. For example, in meta-analyses of cancer trials the treatment effect on both overall survival and disease-free survival is important. Such multiple outcomes are not independent; e.g. a patient's time to recurrence often occurs shortly before their time of death, and so disease-free and overall survival are positively correlated. Most reviewers ignore this correlation and simply use a 'univariate' meta-analysis of each outcome independently. Alternatively, multivariate meta-analysis methods can jointly synthesise the outcomes and properly account for their correlation. This allows meta-analysis results for, say, outcome A to be informed by evidence from studies reporting outcome A and - crucially - also from studies reporting only correlated outcome B. Compared to univariate meta-analysis, this greater use of available information leads to summary results with improved statistical properties and potentially even different clinical conclusions.

This project aims to facilitate the use of multivariate meta-analysis in practice, by showing its clinical impact and overcoming methodology challenges. We will examine how multivariate meta-analyses changes existing clinical inferences within Cochrane Pregnancy and Childbirth Reviews, where correlated outcomes are common. We will derive statistical measures that reveal the impact of utilising correlation and thus 'flag' when multivariate meta-analysis is important. We will then develop and extend methods for estimating the correlation between outcomes, which are needed to apply the multivariate approach; situations involving individual patient data and only summary data will be considered for binary, continuous, and survival outcomes. Finally, we will develop methods for meta-analysing correlated adverse (rare) outcomes, for which within-study distributions other than multivariate normality are needed.

The key aim of the research project is to facilitate greater use of multivariate meta-analysis methods in practice; such methods provide evidence-based results that will impact upon clinician decision-making and patient care. The scope of potential multivariate meta-analysis applications in healthcare is broad, essentially to wherever multiple outcomes or multiple endpoints are of interest. We have observed application to therapeutic interventions [1], dentistry [2], surrogate outcomes [3, 4], predictive biomarkers [5], diagnostic tests[6], genetic epidemiology [7], and risk prediction / multivariable modelling [8], amongst others. The proposed research will provide clinical conclusions whenever it is applied in such settings, and will help clinicians and healthcare professionals make evidence-based decisions about, for example, the best treatment effects, prognostic factors, or diagnostic tests in relation to multiple outcomes that are relevant. Policy-makers both on a national and international level, such as EMA, FDA and NICE, who are making decisions on clinical practice based on all the evidence will benefit from results from the multivariate approach. Specifically, during our project the application of our methods to the Cochrane Pregnancy and Childbirth Reviews may change existing results and clinical conclusions, and this will be fed back to the original reviewers, clinicians, other healthcare professionals, and decision-makers in the field to consider in amended clinical guidelines.

The multivariate meta-analysis approach will also benefit patients. As the multivariate approach uses more information available in the data compared to currently applied meta-analysis methods, this potentially leads to more reliable (less biased) results; this means that summary estimates from a multivariate analysis should be closer to the truth, and so more reliable clinical conclusions can be drawn, that ensure the most appropriate patient care. The method will also support the endeavours of study and trials participants. Such people have agreed to participate to enhance medical research and want their data to be utilised in full; the multivariate approach will achieve this by utilising the correlation between outcomes to extract more information than is currently being used.

Our project also has large potential impact on members and guideline-makers in the Cochrane Collaboration, who need advice on when to recommend multivariate meta-analyses of therapeutic trials, and how they should be performed. Our guidelines, recommendations and methods development will be disseminated back to them for consideration in updates of the Cochrane Handbook [9] and RevMan, their review software.

References
1. Berkey CS, et al: Multiple-outcome meta-analysis of clinical trials. Stat Med 1996, 15:537-557.
2. Berkey CS, et al: Multiple-outcomes meta-analysis of treatments for periodontal disease. J Dent Res 1995, 74:1030-1039.
3. Daniels MJ, Hughes MD: Meta-analysis for the evaluation of potential surrogate markers. Stat Med 1997, 16:1965-1982.
4. Gail MH, et al: On meta-analytic assessment of surrogate outcomes. Biostatistics 2000, 1:231-246.
5. Riley RD, et al.: An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes. Stat Med 2007, 26:78-97.
6. Reitsma JB, et al.: Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005, 58:982-990.
7. Thompson JR, et al: Meta-analysis of genetic studies using Mendelian randomization--a multivariate approach. Stat Med 2005, 24:2241-2254.
8. Gasparrini A, Armstrong B: Multivariate meta-analysis: A method to summarize non-linear associations. Stat Med 2011, 30:2504-2506.
9. Higgins JPT, Green S, (editors). Cochrane Handbook for Systematic Reviews of Interventions. Chichester: John Wiley & Sons; 2008.