Systematic Reviews and Meta-Analysis of Prognosis Studies (REVAMP): development of core methods, reporting guidelines and a methodology handbook

Healthcare research is in an exciting new phase. Researchers now have access to large amounts of information to link an individual's characteristics (such as age, family history or genetic information) with health outcomes (such as death, pain level, depression score). Researchers are using this information to help health professionals and patients understand and predict future outcomes. Such research is called prognosis research, and covers four types: overall prognosis (what is the average outcome risk in a population?), prognostic factors (what characteristics of patients help understand their outcome risk?), prognostic models (can we predict an individual's risk using multiple prognostic factors?), and predictors of treatment effect (can we predict which treatments work best for individual patients?). Prognosis research findings allow health professionals to more accurately predict and therefore intervene to improve the health outcomes of individuals.

Thousands of prognosis studies are published each year, and many have unclear or contradictory findings. This makes it difficult for healthcare professionals to know how best to use these findings to improve the care of their patients. This motivates the need for systematic reviews of prognosis studies. These aim to identify all existing prognosis studies on a particular topic, and to appraise their quality and, if appropriate, combine their results (using meta-analysis) to provide overall answers for healthcare professionals. Systematic review methods are well-established in other areas of medical research, but are only just emerging for prognosis research. In our project, we will develop and make freely available reliable methods to undertake systematic reviews of prognosis studies. We have named this initiative REVAMP, which stands for Systematic Reviews and Meta-Analysis of Prognosis Studies.

We propose a series of five work projects (WPs) to be delivered by researchers with considerable expertise in prognosis research and systematic reviews. In WP1, we will develop and disseminate guidance for reviews of overall prognosis; which aim to provide a summary of the average risk in a particular population (e.g. those diagnosed with breast cancer). A step-by-step guide will be produced, covering aspects such as searching for relevant studies, data extraction, critical appraisal, and meta-analysis.

Clear reporting is essential for those interpreting, critically appraising, and implementing prognosis research findings. However, no reporting guidelines currently exist for systematic reviews of prognosis studies. Therefore, in WP2 we will provide three reporting guidelines (checklists) for prognosis reviews: one for each of overall prognosis, prognostic factors and prognostic models. These will be developed through a consensus process involving researchers, healthcare professionals, organisations developing clinical guidelines, and patients.

In WP3 and 5, we will develop new ways for researchers to calculate how many individuals are needed for a prognosis study. Current studies are often too small and so may have misleading results. We will use mathematical techniques to produce a new way to calculate sample size, so that it will be easier to judge (i) whether an existing prognostic model study was large enough to be fit-for-purpose, and (ii) what size a new prognosis study needs to be to contribute usefully to a future updated review.

In WP4 we will produce step-by-step guidance for conducting reviews of predictors of treatment effect, particularly how to do critical appraisal and meta-analysis when the aim is to summarise whether a treatment's effect depends on patient characteristics such as age.

Our project's findings will be disseminated in a Methodology Handbook for Prognosis Reviews and through a new UK-based research training course, in addition to publications in journals and via blogs and tutorial videos at websites including www.prognosisresearch.com.

Precision medicine relies heavily on prognosis research findings, which allow health professionals to understand and predict outcomes in patients with a particular disease or condition. The PROGRESS framework defines four types of prognosis research: (i) overall prognosis, (ii) prognostic factors, (iii) prognostic models and (iv) predictors of treatment effect. The implementation of prognosis research findings should be evidence-based, with multiple high-quality studies confirming their accuracy and reliability. This motivates the need for systematic reviews and meta-analyses of prognosis studies to identify, critically appraise, synthesise and summarise the evidence.

Although guidance is well developed for reviews of intervention studies, the methodology for prognosis reviews is only just emerging, and is especially sparse for overall prognosis and predictors of treatment effect. Yet prognosis reviews are rapidly being undertaken, as seen from living reviews of COVID-19 prognosis studies. Hence, there is an urgent need to develop robust methods and guidance to support those undertaking prognosis reviews, including Cochrane, Evidence Based Medicine centres, and decision makers such as NICE. Our grant will address this via five work packages (WPs), covering step-by-step guidance for reviews of overall prognosis (WP1); reporting guidelines for types (i) to (iii) (WP2); equations for appraising the sample size of prognostic model studies for risk of bias classifications (e.g. to ascertain overfitting concerns) (WP3); methods to improve risk of bias and meta-analysis in reviews of predictors of treatment effect (WP4); and sample size calculations for new prognosis studies that subsequently will be included in an updated review (WP5).

Our findings will be disseminated in a Handbook for Prognosis Reviews (co-written with Cochrane Prognosis Methods Group), a UK-based training course, journal articles, statistical software, and tutorials at www.prognosisresearch.com.