Driving up the quality and relevance of research-based knowledge through the identification and use of core outcomes: the COMET Initiative

Lead Research Organisation: University of Liverpool
Department Name: Institute of Translational Medicine

Abstract

People doing clinical trials and other types of health research often struggle when trying to choose the outcomes to measure which would be of most use to the patients, practitioners and policy makers who will use their research to help them make decisions. These difficulties for trialists are passed on to those producing and reading systematic reviews of trials, many of whom have experienced the frustration of finding that the original researchers either did not measure certain outcomes or measured them in such different ways that it is difficult or impossible to compare, contrast or combine the studies.

Much could be gained if there was an agreed minimum set of core outcomes for each medical condition, which were measured and reported in all clinical trials in that area. Last year, the COMET Initiative was launched to bring together an international network of individuals and organisations interested in the development, application and promotion of such core outcome sets.

COMET aims to collate relevant resources in a publically available, searchable database; to provide guidance on methods for developing core outcome sets; and to develop reporting standards for such studies. Through personal contacts we already know of work on core outcome sets in over 70 areas but there is likely to be much more than this. This proposal is for a two year project to consolidate and develop the COMET Initiative, that has started and that is supported by many stakeholders. The project will include three workstreams: core activity, research, and strategic development.

Core activity will include periodic searching of the literature to identify all relevant work and keep the database up to date, provision of advice to researchers, delivery of training workshops, collation of information about methodological research, organisation of international meetings, and development of the website as a resource for those interested in outcome measurement. We will also examine the level and nature of any patient involvement in this work, since examples exist where patients have identified an outcome important to them as a group that might not have been considered if the core outcome set was developed by practitioners on their own.

An initial review of the studies identified to date shows that the methods used to develop core outcome sets vary and there is limited empirical evidence regarding whether different methods result in similar or different conclusions. Our research will involve undertaking a survey of researchers to understand their experience of core outcome set development and their choice of approach. From the survey, we will identify key methodological research questions and subsequently undertake feasibility work alongside ongoing core outcome set development in particular clinical areas.
This workstream will provide a starting point for methodological research in this area, with the goal of improving methodological standards for COS development.

Our strategic development will involve working towards three main goals: (i) to increase the number of core outcome sets developed using evidence-based methods, (ii) to increase their impact on the quality of research-based knowledge, by raising awareness and increasing implementation, and (iii) to establish methods for standards of core outcome set development.

Technical Summary

A 'core outcome set' (COS) is a set of outcomes which should be measured and reported, as a minimum, in all clinical trials for a specific condition. COS could have implications across all areas of research in health and health care, reduce heterogeneity between trials, lead to research that is more likely to have measured relevant outcomes, and be of potential value in clinical audit. COS would enhance the value of evidence synthesis by reducing the risk of outcome reporting bias and ensuring that all trials contribute usable information. COS will increase the efficiency of the research process.

The COMET (Core Outcome Measures in Effectiveness Trials, http://www.comet-initiative.org.uk) Initiative brings together researchers interested in the development and application of COS. The objectives of this 2-year project are to consolidate and develop the ongoing COMET Initiative, which is supported by many stakeholders. There are three workstreams (section 3.3): core activity, research, and strategic development.

We will systematically search for and classify studies that have determined which outcomes to measure in clinical trials, and include them in a publically available internet-based resource. Our research objectives are (i) to undertake a survey of COS developers to understand their choice of approach, to describe their experience of COS development and implementation, and to identify priority areas for methodological research, and (ii) to undertake feasibility work for methodological projects alongside COS development.

The results of our work will provide a useful, central resource for clinical trialists, systematic reviewers, trial funders, regulators and guideline developers, so they may identify existing core outcome sets in specific clinical areas. Our feasibility studies will benefit other academics working in the fields of consensus methodology research. The project has implications across all areas of research in health and healthcare.

Planned Impact

People doing clinical trials and other types of health research often struggle when trying to choose the outcomes to measure which would be of most use to the patients, practitioners and policy makers who will use their research to help them make decisions. These difficulties for trialists are passed on to those producing and reading systematic reviews of trials, many of whom have experienced the frustration of finding that the original researchers either did not measure certain outcomes or measured them in such different ways that it is difficult or impossible to compare, contrast or combine the studies.

A 'core outcome set' (COS) is a set of outcomes which should be measured and reported, as a minimum, in all clinical trials for a specific condition. COS could have implications across all areas of research in health and health care, reduce heterogeneity between trials, lead to research that is more likely to have measured relevant outcomes, and be of potential value in clinical audit. COS would enhance the value of evidence synthesis by reducing the risk of outcome reporting bias and ensuring that all trials contribute usable information. COS will increase the efficiency of the research process, by reducing the inefficiencies of trials of similar topics measuring outcomes in different ways.

The results of our work will provide a valuable, central resource for clinical trialists, systematic reviewers, trial funders, health service users, clinical teams, medical sociologists, journal editors, policy makers, trials registries regulators and guideline developers, by identifying the consensus about outcomes that are important in particular clinical areas.

The work will benefit other researchers in the field wishing to develop new core outcome sets, by developing a unique resource which should serve to minimise duplication, and by providing a starting point for methodological research in this area, with the goal of improving methodological standards for COS development.

Outcomes need to be relevant to health service users and other people making decisions about health care, if the research findings of pragmatic trials are to influence health-care decision-making. Patients and the public will be able to access information on outcomes recommended for clinical research.

Publications

10 25 50
 
Description HTA outline proposal form now includes reference to COMET
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Paediatric asthma COS/COMET: response to EMA
Geographic Reach Asia 
Policy Influence Type Contribution to a national consultation/review
 
Description SPIRIT 2013 guidance for protocols of clinical trials now includes reference to COMET
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Butterfield Awards for UK-Japan Collaboration in Medicine and Health
Amount £3,000 (GBP)
Organisation The Great Britain Sasakawa Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 08/2012 
End 08/2013
 
Description European Commission H2020
Amount £138,381 (GBP)
Funding ID CORBEL 65428 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 08/2015 
End 08/2019
 
Description MRC Network of Hubs for Trials Methodology Research N55
Amount £14,950 (GBP)
Funding ID N55 
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 09/2014 
End 03/2016
 
Description RCS The PreBRA Study
Amount £61,470 (GBP)
Organisation Royal College of Surgeons of England 
Sector Charity/Non Profit
Country United Kingdom
Start 09/2018 
End 10/2019
 
Title The COMET Database (redesign) 
Description The COMET Initiative has developed a database of all studies relevant to the development of core outcome sets for use in clinical trials. It is maintained through regular updates and an annual systematic review to make sure it contains the most up to date studies. The COMET website was redesigned in January 2020 to provide a more contemporary layout that makes better use of space. The database has also undergone changes to both the search functionality and the results display. It now uses smarter searching to allow searches to be run more easily, and the search results are displayed whereby studies that are part of the same piece of research are grouped together. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact Other COS developers search this database when writing grant proposals, study protocols etc. It is also a repository of ongoing or planned work. Through this, we have successfully avoided duplication of work and facilitated international collaborations. To date, the website has had 56,893 database searches and 177,017 unique visitors from 199 countries. 
URL https://www.comet-initiative.org/Studies
 
Description MRC Trials Methodology Research Partnership 
Organisation University of Leeds
Country United Kingdom 
Sector Academic/University 
PI Contribution Co-lead
Collaborator Contribution Co-leads
Impact Multidisciplinary. Trialists, social scientist, methodologist
Start Year 2019
 
Description MRC Trials Methodology Research Partnership 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Co-lead
Collaborator Contribution Co-leads
Impact Multidisciplinary. Trialists, social scientist, methodologist
Start Year 2019
 
Description MRC Trials Methodology Research Partnership 
Organisation University of Oxford
Country United Kingdom 
Sector Academic/University 
PI Contribution Co-lead
Collaborator Contribution Co-leads
Impact Multidisciplinary. Trialists, social scientist, methodologist
Start Year 2019
 
Title COMET Website (redesign) 
Description The COMET website hosts a searchable database. The website is currently live. The website was redesigned in 2020 using funding from the NIHR Senior Investigator award. 
Type Support Tool - For Fundamental Research
Current Stage Of Development Small-scale adoption
Year Development Stage Completed 2020
Development Status Under active development/distribution
Impact To date, the website has had 56,893 database searches and 177,017 unique visitors from 199 countries. 
URL http://www.comet-initiative.org/
 
Description British assosiation of hand therapists 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Presentation followed by discussion and panel discussion.

To be determined.
Year(s) Of Engagement Activity 2014
 
Description COMET talk, 2nd UK Clinical Trials Methodology Conference, Edinburgh, 2013 (LG) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Paper Presentation
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Abstract accepted to present systematic review

Not known yet.
Year(s) Of Engagement Activity 2013
 
Description COMET talk, ASGBI, UK, 2013 (JB) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach National
Primary Audience Health professionals
Results and Impact Delivered talk to surgeons about importance of choice of outcomes, which included COMET.

Not known yet.
Year(s) Of Engagement Activity 2013
 
Description COMET talk, ECRIN 2013 PW 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The aims and progress of the COMET Initiative were described

The talk raised awareness amongst international clinical research groups
Year(s) Of Engagement Activity 2013
 
Description COMET talk, Exploiting Existing Data for Health Research" 4th International Conference hosted by the Scottish Health Informatics Programme in association with the four UK Ehealth Informatics Research Centres, St Andrews, 2013 (PW) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Paper Presentation
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Abstract accepted for oral presentation.

Opened up a new area of interest.
Year(s) Of Engagement Activity 2013
 
Description COMET talk, HSRN Symposium 2013, UK (PW) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Paper Presentation
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Abstract selected for oral presentation.

Not known yet.
Year(s) Of Engagement Activity 2013
 
Description COMET talk, InFACT/ISF Outcome Measures Colloquium, Amsterdam, 2013 (PW) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Talk delivered and active participation in workshop.

Ongoing collaboration.
Year(s) Of Engagement Activity 2013
 
Description COMET talk, Network of Hubs for Trials Methodology Annual Meeting, Oxford, 2013 (LG) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Talk delivered. Question and answer session.

Slides of presentation shared.
Year(s) Of Engagement Activity 2013
 
Description COMET talk, PSI Annual conference, Glasgow, 2013 (PW) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Delivered to wider audience.

Talk well received, audience participation and questions. Advice provided.
Year(s) Of Engagement Activity 2013
 
Description COMET talk, Workshop on the methods for benefit-risk analysis in drug regulation and health technology assessment, Bangor, Jan 2013 (PW) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Delivered to specialist audience, increased awareness and interest in the COMET Initiative.

Further discussions.
Year(s) Of Engagement Activity 2013
 
Description COMET workshop, 21st Cochrane Colloquium, Canada, 2013 (PW) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Workshop Facilitator
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Interactive workshop delivered, attended by 25 individuals.

Feedback sought from systematic reviewers about suitability of workshop for this audience.
Year(s) Of Engagement Activity 2013
 
Description COMET workshop, Kyoto, Japan, 2013 (PW, MC and LG) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Workshop Facilitator
Geographic Reach International
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact 2 day workshop delivered, attended by approximately 100 individuals from all over Japan.

Introduced Japanese researchers to the COMET Initiative and the idea of core outcome sets. Recieved excellent feedback and was very well received. On-going discussion regarding possibility of future collaboration.
Year(s) Of Engagement Activity 2013
 
Description European school of oncology masterclass 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A presentation followed by small group work.

Introduced core outcome sets to a new audicence.
Year(s) Of Engagement Activity 2014
 
Description Invited talk: Population Health Sciences launch meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact The Bluebelle Study - qualitative methods to inform trial design
Year(s) Of Engagement Activity 2017
 
Description Joint webinar between the COMET Initiative and EURORDIS (HB) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Raising awareness of core outcome sets and the work of the COMET initiative

NA
Year(s) Of Engagement Activity 2015
 
Description Talk, GRADE/DECIDE joint meeting, March 2015, Amsterdam 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Raising awareness of core outcome sets and the work of the COMET initiative
Year(s) Of Engagement Activity 2015
 
Description Talk, German Pain Congress, 14-17 June 2015, Mannheim LG 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Raising awareness of core outcome sets and the work of the COMET initiative
Year(s) Of Engagement Activity 2015
 
Description Talk, HRB-TMRN Symposium, Dublin 2015 PW 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Raising awareness of core outcome sets and the work of the COMET initiative
Year(s) Of Engagement Activity 2015
 
Description Talk, HTAi 12th Annual meeting 2015 LG 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Raising awareness of core outcome sets and the work of the COMET initiative
Year(s) Of Engagement Activity 2015
 
Description Talk, ISPOR 18th Annual European Congress, November 2015, Milan LG 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Raising awareness of core outcome sets and the work of the COMET initiative
Year(s) Of Engagement Activity 2015
 
Description Talk, RCOG, January 2015 PW 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Raising awareness of core outcome sets and the work of the COMET initiative
Year(s) Of Engagement Activity 2015
 
Description Talk, Second Annual Conference of the International Collaborative for Best Care for the Dying Person, 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Raising awareness of core outcome sets and the work of the COMET initiative
Year(s) Of Engagement Activity 2015
 
Description UK and Ireland Cochrane meeting 2014 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Health professionals
Results and Impact We delivered the COMET workshop to a wide audience (e.g.researchers, systmetaic reviewers, policy makers) which sparked discussion and group work exercises.

Raised awareness of the COMET Initiative, demonstrated by an increased number of visits to the website (99 visits that day).
Year(s) Of Engagement Activity 2014