Pragmatic Evidence-Based Medicine: closing the implementation gap by promoting 'user-friendly' evidence

Lead Research Organisation: University of Oxford
Department Name: Population Health

Abstract

When a patient visits his doctor with a sore back, he wants to know whether the experimental intervention is better than the best existing intervention. This information is best obtained in an ‘active controlled trial‘. Yet, current wisdom has it that the best way to measure effectiveness is in a placebo controlled trial. My thesis examined the arguments that placebo controls are methodologically superior to active controls, and found them to be wanting - the apparent tension between good methods and usefulness is largely illusory. I intend to expand my thesis results in two ways. First, I will conduct social research into what patients/clients, practitioners, and policy makers regard as ‘user-friendly‘ evidence, i.e. evidence that would be easy to put into practice. Second, I intend to investigate the methods for comparing the experimental intervention with all available interventions for a given ailment (rather than just one or two). The impact of my project, both in medical and social science research, could be to have an immediate effect on the delivery and uptake of the most effective interventions.

Technical Summary

Objective I. Disseminate of my thesis findings to methodological and medical , and social intervention communities.
My doctoral and proposed extension of my doctoral work are both of particular interest to both the MRC and the ESRC. In particular my work will be able to inform any revisions of the ESRC‘s Research Methods Programme, and will contribute to the MRC‘s Methodology Research Programme.
An ‘active‘ controlled trial is one that compares the experimental intervention with existing treatment. A placebo controlled trial is one that compares the experimental intervention with a placebo, or ‘sham‘ intervention (i.e. sugar pill). Placebo controlled trials are regarded as methodologically superior to active controlled trials (Temple 2000; ICH E10 2000), but active-controlled trials are more ‘user-friendly‘, since they help the used decide between two or more available alternatives. My investigation concluded that the arguments for the methodological superiority of placebo over active controls are on the whole wanting. The importance of my work for the methodology of social science and medicine, where placebo controls are often difficult to design, is clear.

Objective II: (Thesis Extension) How do the relevant groups in society define user-friendly evidence
The arguments for the superiority of placebo controlled trials are made mostly by statisticians and researchers. It would be interesting to know whether potential users (i.e. patients), practitioners, and policy makers regard the information gained from active controlled trials to be more practically useful. In this section of my research I will conduct a small pilot study consisting of semi-structured interviews of patients, practitioners, policy makers, as well as statisticians and clinical researchers. I will use the results of this study to guide a methodological analysis of indirect comparisons of competing interventions.

Objective III: (Thesis Extension): Maximally User-Friendly Evi useful than simple active controlled trials, studies that compare all available interventions for a given ailment are more useful than studies that compare only two or three interventions. Such a study would providee way to accomplish this is to make ‘indirect comparisons‘ of the effectiveness of each intervention trials that have already been accomplished (Glenny, Altman, et al: 2005). Beyond a few notable examples (Caldwell et al: 2005) the potentially useful indirect comparisons have few advocates and are rarely done. Further, indirect comparisons are regarded as methodologically problematic, not least of all because the populations could be different across trials. I propose to analyze methodological and empirical problems with indirect comparisons of all available interventions.

Publications

10 25 50
 
Description MEMBER OF THE GRADE WORKING GROUP
Geographic Reach Multiple continents/international 
Policy Influence Type Membership of a guideline committee
 
Description Mechanisms in Clinical Research
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a advisory committee
 
Description OCEBM TABLE OF EVIDENCE 2010
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a advisory committee
Impact The OCEBM Levels of Evidence is widely used by patients, caregivers, and policy-makers. The revised version (forthcoming December 2010) is easier to use and the feedback from all stakeholders has been positive.
 
Description Producing Guidelines on the use of mechanistic evidence
Geographic Reach Europe 
Policy Influence Type Membership of a guideline committee
Impact Based on my evidence-based medicine (EBM) expertise--specifically my critique of the EBM refusal to consider evidence from mechanisms, I am a member of a panel with the Dutch Health Ministry to produce guidelines, which I have drafted.
 
Description Setting up centres for evidence-based medicine (EBM) in Lithuania and Uganda
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact Based on my EBM expertise, specifically my ability to translate it for new audiences, I have participated in the founding of centres for EBM in Lithuania and Uganda.
URL https://www.ncbi.nlm.nih.gov/pubmed/25955430
 
Description British Medical Association Dawkins and Strutt
Amount £60,000 (GBP)
Organisation British Medical Association (BMA) 
Sector Learned Society
Country United Kingdom
Start 10/2016 
End 09/2017
 
Description National Institute for Health Research NSPCR Non Clinical Research Fellowship
Amount £250,000 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 10/2012 
End 09/2015
 
Description Research project: Empathica: Expectation Management for Patients in Primary Care: Developing and Feasibility Testing a New Digital Intervention for Practitioners
Amount £394,262 (GBP)
Organisation University of Southampton 
Sector Academic/University
Country United Kingdom
Start 10/2018 
End 09/2020
 
Title CRITICAL THINKING FOR MEDICAL RESEARCH 
Description I HAVE SUCCESSFULLY BRIDGED THE GAP BETWEEN MEDICINE, PHILOSOPHY, AND SOCIAL SCIENCE. MY WORK HAS BEEN PUBLISHED IN BOTH THE MEDICAL AND PHILOSOPHICAL LITERATURE, I HAVE PRESENTED MY WORK TO BOTH DISCIPLINES, AND I HAVE BEEN INVITED, AS AN EXPERT, TO CONTRIBUTE TO DEBATES IN BOTH FIELDS. 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact I HAVE DEMONSTRATED THAT INTER-DISCIPLINARY WORK IS BOTH POSSIBLE, AND FRUITFUL. I'VE INCREASED THE IMPACT OF PHILOSOPHICAL WORK, AND ALSO CONTRIBUTED TO THE EBM 'SYSTEM' OF EVIDENCE. 
 
Description BIOLOGICAL MECHANISMS IN EVIDENCE-BASED MEDICINE 
Organisation Johns Hopkins University
Department Johns Hopkins Bloomberg School of Public Health
Country United States 
Sector Academic/University 
PI Contribution I PROVIDED EXPERTISE ON THE EVIDENTIAL ROLE OF MECHANISMS TO A PANEL AT JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH (BALTIMORE, MARYLAND).
Collaborator Contribution I PROVIDED EXPERTISE ON THE EVIDENTIAL ROLE OF MECHANISMS TO BENCH RESEARCHERS.
Impact (CONFIDENTIAL UNTIL PUBLISHED)
Start Year 2009
 
Description International Point of Care Testing Working Group 
Organisation Cleveland Clinic
Department Lerner Research Institute
Country United States 
Sector Academic/University 
PI Contribution Survey of UK, American, Dutch, and Australian general practitioners regarding their use of, and desire for, various point of care tests.
Collaborator Contribution Survey of UK, American, Dutch, and Australian general practitioners regarding their use of, and desire for, various point of care tests.
Impact International survey of point of care testing use and desire for use.
Start Year 2012
 
Description Oxford Centre for Evidence-Based Medicine Levels of Evidence Working Group 
Organisation University of Oxford
Department Nuffield Department of Primary Care Health Sciences
Country United Kingdom 
Sector Academic/University 
PI Contribution I recently organized an international group of experts on evidence to revise the Oxford Centre for Evidence-Based Medicine Evidence Table. His role involved mediating between diverging positions and coming up with documents that all members of the group endorsed. The group included: Professor Paul Glaziou Dr. Carl Heneghan Hazel Thornton Professor Alessandro Liberatti (Modena, Italy) Dr. Ivan Moschetti (Italian Cochrane Centre) Professor Patricia Greenhalgh
Collaborator Contribution Paul was a chair of the consensus meeting and participated actively in the development of the Table and supporting documents.
Impact 1. (forthcoming) New OCEBM Table of Evidence (beta version available: http://www.cebm.net/index.aspx?o=5653) 2. (forthcoming) Publication of Table and Introductory Document (BMJ or EBM Journal). 3. (as a more direct part of my postdoctoral fellowship): publication of part of the background document as a manifesto for 'user friendly evidence'.
Start Year 2009
 
Description Placebo Working Group 
Organisation University of Southampton
Country United Kingdom 
Sector Academic/University 
PI Contribution 1. Survey of placebo use among UK general practitioners (Jeremy Howick as lead author, in press). 2. Systematic review of placebo effects compared with treatment effects across different conditions (Jeremy Howick as lead author, in press).
Collaborator Contribution 1. Survey of placebo use among UK general practitioners (Jeremy Howick as lead author, in press). 2. Systematic review of placebo effects compared with treatment effects across different conditions (Jeremy Howick as lead author, in press).
Impact 1. Survey of placebo use among UK general practitioners (Jeremy Howick as lead author, in press). 2. Systematic review of placebo effects compared with treatment effects across different conditions (Jeremy Howick as lead author, in press).
Start Year 2011
 
Description Blog on www.trusttheevidence.net 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact The OCEBM Website gets 2 million hits per year.

Open access, easily understood information about relevant topics.
Year(s) Of Engagement Activity 2011
 
Description CEBM Mailing List 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Over 50 people responded to a call for feedback on the beta version of the new OCEBM Table of Evidence. I (Dr. Jeremy Howick) monitored the list and took all feedback into account.

Doctors, patients and policy makers commented that the new Table would help them make better-informed care decisions.
Year(s) Of Engagement Activity 2009,2010
 
Description Public Patient Panel 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Participants in your research and patient groups
Results and Impact I have a team of patient/public representatives with whom I consult on a regular basis for my placebo research.

Improving research quality, making research relevant to real patients, involving members of the public.
Year(s) Of Engagement Activity 2011,2012
 
Description Public engagement talks 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact 10 November 2015. "Why it is ethical to prescribe placebos in Clinical Practice". Oxford Café Scientifique. Oxford, UK.

14 October 2015. "Why it is ethical to prescribe placebos in Clinical Practice". Bath Science Café. Bath, UK.
Year(s) Of Engagement Activity 2015
 
Description Publishing in Lay Press 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Several newsagents, including Reuters and the Wall Street Journal reported the results of a paper on which I was the senior author ('What's in a Placebo')

Our aim with the paper was to change policy regarding revealing 'placebo' ingredients in the reports of clinical trials. The response to the paper was positive, and we expect that the next incarnation of the CONSORT Statement will take our findings into account.
Year(s) Of Engagement Activity 2010,2011
 
Description Translation of OCEBM Levels of Evidence into French 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Type Of Presentation Workshop Facilitator
Geographic Reach International
Primary Audience Participants in your research and patient groups
Results and Impact Translation of the OCEBM Levels of Evidence, and associated documents (Introductory Document, Background Document) into French.

Reaching french audience, improving quality of evidence internationally.
Year(s) Of Engagement Activity 2011,2012