Quantitative methods for the assessment of systematic error in observational studies: improving causal research

Lead Research Organisation: London Sch of Hygiene and Trop Medicine
Department Name: Epidemiology and Population Health

Abstract

Results from epidemiological studies often appear contradictory, leading to some cynicism regarding medical research among the general public. Many of these contradictions might be avoided if the true extent of the uncertainties that affect such research was assessed and explained. Presently, uncertainty arising from studying finite samples is typically the only uncertainty presented (as confidence intervals). However, other sources of uncertainty may dwarf this uncertainty: (i) important unmeasured factors in the study population (unmeasured confounding); (ii) non-random inclusion of subjects (selection bias); (iii) errors in measurements of exposure and outcome (measurement error) (iv) absence of outcome/exposure data for some subjects (missing data). Recently new approaches, which take account of these sources of uncertainty, have been proposed, but face both operational (e.g. how to specify the likely magnitude of selection bias) and methodological (e.g. how to combine this information with the observed data) difficulties. We shall investigate three alternative approaches to improved quantification of uncertainty; classical sensitivity analysis; Monte Carlo sensitivity analysis and Bayesian bias analysis. We will explore the feasibility of these approaches in different contexts and provide practical guidelines to practitioners (including user-friendly software) on methods which are robust, transparent and accessible to a wide range of practitioners.

Technical Summary

Results from epidemiological studies often appear contradictory, leading to some cynicism regarding medical research among the general public. Many of these contradictions are attributable to the inadequate reporting of the uncertainties that affect this type of research.

Uncertainty arising from random sampling variation is typically the only uncertainty presented (in the form of confidence intervals). There are, however, other sources of uncertainty which may dwarf the uncertainty due to sampling variation. The extent of such uncertainties will vary from study to study but in general they arise from: (i) unmeasured confounders; (ii) selection bias; (iii) measurement error (iv) missing data.

Recently attempts have been made to incorporate multiple sources of bias within a single modelling framework. However, considerable work remains to be done as presently these approaches are complex to implement and are not immediately generalizable to continuous exposures and outcomes. The objectives of this proposal are as follows:

(1) To develop an inventory and classification of biases and existing bias-adjustment methods to quantify the impact of potential systematic errors.
(2) Identify data sources which provide information on the potential magnitude of sources of bias whose adjustment requires external information.
(3) Establish appropriate conceptual, methodological and computational bases for multi-bias adjustment and sensitivity analyses.
(4) Compare the performance and practicability of different combinations of methods.
(5) On the basis of the above, make recommendations concerning bias adjustment approaches which are statistically robust, transparent and accessible. community.

The work will be performed in a series of overlapping steps as follows:

(1) a literature search for relevant methods and data;
(2) a methodological exploration of existing work to establish the core principles behind, and links among, existing approaches;
(3) a computational assessment of the alternatives;
(4) further methodological development if necessary;
(5) comparison of the performance and practicability of different combinations of methods
(6) development of guidelines and recommendations

We expect our research to be of great interest to the epidemiological research community. We shall disseminate our findings through peer-reviewed journal articles, conference presentations and through teaching at LSHTM and elsewhere. Any software routines developed will be placed in the public domain. We do not expect our research to result in commercially exploitable outputs.

Publications

10 25 50
 
Description A short course on concepts and methods in causal inference, Cambridge, September 2013
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Keynote lecture: Introduction to Causal Inference. Swedish Association of Local Authorities and Regions, Stockholm, May 2013
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Preconference course: Methods for dealing with time-varying confounding. World Congress of Epidemiology. Edinburgh, August 2011
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: A method for improving the robustness of multiple imputation. Royal Statistical Society. 30 March 2010
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: An example of mediation analysis, London, 2013
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Causal Directed Acyclic Graphs for Missing Data Problems. World Congress of Probability and Statistics, Istanbul, July 9, 2012.
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Causal diagrams: shedding light on missing data problems. Statistical Methods and Medical Research: New Challenges for an Old Marriage. Celebrating the Ruby Anniversary of theMSc in Medical Statistics. LSHTM, April 2009.
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Causal inference from randomised trials with survival outcomes: dealing with treatment switching. Oxford Outcomes, January 11, 2011
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Causal inference with g-computation. Bristol University, November 2011
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Causal inference with g-computation. UCL, July 2013
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Confounded about Confounding? How modern causal thinking brings fresh clarity to old concerns. LSHTM September 23 2011
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Current and new approaches to estimating causal pathways from observational data. ESRC Research Methods Festival, Oxford, July 4, 2012.
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Definitions and estimation of direct causal effects: on recursive regressions and the g-computation formula. Sweden, 18 May 2011
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Formalising the use of causal diagrams in missing data problems. International Biometrics Conference, Florianopolis, Brazil, December 2010
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Intermediate confounding, measurement error and missing data: a tentative path through the epidemiologist's reality. Turin, November 2010
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Maternal characteristics, childhood growth and eating disorder: a study of mediation using gformula.Italian Stata Users Group meeting, Bologna, September 20, 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Maternal characteristics, childhood growth and eating disorder: a study of mediation. LSE Seminar series, 30 November 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Mediation analysis in life course studies. Statistical Challenges in Lifecourse Research Conference, Leeds, 17-18 July 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Mediation analysis with SEM or causal inference: Where is the difference? 3rd UK Mplus Conference, LSHTM, 24th May 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Mediation analysis with multiple causally-ordered mediators. Centre for Biostatistics Manchester University, the RSS Manchester local group and the RSS Primary Health Care study group, Manchester University, October 17, 2012.
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Mediation by path analysis or causal inference: What is the difference? UK Causal Inference Meeting, May 2013, Manchester
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Mediation in life course epidemiology. INSERM Worskshop on life course epidemiology, Bordeaux 23 March 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Methods for dealing with time-varying confounding. EUROEPI conference, Florence, Italy, November 2010
Geographic Reach Asia 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Novel approaches to estimating causal pathways from observational data. Manchester, December 2012
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Social disadvantage and infant mortality. ESRC Research Methods Festival, Oxford, 4th July 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: The use of the stochastic g-computation formula in causal inference. National Cancer Institute at the National Institutes of Health, Bethesda MD, April 24, 2012.
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: Using causal diagrams to guide analysis in missing data problems. Causal Inference Research Group, University of North Carolina Chapel Hill, March 29, 2012.
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: What is the difference between association and causation? (And why should we bother being formal about it?). ESRC Research Methods Festival, Oxford, July 5, 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Presentation: gformula: Estimating causal effects in the presence of time-varying confounding or mediation. Italian Stata Users Group meeting, Bologna, September 20, 2012.
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short Course in Causal Inference in Epidemiology, run annually at LSHTM since 2008
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short course: A short course on concepts and methods in causal inference. Centro San Benedetto Menni - Fatebenefratelli, Rome (Italy), 17-19 September 2012
Geographic Reach Asia 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short course: A short course on concepts and methods in causal inference. Swansea University, 11-12 October 2012.
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short course: A short course on concepts and methods in causal inference. Uppsala University (Sweden), 8-9 December 2011.
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short course: Missing Data - when is it likely to cause a problem? Society for Social Medicine Conference Workshop, London, 13 September 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short course: One day short course on causal inference. Modena, Italy 2009
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Short course: Studying pathways between social and biological factors: can modern methods in causal inference help? Society for Social Medicine Pre-conference Meeting, London, 11 September 2012
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description ESRC NCRM Phase 3 Methodology Node
Amount £742,141 (GBP)
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 10/2011 
End 09/2014
 
Description MRC Career Development Award in Biostatistics
Amount £301,916 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 09/2011 
End 09/2014
 
Description Travel bursary to attend the International Biometrics Conference/International Biometrics Society
Amount £500 (GBP)
Organisation International Biometrics Society 
Sector Academic/University
Country Global
Start 11/2010 
End 12/2010
 
Title Computer routine for performing G-computation 
Description A computer routine written in Stata to perform G-computation for the analysis of time varying confounding has been written and has been made freely available at http://econpapers.repec.org/software/bocbocode/S457204.htm 
Type Of Technology Software 
Year Produced 2010 
Open Source License? Yes  
Impact Unknown 
URL http://econpapers.repec.org/software/bocbocode/S457204.htm