Development and practical application of landmarking in studies of time-varying exposures and survival

Lead Research Organisation: London Sch of Hygiene and Trop Medicine
Department Name: Epidemiology and Population Health

Abstract

Context

It is of great importance to understand the effects of the features of an individual on their survival - that is, for example, on mortality or disease diagnosis rates. These features, which we refer to collectively as 'exposures', may be lifestyle factors, treatments received, or clinical measurements such as blood pressure. It is often of interest to study exposures which are changing in an individual over time ('time-varying exposures') and how they relate to survival. By understanding these relationships we can gain insight into biological mechanisms. This helps towards the development of new treatments and can inform public health policy. Understanding how time-varying exposures impact on health and survival can also guide clinicians in their decision making, enabling more individualised treatment and prognosis for patients.

To study relationships between time-varying exposures and survival requires detailed data on large populations collected over long time periods. Data of this type can be obtained from patient data registries and electronic health records, which are for example obtained from records of participating GP practices, and banks of biological material collected from volunteers (biobanks).

Aims

To analyse the types of data mentioned above requires complex statistical methods and this is still a developing area of research. This fellowship would enable me to develop statistical methods which enable researchers to perform analyses which answer a range of important questions regarding the relationships between time-varying exposures and survival.

There are a number of different questions which we may wish to try to investigate, including:

- For a person with a particular diagnosis and knowing that person's time-varying exposure measurements up to the present time, what is their predicted probability of surviving up to at least 5 years from now?

- Which exposures are having a causal impact on survival and therefore would be the most important targets for treatment?

- What are the impacts on survival of different patterns over time in time-varying exposures?

Addressing these questions becomes especially complicated when there are lots of variables to be considered simultaneously which are all influencing one another over time. It is a big challenge to try to disentangle the associations of different time-varying exposures with survival in order to try to establish causal effects of the measurements we are primarily interested in. To address the above questions currently requires the use of quite different complex statistical techniques. This means that researchers are discouraged from using them and we do not get the best information out of the available data. My aim in this research is to develop statistical methods based on an approach called 'landmarking' to provide a way of addressing the types of questions given above in a similar way and in a way which is intuitive and accessible to a wide range of researchers, including clinicians.

Part of my work will be to incorporate into the statistical methods the capacity to handle the difficulties which come from using data in practice. I will focus on two particular challenges, which are the occurrence of missing measurements in some individuals at some time points, and the fact that many exposures are measured with some degree of error.

It is important to me to provide methods which are widely accessible. I will provide software code which can be used to implement my methods.

Applications

The methods developed will have important applications in electronic health records data and will therefore be able to have an impact on the many health conditions which can be studied using these databases. As part of this fellowship I will apply the statistical methods developed to study questions concerning the survival of people with cystic fibrosis using data from a large patient registry.

Technical Summary

It is vital to understand the effects of time-varying exposures on survival, for example to gain insight into biological mechanisms and to inform treatment decisions. Data such as electronic health records, which contain patient-level variables recorded longitudinally, enable us to study these effects.

My proposed research focuses on (1) making dynamic predictions of survival to a future time horizon based on an individual's measurements up to a given time; (2) estimating the causal effects on survival of patterns in time-varying exposures, including continuous and binary exposures and intermediate events, taking into account time-varying confounding.

These investigations can currently require quite different complex statistical methods, including joint modelling of exposure and survival data, multi-state modelling and causal inference methodology. A unified approach does not exist but is essential to enable applications from a wide range of researchers. I will develop statistical methods based on an approach called 'landmarking' to provide a unified and intuitive approach. A key advantage of landmarking is that it is based on the familiar method of Cox regression modelling. However, landmarking is currently limited to dynamic prediction and by lack of extensions to accommodate practical constraints.

In Part 1 I will devise how to use multiple imputation to handle missing data and measurement errors in longitudinal measurements used in the setting of dynamic prediction. In Part 2 I will develop landmarking to enable estimation of the causal effects of time-varying exposures on survival, allowing for time-varying confounding. Despite the importance of nested case-control and case-cohort studies, especially in biobanks, there have been few attempts at extensions to this setting. In Part 3, I will address this by extending the methods of Parts 1 and 2. I will place an emphasis on practical application and provide software code where needed.

Planned Impact

The statistical methods developed in this project will enable statisticians, epidemiologists and other researchers to answer important questions concerning relationships between time-varying exposures and survival using the landmarking approach. The methods will be especially aimed at and important for researchers analysing data from electronic health records, patient data registries and biobanks. Specifically, researchers will be able to develop dynamic prediction models and to estimate the causal effects of time-varying exposure on survival, including in the complex situation in which there exists time-varying confounding.

The methods developed will enable researchers to make the best use of the data available to them and to incorporate the practical difficulties of missing data and measurement error, which are inherent in observational data, in order to obtain unbiased estimates. The landmarking approach is quite intuitive and based on methods already familiar to any researchers. This will enable researchers to have an in-depth understanding of the methods they are using and to be aware of the assumptions being made. My provision of example code and new software where needed will be essential to encourage uptake of the methods and will result in better and increased use of electronic health records data and related data sources.

Through my extensions of landmarking to the case-control setting, researchers faced with nested case-control and case-cohort data, which will arise in particular from biobanks, will be able to develop dynamic prediction models and estimate causal effects as if full-cohort data were available. This work will be novel and will enable questions to be addressed using case-control data which have not previously been possible. The availability of methods which can be applied to case-control-type data may also impact on how biobank resources are used, resulting in savings in cost and enabling more studies to be performed using the available biological material.

Once the methods developed as part of this fellowship become known and used by other researchers, the methods will in the future be applied to study a range of health conditions. The impact will be improved knowledge of the causal effects of exposures on ill health and improved models for predicting survival for individuals. Consequently, doctors will be able to make better-informed decisions about treatment and offer more individualised and prognosis for their patients. Patients will ultimately benefit by receiving more appropriate treatment and by having a better understanding of their condition and their prognosis. Improved understanding of the effects of time-varying exposures on survival may also ultimately impact on health policy makers.

A direct impact of the work of this fellowship will be for the cystic fibrosis community, including clinicians and patients, by application of my methods to data from a large patient registry. The impact of this work will be a dynamic prediction model for survival based on clinical characteristics and improved knowledge about the causal effects of lung function, bacterial airway infections and lung transplantation on survival.

My overall goal is for the work proposed in this application to encourage and enable more and better use of data from electronic records and related data sources. As a result, I hope that this work will ultimately have a positive impact on how the use of electronic records data is viewed by the general public in the UK, which in turn will encourage people to give consent for their data to be used in the future and have a positive impact on future research.

My collaborations will be an important route through which the impacts will be realised during the project and beyond. The skills and connections I am able to develop as part of this fellowship will set me in good stead to ensure that the impacts continue beyond the fellowship and have maximum benefit.

Publications

10 25 50

publication icon
Brakenhoff TB (2018) Measurement error is often neglected in medical literature: a systematic review. in Journal of clinical epidemiology

publication icon
Cox DR (2018) Big data: Some statistical issues. in Statistics & probability letters

publication icon
Granchelli AM (2018) Microbial Interactions in the Cystic Fibrosis Airway. in Journal of clinical microbiology

publication icon
Keogh RH (2018) A guide to interpreting estimated median age of survival in cystic fibrosis patient registry reports. in Journal of cystic fibrosis : official journal of the European Cystic Fibrosis Society

 
Description Cystic Fibrosis Foundation Working Group on modelling Lung Function Decline
Geographic Reach North America 
Policy Influence Type Membership of a guideline committee
 
Description Epidemiological Methods for Implementation Research conference, Antwerp, Belgium: Short course
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact I was invited to contribute to a short course at WEON 2017: Epidemiological Methods for Implementation Research conference, Antwerp, Belgium. The course was on the topic of measurement error and I contributed the following sections: 1.Introduction to measurement error and why we should care about it. 2. Practical methods for measurement error correction. Attendees were primarily postgraduate students and young researchers in statistics and epidemiology.
 
Description Joint Conference on Biometrics and Biopharmaceutical Statistics (CEN ISBS), Vienna: Short course
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact I was invited to give a short course (With Dr Pamela Shaw, Penn State University) at the Joint Conference on Biometrics and Biopharmaceutical Statistics (CEN ISBS), Vienna, Austria. Course title: "Education for Statistics in Practice. Understanding and tackling measurement error: a whistle stop tour of modern practical methods.". Attendees were provided with an overview of the impact of error of measurement in different studies, and information on how to mitigate the effects of measurement error, including software implementation. We were invited to prepare a tutorial paper based on the course materials, which is in preparation.
 
Description North American Cystic Fibrosis Conference: Short course
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact I contributed as a teacher on a short course at the North American Cystic Fibrosis Conference, Indianapolis, US. Course title: Exercise and Physical Activity as Outcome Measures in Research in Cystic Fibrosis.Contribution title: Study design and making statistics work for you. Participants were made aware of issues relating to study design and analysis in studies of the impact of exercise on health in cystic fibrosis patients, and on measuring exercise capacity in this patient population. Attendees included researchers and healthcare practitioners.
 
Description Evidence-based valuation of patient-centred outcomes in Cystic Fibrosis
Amount £149,182 (GBP)
Funding ID PB-PG-1217-20018 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 10/2019 
End 04/2021
 
Description Strategic Research Centre Grant
Amount £747,259 (GBP)
Organisation Cystic Fibrosis Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 10/2015 
End 09/2019
 
Description Vertex Circle of Care Award
Amount $100,000 (USD)
Organisation Vertex Pharmaceuticals 
Sector Private
Country United States
Start 10/2018 
End 09/2019
 
Description Collaboration with Department of Biostatistics, Cincinnati Children's Hospital. 
Organisation Cincinnati Children's Hospital Medical Center
Country United States 
Sector Hospitals 
PI Contribution I am involved in a collaboration with Dr Rhonda Szczesniak, Associate Professor at the above Institution. We are working together on several research projects currently in preparation for publication. I have contributed statistical expertise and knowledge and experience of data sets to this collaboration.
Collaborator Contribution Dr Rhonda Szczesniak contributes statistical expertise and knowledge and experience of data sets to this collaboration.
Impact Several papers are in preparation. Dr Szczesniak and I have proposed an invited conference session.
Start Year 2016
 
Description Collaboration with Department of Public Health and Primary Care, University of Cambridge. 
Organisation University of Cambridge
Department Cambridge Stem Cell Institute
Country United Kingdom 
Sector Academic/University 
PI Contribution I visited Dr Angela Wood and her team at the University of Cambridge for 1 week in December 2016. The group is working on research related to mine. I participated in discussions with the group and contributed to a paper, undertook joint research to be published in the future, and developed new collaborative projects. We are also working to prepare software code in R and Stata for general implementation of landmarking for dynamic prediction. A paper fot the Journal of Statistical Software is in preparation, and code will be made available through Github. I prepared the R code for this joint project.
Collaborator Contribution This department hosted me for 1 week. They provided data used in a joint publication. Other members of the group are involved in testing of code and drafting of a manuscript (see above). Collaborators in this group contributed to a workshop that I organised at LSHTM in January 2019 on the topic of landmarking.
Impact A paper has been published based on contributions made while I was visiting the department (Paige et al, Am J Epi 2018). I also published a first author paper based on research conducted during the research visit and subsequent collaboration (Keogh et al, Biometrics 2018). As noted above, a softare paper and software are also in preparation.
Start Year 2016
 
Description ROBINS-I tool: a tool for assessing risk of bias in non-randomised studies of interventions. 
Organisation University of Bristol
Country United Kingdom 
Sector Academic/University 
PI Contribution I am a member of a working group to develop a tool to assess risk of bias in case-control studies, with a focus on evaluating the effects of health interventions. This is part of an MRC methodology research programme project and following on from the ROBINS-I tool (https://sites.google.com/site/riskofbiastool/). I am contributing my expertise to the development of the tool. This has involved participation in phone calls and workshops.
Collaborator Contribution Other group members are contributing my expertise to the development of the tool.
Impact A tool is in preparation and will be published later in 2017.
Start Year 2016
 
Description STRATOS Initiative member 
Organisation STRengthening Analytical Thinking for Observational Studies (STRATOS) Initiative
Country Global 
Sector Public 
PI Contribution The STRATOS Initiative is a large collaboration of experts in many different areas of biostatistical research. The objective of STRATOS is to provide accessible and accurate guidance in the design and analysis of observational studies. I am an expert member of the 'measurement error and misclassification' topic group. I participate in regular telephone meetings of the group. I have contributed as an author to three tutorial/review papers - one has been pblished (Shaw et al 2018), and two are under review. I have contributed to an international workshop and a conference symposium organised by the STRATOS Initiative. In 2018 I was appointed to the executive committee of STRATOS as a co-chair.
Collaborator Contribution The STRATOS Initiative organised a week-long workshop at the Banff International Research Station, Canada (July 2016) - I attended this free of charge. The Initative provides links with international experts on a range of topics in statistics and epidemiology.
Impact Workshop presentation at the STRATOS Initiative 1st group meeting (Banff, July 2016). Symposium presentation at the ISCB Conference, 2016. Invited session presentation at the Royal Statistical Society Conference, 2018.
Start Year 2015
 
Title Contribution to R package for handling missing data (smcfcs) 
Description I contributed two new functions to the 'smcfcs' package in R (by Jonathan Bartlett), which is a package enabling multiple imputation of missing values for use in regression analysis. More specifically, the package implements multiple imputation of missing covariates by Substantive Model Compatible Fully Conditional Specification. The two new functions that I have contributed (smcfcs.casecohort and smcfcs.nestedcc) extend the previous available methods to nested case-control studies and case-cohort studies. The software relates to work in a paper currently under review. The software is freely available at: https://github.com/jwb133/smcfcs 
Type Of Technology Software 
Year Produced 2017 
Impact The software was applied in simulation studies and to an example in the paper noted above. The software has not yet been used by others to my knowledge. 
URL https://github.com/jwb133/smcfcs
 
Title Software: R code for application of landmarking for dynamic prediction 
Description I have developed methodology and R code for implementation of landmarking for dynamic prediction, in particular to allow modelling of longitudinal predictors. This code accompanies a published paper: https://www.ncbi.nlm.nih.gov/pubmed/30234550 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact It is as yet unclear how widely the software has been used. Further work is underway with collaborators to prepare an article and more general code to publicise and increase the use of the software. 
URL https://github.com/ruthkeogh/landmark_CF
 
Title Software: R code for multiple imputation in Cox regression with time-varying effects 
Description I have developed methodology and R code for implementation of multiple imputation of missing covariate data in Cox regression when there are time-varying effects of exposures. the methodological work has been provisionally accepted for publication in Statistics in Medicine. The code is freely available at https://github.com/ruthkeogh/MI-TVE 
Type Of Technology Software 
Year Produced 2017 
Impact The software was applied in simulation studies and to an example in the paper noted above. The software has not yet been used by others to my knowledge. 
URL https://github.com/ruthkeogh/MI-TVE
 
Description American College of Epidemiology Annual Meeting. Invited session speaker: Dynamic Prediction Methods to Advance Precision Epidemiology. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This was a talk in an invited organised session at the American College of Epidemiology Annual Meeting. The talk title was "Landmarking for Dynamic Predictions of Outcome: Examples from Patient Registries". The audience of 50-100 was primarily made up of epidemiologists and clinicians.
Year(s) Of Engagement Activity 2018
URL https://acoe.memberclicks.net/2018-annual-meeting-concurrent3
 
Description Annual conference of the International Society of Clinical Biostatistics (ISCB), Vigo, Spain: Contributed talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Annual conference of the International Society of Clinical Biostatistics (ISCB), Vigo, Spain. I gave a talk on the topic of: "Flexible modelling of personalised dynamic prediction curves using landmarking, with a case-study in cystic fibrosis."
Year(s) Of Engagement Activity 2017
 
Description Blog aimed at people with cystic fibrosis and researchs in this area 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Patients, carers and/or patient groups
Results and Impact The aim of this personal blog is to provide resources for people with cystic fibrosis which enable them to engage with research. The blog was originally conceived as a route for presenting the results from an online questionnaire, as described elsewhere in the engagement activities section.
Year(s) Of Engagement Activity 2016,2017
URL http://blogs.lshtm.ac.uk/ruthkeogh/
 
Description Conference presentation (International Conference on Clinical Biostatistics, Utrecht) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This was a presentation at the International Conference on Clinical Biostatistics 2015, held in Utrecht, Netherlands. The title of the presentation was: Multiple imputation in Cox regression when there are time-varying effects of exposures. This talk generated a number of questions, plus discussions afterwards. It also resulted in me being invited to give a talk at another institution in 2016.
Year(s) Of Engagement Activity 2015
 
Description Conference workshop invited presentation: International Conference on Clinical Biostatistics, Birmingham, UK. STRATOS workshop. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact An invited workshop presentation of 25 minutes, entitled: "Measurement error in nutritional epidemiology: challenges, current practice, and the scope for improvement.". This took place in August 2016. Over 100 people attended the workshop, which was a half-day symposium of the STRATOS Initiative ("Strengthening Analytical Thinking for Observational Studies" Initiative). The presentation motivated audience discussion, which are informing a related tutorial paper. This presentation, and related activities, resulted in me being invited to give a half-day workshop on a related topic at two future events in 2017.
Year(s) Of Engagement Activity 2016
URL http://www.stratos-initiative.org/node/48
 
Description Contributed conference presentation: International Biometric Conference, Victoria, Canada. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A contributed workshop presentation of 20 minutes, entitled: "Dynamic prediction of survival using landmarking in large longitudinal observational patient databases: Challenges and solutions.". This took place in July 2016. Around 40 people attended this talk, and the audience were primarily statisticians, including postgraduate students. I received useful feedback on my research and audience members said they were keen to try out the methods I presented.
Year(s) Of Engagement Activity 2016
URL https://biometricconference.org/
 
Description Contributed conference presentation: International Conference on Clinical Biostatistics, Birmingham, UK. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A contributed workshop presentation of 15-20 minutes, entitled: "Dynamic prediction of survival using landmarking in large longitudinal observational patient databases: Challenges and solutions.". This took place in August 2016. Around 40 people attended this talk, and the audience were primarily statisticians, including postgraduate students. I received useful feedback on my research and audience members said they were keen to try out the methods I presented. This talk resulted in an invitation to give a seminar at the MRC Biostatistics Unit, Cambridge (see other engagement activity).
Year(s) Of Engagement Activity 2016
URL http://www.iscb.info/ISCB2016.html
 
Description Contributed conference presentation: Population-based Time-to-event Analysis International Conference 2016, LSHTM, UK. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A contributed workshop presentation of 20 minutes, entitled: "Dynamic prediction of survival using landmarking in large longitudinal observational
patient databases, with an application in Cystic Fibrosis.". This took place in August 2016. Over 100 people attended this talk, and the audience were primarily statisticians, including postgraduate students. I received useful feedback on my research and audience members said they were keen to try out the methods I presented. This resulted in a contact who requested advice on using my methods (which was given) which it is hoped will lead to a future collaboration.
Year(s) Of Engagement Activity 2016
URL http://csg.lshtm.ac.uk/pta2016/
 
Description Contributed conference talk. Royal Statistical Society Conference, Cardiff. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I gave a contributed conference talk entitled "Dynamic personalised prediction of survival using routinely collected data: An empirical comparison of landmarking and joint modelling.".
Year(s) Of Engagement Activity 2018
URL http://www.rss.org.uk/RSS/Events/RSS_Conference/2018_International_Conference/RSS/Events/Conference/...
 
Description Cystic Fibrosis Trust news item: explaining median survival 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Patients, carers and/or patient groups
Results and Impact I was invited to write an online news article explaining a recently published paper (reference below). The article was aimed at people with cystic fibrosis and the wider cystic fibrosis community and was prepared with advice from media professionals at the UK Cystic Fibrosis Trust. The article was published on the Cystic Fibrosis Trust website and publicised on social media. The Cystic Fibrosis Trust has a social media reach of over 17 million people and engagement of 600,000 people. The article attracted positive comments on the Cystic Fibrosis Trust facebook page.
Keogh RH, Stanojevic S. A guide to interpreting estimated median age of survival in cystic fibrosis patient registry reports. Journal of Cystic Fibrosis 2018. https://doi.org/10.1016/j.jcf.2017.11.014
https://www.cysticfibrosis.org.uk/news/what-is-median-survival-age
Year(s) Of Engagement Activity 2018
URL https://www.cysticfibrosis.org.uk/news/what-is-median-survival-age
 
Description Cystic Fibrosis Trust news item: what if I'm already 30? 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Patients, carers and/or patient groups
Results and Impact I was invited to write an online news article explaining a recently published paper (reference below). The article was aimed at people with cystic fibrosis and the wider cystic fibrosis community and was prepared with advice from media professionals at the UK Cystic Fibrosis Trust. The article was published on the Cystic Fibrosis Trust website and publicised on social media. The Cystic Fibrosis Trust has a social media reach of over 17 million people and engagement of 600,000 people. The article attracted positive comments on the Cystic Fibrosis Trust facebook page.
Keogh RH, Szczesniak R, Taylor-Robinson D, Bilton D. Up-to-date and projected estimates of survival for people with cystic fibrosis using baseline characteristics: A longitudinal study using UK patient registry data. Journal of Cystic Fibrosis 2018. https://doi.org/10.1016/j.jcf.2017.11.019.
https://www.cysticfibrosis.org.uk/news/survival-statistics-what-if-im-already-30#
Year(s) Of Engagement Activity 2018
URL https://www.cysticfibrosis.org.uk/news/survival-statistics-what-if-im-already-30#
 
Description Cystic Fibrosis Trust website: Q&A on "Challenging topics: statistics and life expectancy" 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Patients, carers and/or patient groups
Results and Impact In my research I am using data from the UK Cystic Fibrosis Patient Registry. The Cystic Fibrosis Trust used a webpage article in a "Question and Answer" style interview with me to explain some of my research and its intended impacts. The article was aimed at people with cystic fibrosis, their families, and their care teams. The webpage was also used to promote an online patient questionnaire (please see other engagement activity for details). The web article was advertised by the Cystic Fibrosis Trust via social media. There were 832 clicks on the link from Facebook, and 292 'likes'.
These engagement activities with the Cystic Fibrosis Trust have resulted in plans for future engagement activities.
Year(s) Of Engagement Activity Pre-2006,2006,2016
URL https://www.cysticfibrosis.org.uk/news/life-expectancy
 
Description Departmental seminar presentation (UCL) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact This was a presentation to the Statistics Departmental seminar series at UCL given to 15-20 researchers, including postgraduate students. The title of the presentation was: Multiple imputation in Cox regression when there are time-varying effects of exposures.
Year(s) Of Engagement Activity 2015
 
Description Departmental seminar series (Bristol University) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact This was a presentation to the Mathematics Department statistics seminar series at Bristol University given to 30-40 researchers, including postgraduate students. The title of the presentation was: Multiple imputation in Cox regression when there are time-varying effects of exposures.
Year(s) Of Engagement Activity 2015
 
Description Departmental seminar series (Leiden University) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This was a presentation to the Statistics Departmental seminar series at Leiden University given to 15-20 researchers, including postgraduate students. The title of the presentation was: Multiple imputation in Cox regression when there are time-varying effects of exposures.
Year(s) Of Engagement Activity 2015
 
Description Discussion group (Cambridge) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact This was a presentation on work-in-progress to 15-20 researchers interested in risk prediction. The researchers were from several institutions in Cambridge and London. I received useful feedback and comments on my proposed methods.
Year(s) Of Engagement Activity 2015
 
Description Epidemiological Methods for Implementation Research conference: Invited short course 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact WEON 2017: Epidemiological Methods for Implementation Research conference, Antwerp, Belgium. Invited pre-conference course. Topics:
1.Introduction to measurement error and why we should care about it.
2. Practical methods for measurement error correction.
Year(s) Of Engagement Activity 2017
 
Description Essex University: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Essex University, Statistics Department seminar series. I was invited to give a talk on the topic: "Personalised dynamic prediction of survival using patient registry data: An anatomy of a landmarking analysis."
Year(s) Of Engagement Activity 2018
 
Description European Cystic Fibrosis Conference, Seville, Spain: Contributed talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact European Cystic Fibrosis Conference, Seville, Spain. I gave a talk on the topic of:
"How can we provide better information on survival for people with CF, accounting for current age, sex and genotype?".
Year(s) Of Engagement Activity 2017
 
Description European Cystic Fibrosis Conference: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact European Cystic Fibrosis Conference 2017, Seville, Spain. Invited symposium contribution.
Topic: "Survival: often mentioned, often misunderstood. Definitions and expectations."
Year(s) Of Engagement Activity 2017
 
Description Imperial College London: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Imperial College London, Statistics Department seminar series. I was invited to give a talk on the topic: "Personalised dynamic prediction of survival using patient registry data: An anatomy of a landmarking analysis."
Year(s) Of Engagement Activity 2017
 
Description International Society of Behavioral Nutrition and Physical Activity (ISBNPA) Annual Meeting: Invited talk 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact International Society of Behavioral Nutrition and Physical Activity (ISBNPA) Annual Meeting, Victoria, Canada. Invited workshop contribution.I was invited to give a talk on the topic: "Integrating biomarkers: Implications for designing nutrition intervention trials."
Year(s) Of Engagement Activity 2017
 
Description Invited conference presentation: International Biometric Conference, Biometrics Showcase. Victoria, Canada. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Handling missing data in matched case-control studies using multiple imputation.

This was an invited presentation of 30 minutes, entitled: "Handling missing data in matched case-control studies using multiple imputation.". It took place in July 2016. This was a talk in the Biometrics Showcase, to which I was invited due to a prize for the best paper in Biometics 2015. Around 30 people attended this talk, and the audience were primarily statisticians, including postgraduate students. This talk increased awareness of the methods described in the related paper.
Year(s) Of Engagement Activity 2016
 
Description Invited conference presentation: North American Cystic Fibrosis Conference. Orlando, US. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation of 25 minutes, entitled: "Does accounting for FEV1% variability enable better prediction of survival in CF?.". This took place in October 2017. Approximately 100 people attended the seminar, including medical practitioners, other health care providers, epidemiologists, statisticians. I received useful feedback on my research. The audience included medical practitioners, who expressed interest in putting some of the methods I presented into clinical practice to provide information to patients and their care teams. This presentation, along with other related activities, resulted in me being invited to give a presentation at the European Cystic Fibrosis Conference in June 2017 and being invited to participate in a collaboration between the UK and US Cystic Fibrosis patient Registries.
Year(s) Of Engagement Activity 2016
URL https://www.nacfconference.org/
 
Description Invited seminar: Bristol University, MRC Integrative Epidemiology Unit 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Seminar of 45 minutes, entitled: "Dynamic prediction of survival using landmarking in large healthcare databases, with an application in cystic fibrosis.". This took place on 16th February 2017.

Approximately 30 people attended the seminar, including statisticians, epidemiologists and postgraduate students. I received useful feedback on my research and audience members said they were keen to try out the methods I presented.
Year(s) Of Engagement Activity 2017
 
Description Invited seminar: Gustave Roussy Institute, Department of Biostatistics and Epidemiology Seminar Series, Paris. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A conference presentation of 45 minutes, entitled: "Dynamic prediction of survival using landmarking in large longitudinal observational patient databases: With an application in Cystic Fibrosis.". This took place in May 2016. Around 20-30 people attended this talk, and the audience were primarily statisticians, including postgraduate students. I received useful feedback on my research and audience members said they were keen to try out the methods I presented.
Year(s) Of Engagement Activity 2016
 
Description Invited seminar: MRC Biostatstics Unit, Cambridge 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Seminar of 45 minutes, entitled: "Dynamic prediction of survival using landmarking in large healthcare databases, with an application in cystic fibrosis.". This took place on 19th January 2017. Approximately 60 people attended the seminar, including statisticians, epidemiologists and postgraduate students. I received useful feedback on my research and audience members said they were keen to try out the methods I presented.
Year(s) Of Engagement Activity 2017
 
Description Invited seminar: UCL Biostatistics Network Seminar Series 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Seminar of 45 minutes, entitled: "Dynamic prediction of survival using landmarking in large healthcare databases, with an application in cystic fibrosis.". This took place on 31st January 2017. Approximately 20 people attended the seminar, including statisticians, epidemiologists and postgraduate students. I received useful feedback on my research and audience members said they were keen to try out the methods I presented.
Year(s) Of Engagement Activity 2017
 
Description Invited talk. Conference of the German Region of the International Biometric Society, Frankfurt. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I gave an invited talk on the topic of "Estimating causal effects of time-varying exposures using observational data: What can we do using standard regression methods?", for a session on "Statistical Methods in Epidemiology".
Year(s) Of Engagement Activity 2018
URL http://www.biometrisches-kolloquium2018.de/index.php/en/
 
Description Invited talk. Essex University. Statistics seminar series. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact I gave a talk entitled "Personalised dynamic prediction of survival using patient registry data: An anatomy of a landmarking analysis.". The audience included researchers, and undergraduate and post-graduate students. The talk was followed by questions, discussion an requests for further information.
Year(s) Of Engagement Activity 2018
URL https://www.essex.ac.uk/events/2018/02/01/personalised-dynamic-prediction-of-survival-using-patient-...
 
Description Invited talk. Keele University. Biostatistics Group Seminar. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact I gave a talk entitled "Estimating causal effects of time-varying exposures using observational data: What can we do using standard regression methods?" to an audience of around 20 people, including researchers and post-graduate students. The talk was followed by questions and discussion.
Year(s) Of Engagement Activity 2019
URL https://www.keele.ac.uk/pchs/research/biostatistics/methodologyseminarseriesmethss/
 
Description Invited talk. University of Oslo, Biostatistics Seminar Series. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact 20-30 researchers, including biostatisticians, epidemiologists and clinicians, attended this talk entitled "Exploring methods for estimating effects of treatment on survival using longitudinal observational data: Application to the UK Cystic Fibrosis Registry".

Outcomes have been a new collaboration with those who invited me to give the seminar, an invitation to be a external thesis examiner, and an invitation to give a keynote talk at a conference in 2019.
Year(s) Of Engagement Activity 2018
URL https://www.med.uio.no/imb/english/research/centres/ocbe/events/biostat-seminar/
 
Description Invited talk: University of Leicester, Biostatistics Seminar series. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Policymakers/politicians
Results and Impact I gave a talk enetitled: "How to impute missing data in Cox regression: New developments incorporating non-proportional hazards.", and showcased some software that I have developed for implementation of the methods described.
Year(s) Of Engagement Activity 2017
 
Description Invited workshop presentation: STRATOS Initiative workshop, Banff, Canada. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Members of the STRATOS Initiative ("Strengthening Analytical Thinking in Observation Studies") attended a week-long workshop at the Banff International Research Station, Canada, in July 2016. I was invited to give a presentation entitled "Error in measurements of dietary intake used in nutritional epidemiology: Impact, corrections, and recommendations.". The talk motivated much discussion among the audience, which has informed two tutorial papers, which are currently works in progress. The audience comprised international experts in various areas of statistical and epidemiological research. This talk, and related activities, resulted in me being invited to talk at the ISCB STRSATOS symposium (see other engagement activity), and two further events in 2017.
Year(s) Of Engagement Activity 2016
URL http://www.stratos-initiative.org/node/49
 
Description Joint Conference on Biometrics and Biopharmaceutical Statistics (CEN ISBS), Vienna, Austria: Contributed talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Joint Conference on Biometrics and Biopharmaceutical Statistics (CEN ISBS), Vienna, Austria. I gave a talk on the topic of: "Flexible modelling of personalised dynamic prediction curves using landmarking, with a case-study in cystic fibrosis.". As a result of this talk I was invite to give a seminar at the University of Ulm.
Year(s) Of Engagement Activity 2017
 
Description Joint Conference on Biometrics and Biopharmaceutical Statistics (CEN ISBS), Vienna: Short course 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Joint Conference on Biometrics \& Biopharmaceutical Statistics (CEN ISBS), Vienna, Austria. I was invited to contribute a short course to this conference. Topic: "Education for Statistics in Practice. Understanding and tackling measurement error: a whistle stop tour of modern practical methods.". With Dr Pamela Shaw, Penn State University.
Year(s) Of Engagement Activity 2017
 
Description Keynote talk. UK Cystic Fibrosis Medical Association Meeting, Birmingham 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact I gave a keynote talk for this inaugural meeting of the UK Cystic Fibrosis Medical Association, entitled "Survival and patient numbers: prediction, projection and communication.". This was an opportunity to engage with clinicians about my research in the area of cystic fibrosis. Audience members reported that they discovered new information from the talk. There was substantial discussion and questions following the talk.
Year(s) Of Engagement Activity 2019
 
Description Landmarking workshop and symposium 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact I organised a workshop of 9 international researchers on the topic of "Landmarking for dynamic prediction". This was followed by a half-day symposium hosted by the Centre for Statistical methodology at the London School of Hygiene and Tropical Medicine: "Landmarking for survival analysis: theory, application and future directions". The symposium had 5 speakers, including myself, and was attended by 60-80 researchers from across the UK.
Year(s) Of Engagement Activity 2019
URL https://csm.lshtm.ac.uk/mini-symposium-on-landmarking-for-survival-analysis/
 
Description MRC Clinical Trials Unit at UCL: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact MRC Clinical Trials Unit at UCL seminar series. I was invited to give a talk on the topic: "Dynamic prediction of survival using landmarking: Introduction, recent developments, and an application in cystic fibrosis."
Year(s) Of Engagement Activity 2017
 
Description North American Cystic Fibrosis Conference: Short course contribution 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact North American Cystic Fibrosis Conference, Indianapolis, US. Short course contribution. Course title: Exercise and Physical Activity as Outcome Measures in Research in Cystic Fibrosis. Title of presentation: Study design and making statistics work for you.
Year(s) Of Engagement Activity 2017
 
Description Online questionnaire: "What would people with cystic fibrosis aged 16+ like to learn about their life expectancy and other outcomes?" 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact An online questionnaire was used to collect information from people with CF aged 16 and above on attitude to information on life expectancy. The aim was to ascertain the current sources of information on life expectancy, attitudes to receiving such information, and how it might be received in the future. The questionnaire was developed in consultation with two patient advisors and was available online for two weeks (4-18 July 2016). It was advertised by the Cystic Fibrosis Trust using Facebook and Twitter using a link to an article on the Cystic Fibrosis Trust website [https://www.cysticfibrosis.org.uk/news/life-expectancy#]. There were 85 respondents. The results were fed back to participants via an information sheet available at the following link:
http://blogs.lshtm.ac.uk/ruthkeogh/files/2016/06/CFQ-infosheet.pdf
Year(s) Of Engagement Activity 2016
URL http://blogs.lshtm.ac.uk/ruthkeogh/files/2016/06/CFQ-infosheet.pdf
 
Description Organised invited session. Conference of the International Biometric Society, Barcelona. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I proposed, organised and chaired an invited session at the International Biometric Society, Barcelona, on the topic of "Dynamic individualised risk prediction: Recent developments and real world applications". There were 3 internationally renowned speakers and a discussant. This event reached an audience of estimated 80-100 and was voted as one of the most popular sessions of the conference. This event resulted in a new collaboration and a subsequent workshop.
Year(s) Of Engagement Activity 2018
URL http://2018.biometricconference.org/invited-sessions/
 
Description Oslo University Statistics Department Seminar Series: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I gave an invited talk at the Statistics Department at the University of Oslo. Title: Multiple imputation in Cox regression when there are time-varying effects of exposures.. The audience of around 20 included researchers and postgraduate students.
Year(s) Of Engagement Activity 2017
 
Description Oxford Causal Inference seminar April 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact University of Oxford Causal Inference in Epidemiology Seminar (Nuffield Department of Population Health). Talk title: "Creating a 'target trial' within a longitudinal cohort: application to estimate treatment effects in cystic fibrosis". The seminar sparked questions and discussion and requests for further information about how to use the techniques I described in different application areas.
Year(s) Of Engagement Activity 2019
URL https://www.ndph.ox.ac.uk/events/causal-inference-in-epidemiology-seminar-creating-a-target-trial-wi...
 
Description Poster presentation/interactive poster talk: Conference of the European Cystic Fibrosis Society, Belgrade. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Patients, carers and/or patient groups
Results and Impact I presented a physical poster at the Conference of the European Cystic Fibrosis Society, Belgrade, reaching a large audience of clinicians, researchers nd pharmaceutical industry represetatives. The poster was also presented orally in a quick-fire poster session.

I was awarded the Best Poster Prize.
Year(s) Of Engagement Activity 2018
URL https://www.ecfs.eu/conferences/awards
 
Description Royal Statistical Society Conference. Invited session speaker. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I gave a talk in an invited session, entitled "How to impute
missing data in Cox regression: New developments incorporating non-proportional hazards.". This presented work and made the audience aware of software available for implementation of the methods. The audience was of 20-30 statisticians and researchers in related fields.
Year(s) Of Engagement Activity 2018
URL http://www.rss.org.uk/RSS/Events/RSS_Conference/2018_International_Conference/RSS/Events/Conference/...
 
Description Royal Statistical Society, Oxford local group: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Royal Statistical Society, Oxford local group seminar series.
I was invited to give a talk on the topic: "Dynamic prediction of survival using landmarking in large healthcare databases, with an application in cystic fibrosis."
Year(s) Of Engagement Activity 2017
 
Description Statistical Analysis of Multi-Outcome Data (SAM) 2017 conference, Liverpool: Contributed talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Statistical Analysis of Multi-Outcome Data (SAM) 2017 conference, Liverpool. I gave a talk on the topic of:
"Flexible modelling of personalised dynamic prediction curves using landmarking, with a case-study in cystic fibrosis.". I was awarded the prize for best talk. As a result of this conference I was invited to speak at several national seminar series.
Year(s) Of Engagement Activity 2017
 
Description UK Cystic Fibrosis Conference. Keynote speaker. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact I gave a keynote talk entitled "Survival, treatments and transplant: Using the Registry today to inform CF care tomorrow". The UK Cystic Fibrosis Conference brings together researchers, clinicians, and friends and family of people with cystic fibrosis, to explore all aspects of clinical care and research in cystic fibrosis. There was an in-person audience over over 100 and the talk was broadcast online for patients and the cystic fibrosis community to watch, and was also available for subsequent download. This has led to subsequent invitations to present my research, as well as an invitation to join the scientific committee on the European CF Conference.
Year(s) Of Engagement Activity 2018
URL https://www.cysticfibrosis.org.uk/the-work-we-do/conference-and-meetings/ukcfc/ukcfc-2018-video-gall...
 
Description University College London: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact International Centre for Lifecourse Studies in Society and Health, UCL, seminar series. I was invited to give a talk on the topic: "Personalised dynamic prediction of survival using patient registry data: An anatomy of a landmarking analysis."
Year(s) Of Engagement Activity 2017
 
Description University of Bath: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact University of Bath, mathematical Sciences Department seminar series. I was invited to give a talk on the topic: "Personalised dynamic prediction of survival using patient registry data: An anatomy of a landmarking analysis."
Year(s) Of Engagement Activity 2017
 
Description University of Leicester: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact University of Leicester, Biostatistics Department seminar series. I was invited to give a talk on the topic: "How to impute missing data in Cox regression: New developments incorporating non-proportional hazards."
Year(s) Of Engagement Activity 2017
 
Description University of Ulm, Germany. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I gave an invited talk entitled "Personalised dynamic prediction of survival using patient registry data: An anatomy of a landmarking analysis" for a Mathematical Colloquium. The audience were researchers, postgraduate students, and people working in the pharmaceutical industry. I had discussions throughout the day with the researchers, who gave short talks to stimulate discussion.
Year(s) Of Engagement Activity 2018
URL https://www.uni-ulm.de/mawi/statistics/aktuelle-meldung/article/19012018-mathematical-colloquium-dr-...
 
Description University of Ulm: Invited talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I gave an invited talk at the Statistics Department at the University of Ulm. Title: Personalised dynamic prediction of survival using patient registry data: An anatomy of a landmarking analysis. The audience of around 20 included researchers, postgraduate students and statisticians from industry.
Year(s) Of Engagement Activity 2018
 
Description Workshop on Age-Period-Cohort analysis: Contributed talk 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I attended the Workshop on Age-Period-Cohort analysis at Nuffield College, Oxford (where I am an Associate Member). I gave a talk on the topic: "Projecting future survivor curves using flexible parametric survival models: an application in cystic fibrosis using patient registry data.". This event enabled me to present work to economists, demographers and social scientists, as well as statisticians/epidemiologists, who are my more usual audience.
Year(s) Of Engagement Activity 2017