Effective and efficient conduct of trials and meta-analyses

Lead Research Organisation: University College London

Abstract

Our work in this area involves investigating and improving trial data checking and monitoring, as well as reducing bias and improving the validity of our systematic reviews. Some of the important outcomes are:
Evidence-based trial monitoring
• Many trials organisations have implemented trial management and monitoring procedures out of proportion to the risks to trial participants or the public, thereby increasing the workload, complexity and cost of lower-risk trials. We initiated the TEMPER study to evaluate prospectively whether triggered monitoring is effective at distinguishing problem sites in ongoing CTU multicentre cancer trials.
Increasing participant retention in trials
• Our systematic review of strategies to increase participant retention specifically in RCTs has shown that monetary incentives and offers of monetary incentives increase postal and electronic questionnaire response. Non-monetary incentives, additional communication and reminder strategies and most modified questionnaires have little impact.
Informing systematic review conduct
• CTU is leading a Network-funded project to develop a ‘state of the art’ series of papers on Individual participant data (IPD) meta-analyses and recently hosted a workshop of international experts to discuss topics. Most systematic reviews now assess risk of bias of eligible trials based on their publications (often using the Cochrane Collaboration tool), and it has been proposed that these are used as a basis for including or weighting trials in a meta-analysis. An evaluation of publication-based assessments showed that they can be unreliable, compared to those based on information obtained from investigators or protocols (as part of our IPD meta-analyses). In particular, reviewers should be cautious about using these as a basis for trial inclusion, and instead seek additional information from trialists, where possible.

Technical Summary

There is an urgent need for new methods for many clinical studies. One challenge is finding cost-effective, evidence-based methods for running trials. This increases the likelihood of producing reliable results in a timely manner, which in turn frees resources to allow more trials to take place. We will do this by developing and evaluating methods for ensuring that trials are run effectively and efficiently. We are providing practical examples of how novel designs can be implemented; evaluating and implementing strategies to ensure that data quantity and quality are maximised and that staff time locally and at sites is best used. We are also investigating how to get trials started and reported more quickly, and to make appropriate use of e-health resources.

Gains from efficient conduct may be dissipated or lost completely if evidence synthesis of trials is protracted, inefficient or poorly conducted. Furthermore, if understanding or use of systematic reviews is limited, then clinical practice and research may not be informed by the best evidence. We will address these issues by developing a new framework for meta-analysis to speed up the evaluation of individual therapies; providing tools and guidance to promote greater awareness, understanding and use of IPD meta-analysis across a variety of stakeholders and resolving outstanding issues in systematic review conduct.

Organisations

Publications

10 25 50

publication icon
Choodari-Oskooei Babak (2016) Quantifying the uptake of user-written commands over time in STATA JOURNAL

 
Description Data sharing guidance cited in CRUK guidance
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL http://www.cancerresearchuk.org/sites/default/files/hands_on_data_sharing_advice_-_clinical.pdf
 
Description Guidance on sharing of data from publicly funded trials
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL http://www.ct-toolkit.ac.uk/routemap/
 
Description Patient Information Sheet template and guidance referenced in HRA guidance
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL http://www.hra.nhs.uk/documents/2017/01/applying-proportionate-approach-process-seeking-consent.pdf
 
Description Patient Information Sheet template referenced in HRA guidance
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://www.hra.nhs.uk/documents/283/applying-proportionate-approach-process-seeking-consent_R3gbJKn...
 
Description CR UK Population Science Committee, Project Grant (TEMPER study)
Amount £205,540 (GBP)
Funding ID C1495/A13305 
Organisation Cancer Research UK 
Sector Charity/Non Profit
Country United Kingdom
Start 05/2012 
End 05/2016
 
Description Health Data Research (UK) funding
Amount £202,717 (GBP)
Organisation Health Data Research UK 
Start 10/2018 
End 10/2021
 
Description MRC Hub for Trials Methodology Research (HTMR) - extension
Amount £500,000 (GBP)
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 04/2014 
End 04/2015
 
Description MRC Methodology Research Panel: Missing data in propensity score analyses of Electronic Health Records Data
Amount £400,000 (GBP)
Funding ID MR/M013278/1 
Organisation Medical Research Council (MRC) 
Sector Academic/University
Country United Kingdom
Start 09/2016 
End 08/2019
 
Title Induction Pack for PPI (Public Patient Involvement) Contributorson Trial Oversight Committees 
Description A guidance pack to help researchers design relevant PPI for their trial 
Type Of Material Improvements to research infrastructure 
Year Produced 2018 
Provided To Others? Yes  
Impact None yet 
URL https://www.ctu.mrc.ac.uk/media/1416/induction_pack_for_ppi_contributors_on_trial_oversight_committe...
 
Title PIS template 
Description New Patient Information Sheet template and associated guidance, developed for use at the CTU but shared externally with over 60 other researchers (so far). 
Type Of Material Improvements to research infrastructure 
Year Produced 2015 
Provided To Others? Yes  
Impact HRA is due to include a reference to this tool in their updated PIS guidance. 
URL http://www.ctu.mrc.ac.uk/resources/patient_involvement/
 
Title ORRCA - Online Resource for Recruitment research in Clinicals triAls 
Description Online database of recruitment research, with search functions 
Type Of Material Database/Collection of data 
Year Produced 2016 
Provided To Others? Yes  
Impact Not known 
URL http://www.orrca.org.uk/
 
Description Children and Young People in trials - online learning resource 
Organisation Queen's University Belfast
Country United Kingdom 
Sector Academic/University 
PI Contribution Contributions made to planning of this project, coordinated by researchers at Queens University, Belfast.
Collaborator Contribution Coordination and overall responsibility for the project.
Impact None so far
Start Year 2015
 
Description Cochrane IPD Synthesis Working Group 
Organisation Brown University
Country United States 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation Keele University
Country United Kingdom 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation Queen's University Belfast
Country United Kingdom 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation Radboud University Nijmegen
Country Netherlands 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation University Medical Center Utrecht (UMC)
Country Netherlands 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation University of Bristol
Country United Kingdom 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation University of Hasselt
Department International Drug Development Institute
Country Belgium 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation University of Sydney
Country Australia 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description Cochrane IPD Synthesis Working Group 
Organisation University of York
Department Centre for Reviews and Dissemination (CRD)
Country United Kingdom 
Sector Academic/University 
PI Contribution Participated in teleconference discussions and preparation of the final report
Collaborator Contribution Participated in teleconference discussions and preparation of the final report
Impact Cochrane IPD Synthesis Working Group Report
Start Year 2017
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation Amgen Inc
Country United States 
Sector Private 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation AstraZeneca
Country United Kingdom 
Sector Private 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation Bayer
Country Germany 
Sector Private 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation BioMarin Pharmaceutical
Country United States 
Sector Private 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation F. Hoffmann-La Roche AG
Country Global 
Sector Private 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation GlaxoSmithKline (GSK)
Country Global 
Sector Private 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation Medical Research Council (MRC)
Department MRC North West Hub for Trials Methodology Research
Country United Kingdom 
Sector Public 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation Novartis
Country Global 
Sector Private 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation Saarland University
Country Germany 
Sector Academic/University 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation University of Manchester
Country United Kingdom 
Sector Academic/University 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description EFSPI/PSI Data Sharing Working Group 
Organisation Vectura Ltd
Country United Kingdom 
Sector Private 
PI Contribution Input into an EFSPI / PSI working group and conrtibution to manuscript development
Collaborator Contribution Aims of the working group are: •To identify and prospectively prioritize statistical issues in data transparency •To co-ordinate statistical contributions across Europe to the data transparency debate •To disseminate relevant information on the topic across the statistical community •To develop and share a vision of the potential longer term impact of data transparency. Members of the collaboration led the working group, all partners involved in planning and contributing to manuscripts
Impact Series of manuscripts outlining best practice for preparation and analysis of shared clinical trial data is in preparation (two submitted)
Start Year 2014
 
Description Evidence Synthesis Working Group 
Organisation Medical Research Council (MRC)
Department MRC Hubs for Trial Methodology Research
Country United Kingdom 
Sector Public 
PI Contribution The HTMR Network supports the Evidence Synthesis Working Group (WG) to regularly meet and exchange ideas. This involves members of the methodology community and Hub members.
Collaborator Contribution The Evidence Synthesis Working Group focuses on the use of evidence synthesis, or systematic reviews, in the design, conduct and analysis of trials. The Working Group has prepared a series of papers to help trialists and others to make best use of evidence synthesis.
Impact A number of papers are in draft form and a survey (INVEST) of use of evidence synthesis was conducted at the 3rd International Clinical Trials Methodology Conference
Start Year 2010
 
Description HDR London (Health Data Research, London) 
Organisation Imperial College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Lead the London bid for the trials component, one of five active sites in London
Collaborator Contribution Active in HDR UK team for London sites
Impact None yet
Start Year 2018
 
Description HDR London (Health Data Research, London) 
Organisation King's College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Lead the London bid for the trials component, one of five active sites in London
Collaborator Contribution Active in HDR UK team for London sites
Impact None yet
Start Year 2018
 
Description HDR London (Health Data Research, London) 
Organisation London School of Hygiene and Tropical Medicine (LSHTM)
Country United Kingdom 
Sector Academic/University 
PI Contribution Lead the London bid for the trials component, one of five active sites in London
Collaborator Contribution Active in HDR UK team for London sites
Impact None yet
Start Year 2018
 
Description HDR London (Health Data Research, London) 
Organisation Queen Mary University of London
Country United Kingdom 
Sector Academic/University 
PI Contribution Lead the London bid for the trials component, one of five active sites in London
Collaborator Contribution Active in HDR UK team for London sites
Impact None yet
Start Year 2018
 
Description Honorary contract with EMA 
Organisation European Medicines Agency
Country United Kingdom 
Sector Public 
PI Contribution Accessing information at the EMA to inform design and conduct of trials to ensure they are better positioned for future licensing applications
Collaborator Contribution Providing access to the EMA documents and databases
Impact Not yet. PhD chapter in development
Start Year 2018
 
Description Methodology Research Collaboration with industry 
Organisation Amgen Inc
Country United States 
Sector Private 
PI Contribution Commitment to developing research activity in to design, conduct or analysis methodology in areas of mutual interest. Detailed discussions ongoing
Collaborator Contribution Commitment to developing research activity in to design, conduct or analysis methodology in areas of mutual interest. Detailed discussions ongoing
Impact None yet
Start Year 2014
 
Description Methodology Research Collaboration with industry 
Organisation AstraZeneca
Country United Kingdom 
Sector Private 
PI Contribution Commitment to developing research activity in to design, conduct or analysis methodology in areas of mutual interest. Detailed discussions ongoing
Collaborator Contribution Commitment to developing research activity in to design, conduct or analysis methodology in areas of mutual interest. Detailed discussions ongoing
Impact None yet
Start Year 2014
 
Description Methodology Research Collaboration with industry 
Organisation GlaxoSmithKline (GSK)
Country Global 
Sector Private 
PI Contribution Commitment to developing research activity in to design, conduct or analysis methodology in areas of mutual interest. Detailed discussions ongoing
Collaborator Contribution Commitment to developing research activity in to design, conduct or analysis methodology in areas of mutual interest. Detailed discussions ongoing
Impact None yet
Start Year 2014
 
Description ORCCA2 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Screening of abstracts to form the database for this project
Collaborator Contribution Initating and management of this project
Impact None yet
Start Year 2018
 
Description ORRCA - Online Resource for Recruitment research in Clinical triAls 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Categorisation of research papers for the ORRCA recruitment database.
Collaborator Contribution Development of, and overall responsibility for resource and related publications.
Impact ORRCA database.
Start Year 2016
 
Description PRISMA-IPD Steering Group 
Organisation Queen's University Belfast
Department MRC All-Ireland Methodology Hub
Country United Kingdom 
Sector Multiple 
PI Contribution Expertise and intellectual contribution. Steering Group membership (Jayne Tierney)
Collaborator Contribution Expertise and intellectual contribution
Impact Development of reporting guidelines and a checklist for the standardised reporting of systematic reviews and meta-analyses of IPD. This has been published
Start Year 2012
 
Description PRISMA-IPD Steering Group 
Organisation Radboud University Nijmegen
Country Netherlands 
Sector Academic/University 
PI Contribution Expertise and intellectual contribution. Steering Group membership (Jayne Tierney)
Collaborator Contribution Expertise and intellectual contribution
Impact Development of reporting guidelines and a checklist for the standardised reporting of systematic reviews and meta-analyses of IPD. This has been published
Start Year 2012
 
Description PRISMA-IPD Steering Group 
Organisation University of Birmingham
Country United Kingdom 
Sector Academic/University 
PI Contribution Expertise and intellectual contribution. Steering Group membership (Jayne Tierney)
Collaborator Contribution Expertise and intellectual contribution
Impact Development of reporting guidelines and a checklist for the standardised reporting of systematic reviews and meta-analyses of IPD. This has been published
Start Year 2012
 
Description PRISMA-IPD Steering Group 
Organisation University of York
Country United Kingdom 
Sector Academic/University 
PI Contribution Expertise and intellectual contribution. Steering Group membership (Jayne Tierney)
Collaborator Contribution Expertise and intellectual contribution
Impact Development of reporting guidelines and a checklist for the standardised reporting of systematic reviews and meta-analyses of IPD. This has been published
Start Year 2012
 
Description Partnership with HRA, MHRA and NCRAS/PHE for source data verification project 
Organisation Medicines and Healthcare Regulatory Agency
Department General Practice Research Database (GPRD)
Country United Kingdom 
Sector Academic/University 
PI Contribution MRC CTU is leading some work on whether electronic health record data could be used to check the quality of clinical trial data.
Collaborator Contribution The partner organisations are helping us set this up and discussing relevant issues; NCRAS/PHE will help by providing data if the project is successful.
Impact None so far
Start Year 2017
 
Description Partnership with HRA, MHRA and NCRAS/PHE for source data verification project 
Organisation NHS Health Research Authority
Country United Kingdom 
Sector Public 
PI Contribution MRC CTU is leading some work on whether electronic health record data could be used to check the quality of clinical trial data.
Collaborator Contribution The partner organisations are helping us set this up and discussing relevant issues; NCRAS/PHE will help by providing data if the project is successful.
Impact None so far
Start Year 2017
 
Description Partnership with HRA, MHRA and NCRAS/PHE for source data verification project 
Organisation National Cancer Registration and Analysis Service
PI Contribution MRC CTU is leading some work on whether electronic health record data could be used to check the quality of clinical trial data.
Collaborator Contribution The partner organisations are helping us set this up and discussing relevant issues; NCRAS/PHE will help by providing data if the project is successful.
Impact None so far
Start Year 2017
 
Description Protocol for systematic review of trial monitoring 
Organisation University of Basel
Country Switzerland 
Sector Academic/University 
PI Contribution Assitance and input to plans to develop a Protocol
Collaborator Contribution Assitance and input to plans to develop a Protocol
Impact Not yet
Start Year 2018
 
Description R20 Workshop and guidance on IPD meta-analysis 
Organisation Queen's University Belfast
Department MRC All-Ireland Methodology Hub
Country United Kingdom 
Sector Multiple 
PI Contribution Leading the development of the funding proposal, project planning and organising, developing and running a workshop. Drafting and reviewing subsequent IPD guidance papers
Collaborator Contribution Helping develop the funding proposal, project planning, developing the workshop. Drafting and reviewing subsequent IPD guidance papers.
Impact We brought international experts to the Unit to discuss key IPD topics, develop guidance outlines and establish writing committees. We have published four papers.
Start Year 2011
 
Description R20 Workshop and guidance on IPD meta-analysis 
Organisation Radboud University Nijmegen
Country Netherlands 
Sector Academic/University 
PI Contribution Leading the development of the funding proposal, project planning and organising, developing and running a workshop. Drafting and reviewing subsequent IPD guidance papers
Collaborator Contribution Helping develop the funding proposal, project planning, developing the workshop. Drafting and reviewing subsequent IPD guidance papers.
Impact We brought international experts to the Unit to discuss key IPD topics, develop guidance outlines and establish writing committees. We have published four papers.
Start Year 2011
 
Description R20 Workshop and guidance on IPD meta-analysis 
Organisation University of Birmingham
Country United Kingdom 
Sector Academic/University 
PI Contribution Leading the development of the funding proposal, project planning and organising, developing and running a workshop. Drafting and reviewing subsequent IPD guidance papers
Collaborator Contribution Helping develop the funding proposal, project planning, developing the workshop. Drafting and reviewing subsequent IPD guidance papers.
Impact We brought international experts to the Unit to discuss key IPD topics, develop guidance outlines and establish writing committees. We have published four papers.
Start Year 2011
 
Description R20 Workshop and guidance on IPD meta-analysis 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Leading the development of the funding proposal, project planning and organising, developing and running a workshop. Drafting and reviewing subsequent IPD guidance papers
Collaborator Contribution Helping develop the funding proposal, project planning, developing the workshop. Drafting and reviewing subsequent IPD guidance papers.
Impact We brought international experts to the Unit to discuss key IPD topics, develop guidance outlines and establish writing committees. We have published four papers.
Start Year 2011
 
Description R20 Workshop and guidance on IPD meta-analysis 
Organisation University of York
Country United Kingdom 
Sector Academic/University 
PI Contribution Leading the development of the funding proposal, project planning and organising, developing and running a workshop. Drafting and reviewing subsequent IPD guidance papers
Collaborator Contribution Helping develop the funding proposal, project planning, developing the workshop. Drafting and reviewing subsequent IPD guidance papers.
Impact We brought international experts to the Unit to discuss key IPD topics, develop guidance outlines and establish writing committees. We have published four papers.
Start Year 2011
 
Description R25- Remits, roles and working models for Trial Steering Committees and Data Monitoring Committees in clinical trials evaluating diagnostic tests, most notably test accuracy studies. 
Organisation University of Southampton
Country United Kingdom 
Sector Academic/University 
PI Contribution MRC Hubs for Trials Methodology Research Network funded proposal project- links in with a suite of Network research
Collaborator Contribution Review, workshop and expert consensus expected in 2015
Impact Review, workshop and expert consensus expected. Collaborative work underway.
Start Year 2012
 
Description R34 An ethnographic study of group decision making and member roles in TSC and TMG 
Organisation Medical Research Council (MRC)
Department MRC ConDuCT Trials Methodology Hub
Country United Kingdom 
Sector Public 
PI Contribution MRC Hubs for Trials Methodology Research Network funded project to work on An ethnographic study of group decision-making and member roles to understand and improve how Trial Steering Committees and Trial Management Groups contribute to trial conduct"
Collaborator Contribution Funding given from the MRC Hubs for Trials Methodology Research Network .
Impact Two papers have been published and one further paper has been submitted for publication.
Start Year 2013
 
Description R39 Agreeing current good practice for clinical trial data access and sharing 
Organisation European Organisation for Research and Treatment of Cancer (EORTC)
Country European Union (EU) 
Sector Charity/Non Profit 
PI Contribution MRC Hubs for Trials Methodology Research Network funded proposal addresses a main priority area for the HTMR Network, that of using existing evidence in the prioritisation, design and planning, conduct, and interpretation of trials, all of which can be helped by improving access to clinical trial data.
Collaborator Contribution MRC Hubs for Trials Methodology Research Network funding provided for project commencement
Impact Two papers published giving views on data sharing, rationale and examples
Start Year 2013
 
Description R39 Agreeing current good practice for clinical trial data access and sharing 
Organisation Hub at the MRC/CRUK/BHF Clinical Trial Service Unit
Country United Kingdom 
Sector Academic/University 
PI Contribution MRC Hubs for Trials Methodology Research Network funded proposal addresses a main priority area for the HTMR Network, that of using existing evidence in the prioritisation, design and planning, conduct, and interpretation of trials, all of which can be helped by improving access to clinical trial data.
Collaborator Contribution MRC Hubs for Trials Methodology Research Network funding provided for project commencement
Impact Two papers published giving views on data sharing, rationale and examples
Start Year 2013
 
Description R39 Agreeing current good practice for clinical trial data access and sharing 
Organisation Medical Research Council (MRC)
Department MRC Midland Hub for Trials Methodology Research (MHTMR)
Country United Kingdom 
Sector Academic/University 
PI Contribution MRC Hubs for Trials Methodology Research Network funded proposal addresses a main priority area for the HTMR Network, that of using existing evidence in the prioritisation, design and planning, conduct, and interpretation of trials, all of which can be helped by improving access to clinical trial data.
Collaborator Contribution MRC Hubs for Trials Methodology Research Network funding provided for project commencement
Impact Two papers published giving views on data sharing, rationale and examples
Start Year 2013
 
Description R39 Agreeing current good practice for clinical trial data access and sharing 
Organisation Medical Research Council (MRC)
Department Network of Hubs for Trials Methodology Research (HTMR)
Country United Kingdom 
Sector Academic/University 
PI Contribution MRC Hubs for Trials Methodology Research Network funded proposal addresses a main priority area for the HTMR Network, that of using existing evidence in the prioritisation, design and planning, conduct, and interpretation of trials, all of which can be helped by improving access to clinical trial data.
Collaborator Contribution MRC Hubs for Trials Methodology Research Network funding provided for project commencement
Impact Two papers published giving views on data sharing, rationale and examples
Start Year 2013
 
Description R39 Agreeing current good practice for clinical trial data access and sharing 
Organisation University of Edinburgh
Department Edinburgh Hub for Trials Methodology Research
Country United Kingdom 
Sector Academic/University 
PI Contribution MRC Hubs for Trials Methodology Research Network funded proposal addresses a main priority area for the HTMR Network, that of using existing evidence in the prioritisation, design and planning, conduct, and interpretation of trials, all of which can be helped by improving access to clinical trial data.
Collaborator Contribution MRC Hubs for Trials Methodology Research Network funding provided for project commencement
Impact Two papers published giving views on data sharing, rationale and examples
Start Year 2013
 
Description R39 Agreeing current good practice for clinical trial data access and sharing 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution MRC Hubs for Trials Methodology Research Network funded proposal addresses a main priority area for the HTMR Network, that of using existing evidence in the prioritisation, design and planning, conduct, and interpretation of trials, all of which can be helped by improving access to clinical trial data.
Collaborator Contribution MRC Hubs for Trials Methodology Research Network funding provided for project commencement
Impact Two papers published giving views on data sharing, rationale and examples
Start Year 2013
 
Description Routine electronic health records (EHR) used as trial data by randomised controlled trials in the UK 
Organisation London School of Hygiene and Tropical Medicine (LSHTM)
Country United Kingdom 
Sector Academic/University 
PI Contribution Development of a Protocol for a systematic review, undertaking review activities and manuscript preparation
Collaborator Contribution Guidance and advice on the development of the Protocol, assistance with review activities
Impact Published Protocol at url above
Start Year 2018
 
Description Routine electronic health records (EHR) used as trial data by randomised controlled trials in the UK 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Development of a Protocol for a systematic review, undertaking review activities and manuscript preparation
Collaborator Contribution Guidance and advice on the development of the Protocol, assistance with review activities
Impact Published Protocol at url above
Start Year 2018
 
Description Show RESPECT study collaboration 
Organisation Cancer Research UK
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Leading Show RESPECT study, looking at best ways to disseminate study results to study participants
Collaborator Contribution Members of steering group, helping to design, conduct and disseminate results of study
Impact None yet
Start Year 2017
 
Description Show RESPECT study collaboration 
Organisation Harvard University
Department Harvard Medical School
Country United States 
Sector Academic/University 
PI Contribution Leading Show RESPECT study, looking at best ways to disseminate study results to study participants
Collaborator Contribution Members of steering group, helping to design, conduct and disseminate results of study
Impact None yet
Start Year 2017
 
Description Show RESPECT study collaboration 
Organisation London School of Hygiene and Tropical Medicine (LSHTM)
Country United Kingdom 
Sector Academic/University 
PI Contribution Leading Show RESPECT study, looking at best ways to disseminate study results to study participants
Collaborator Contribution Members of steering group, helping to design, conduct and disseminate results of study
Impact None yet
Start Year 2017
 
Description Show RESPECT study collaboration 
Organisation NHS Health Research Authority
Country United Kingdom 
Sector Public 
PI Contribution Leading Show RESPECT study, looking at best ways to disseminate study results to study participants
Collaborator Contribution Members of steering group, helping to design, conduct and disseminate results of study
Impact None yet
Start Year 2017
 
Description Show RESPECT study collaboration 
Organisation University of Aberdeen
Country United Kingdom 
Sector Academic/University 
PI Contribution Leading Show RESPECT study, looking at best ways to disseminate study results to study participants
Collaborator Contribution Members of steering group, helping to design, conduct and disseminate results of study
Impact None yet
Start Year 2017
 
Description TEMPER-related collaboration 
Organisation University College Cork
Country Ireland 
Sector Academic/University 
PI Contribution Coordinating two pieces of work related to (extensions of) the TEMPER monitoring study (run from MRC CTU at UCL, funded by CRUK and MRC)
Collaborator Contribution PhD student at Cork, helping plan, conduct and analyse the two small projects.
Impact None yet
Start Year 2017
 
Description Trial conduct methodology collaboration 
Organisation University College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Expertise and intellectual contribution to push forward some specific areas of trial conduct methodological research
Collaborator Contribution Expertise and intellectual contribution to push forward some specific areas of trial conduct methodological research
Impact No outcomes to report as yet - collaboration still active
Start Year 2014
 
Description Trials methodology research partnership 
Organisation Health Data Research UK
PI Contribution Key member of the executive group, designed and formed the network, oversight, provision of PhD student, courses
Collaborator Contribution Key member of the executive group, designed and formed the network, oversight, provision of PhD student, courses
Impact None yet
Start Year 2018
 
Description Trials methodology research partnership 
Organisation The Global Health Network
Country Global 
Sector Charity/Non Profit 
PI Contribution Key member of the executive group, designed and formed the network, oversight, provision of PhD student, courses
Collaborator Contribution Key member of the executive group, designed and formed the network, oversight, provision of PhD student, courses
Impact None yet
Start Year 2018
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation Cancer Research UK
Department Clinical Trials Unit (CTU) Glasgow
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC)
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation Newcastle University
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation Queen Mary University of London
Department Barts Clinical Trials Unit
Country United Kingdom 
Sector Hospitals 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation University College London
Department UCL Cancer Institute
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation University of Edinburgh
Department Edinburgh Clinical Trials Unit
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation University of Leeds
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation University of Liverpool
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation University of Nottingham
Department Nottingham Clinical Trials Unit (NCTU)
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description UKCRC Registered CTUs Network: Participant data sharing group 
Organisation University of Oxford
Country United Kingdom 
Sector Academic/University 
PI Contribution Member of the working group
Collaborator Contribution Partners are involved in a working group to facilitate the implementation of data sharing good practice across the UKCRC registered CTU network, to provide advice and support to CTUs about data sharing, and to liaise with external organisations regarding data sharing.
Impact No outputs as yet
Start Year 2014
 
Description Use of routinely collected data for progression and side-effects (PhD related to STAMPEDE) 
Organisation Public Health England
Country United Kingdom 
Sector Public 
PI Contribution Guidance on study protocol, application for data access
Collaborator Contribution Supervision of PhD, data analysis
Impact Not yet
Start Year 2017
 
Description Use of routinely collected data for progression and side-effects (PhD related to STAMPEDE) 
Organisation University of Birmingham
Country United Kingdom 
Sector Academic/University 
PI Contribution Guidance on study protocol, application for data access
Collaborator Contribution Supervision of PhD, data analysis
Impact Not yet
Start Year 2017
 
Description Use of routinely collected data for progression and side-effects (PhD related to STAMPEDE) 
Organisation University of Warwick
Country United Kingdom 
Sector Academic/University 
PI Contribution Guidance on study protocol, application for data access
Collaborator Contribution Supervision of PhD, data analysis
Impact Not yet
Start Year 2017
 
Title Show RESPECT 
Description The Show RESPECT study is aiming to find practical ways to share the results of clinical trials with the people taking part in it. It is doing this by testing several different approaches within a large ovarian cancer trial (the ICON8 trial). 
Type Products with applications outside of medicine
Current Stage Of Development Late clinical evaluation
Year Development Stage Completed 2018
Development Status Under active development/distribution
Clinical Trial? Yes
Impact Nil 
URL http://www.isrctn.com/ISRCTN96189403
 
Title IPDMETAN/ADMETAN: Stata code for two-stage IPD meta-analysis and aggregate data v3.0 
Description IPDMETAN and ADMETAN are components of the same overall package, but can now be installed and updated with reference to either of the two names. This places "admetan" in direct competition with the older "metan" command as Stata's foremost meta-analysis package. 
Type Of Technology Software 
Year Produced 2018 
Impact IPDMETAN/ADMETAN was downloaded over 200 times per month on average during 2018, and remains the second most popular meta-analysis package, behind "metan". Furthermore, a notable increase was seen in email and forum activity referencing "admetan" over "metan" in the context of performing meta-analysis with Stata. 
URL https://www.stata.com/meeting/uk18/
 
Title IPDMETAN: Stata code for two-stage IPD meta-analysis v1.0 
Description The Stata package "ipdmetan" comprises a set of routines for conducting two-stage individual participant data (IPD) meta-analysis, and associated functionality. 
Type Of Technology Software 
Year Produced 2015 
Impact The package is available from the SSC archive and can be found and installed from within Stata by typing "ssc describe ipdmetan" and "ssc install ipdmetan" respectively. A Stata Journal article describing the functionality of the package has been published 
URL http://econpapers.repec.org/software/bocbocode/s457785.htm
 
Title IPDMETAN: Stata code for two-stage IPD meta-analysis v2.0 
Description IPDMETAN v2.0 with the aggregate-data routine, "admetan", and now has the majority of the functionality of the older "metan". In addition, "admetan" is far more likely to run successfully in resource-limited environments than "metan", due to various programming inefficiencies. 
Type Of Technology Software 
Year Produced 2017 
Impact IPDMETAN was downloaded over 200 times per month on average during 2017, and is now the second most popular meta-analysis package, behind "metan". 
 
Description BBC News - Statistical analysis of Stricly Come Dancing winners 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact PhD student, Susan Connolly, assisted with statistical analysis for BBC News article about predicting the winner of Stricly Come Dancing 2017 (http://www.bbc.co.uk/news/entertainment-arts-41280119 ). Susan's analyis and predictions were mentioned and praised in a subsequent article in Significance magazine (https://www.significancemagazine.com/culture/576-statistical-analysis-strictly-on-request)
Year(s) Of Engagement Activity 2017,2018
URL https://www.significancemagazine.com/culture/576-statistical-analysis-strictly-on-request
 
Description BSA Science Discussion Panel 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact PhD student, Susan Connolly, took part in BSA Science Discussion Panel in Edinburgh, debating science, statistics and the media
Year(s) Of Engagement Activity 2017
 
Description Discussion group about dissemination of trial results to participants in ovarian cancer trials 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Study participants or study members
Results and Impact Discussion group to talk about experiences of dissemination of trial results from the perspective of participants in ovarian cancer trials. Set up to provide Patient and Public Involvement input into a study about this being designed at MRC CTU.
Year(s) Of Engagement Activity 2017
 
Description Discussion of implementing novel clinical trial designs 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Interview with Matt Sydes on Radio 4's World At One to talk about adaptive and novel trial designs. This was organised by MRC Press Office after Jeremy Hunt raised adaptive trials in parliament with reference to Tessa Jowell's comments. This was an opportunity to correct some potential misunderstandings.
Year(s) Of Engagement Activity 2018
 
Description Engage conference 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Presented the MRC CTU Patient Information Sheet template at the Engage conference 2017.
Year(s) Of Engagement Activity 2017
URL https://www.publicengagement.ac.uk/nccpe-projects-and-services/engage-conference
 
Description Focus group about communicating trial results to trial participants (MRC and CRUK) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Study participants or study members
Results and Impact A one-day focus group, held by MRC CTU and Cancer Research UK, with people who had participated in a trial or were close to those who have participated.
Year(s) Of Engagement Activity 2017
 
Description How are phase III trials in the UK monitored? The story from the registered CTUs 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact A webinar was held with interested research groups from clinical trials units across the UK. including people from funding bodies. We presented the results of our survey on the practices of monitoring at trial units in the UK. Folllowing the webinar there were lots of questions about the next steps and interest from members to contribute to further research in this area.
Year(s) Of Engagement Activity 2019
URL https://www.methodologyhubs.mrc.ac.uk/resources/webinars/
 
Description MRC Festival Workshop on Getting Results Faster for Patients 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact As part of the Medical Research Council (MRC) Festival of Medical Research, the MRC Clinical Trials Unit at University College London (MRC CTU at UCL) held a half-day workshop exploring how the Unit is working to get faster results and impact from clinical trials. We invited patient representatives involved in our studies, and patient groups who are interested in clinical trials. 30 people from 20 patient groups and various trials attended the workshop, along with around 20 members of MRC CTU at UCL staff.
Year(s) Of Engagement Activity 2016
URL http://www.ctu.mrc.ac.uk/news/2016/getting_results_faster_workshop_report_010816
 
Description PPI Induction Pack development workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact A workshop to develop a PPI package
Year(s) Of Engagement Activity 2018
 
Description Parliamentary Links Lunch 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Policymakers/politicians
Results and Impact PhD student, Susan Connolly, attended Parliamentary Links Lunch at House of Lords, meeting Daniel Zeicher MP, discussing importance of statistics (organised via RSS Ambassador Scheme)
Year(s) Of Engagement Activity 2017
 
Description Survey about experiences of receiving trial results among trial participants 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Study participants or study members
Results and Impact Short survey, disseminated via social media, to find out trial participants' experiences and views about receiving results of trials they take part in.
Year(s) Of Engagement Activity 2017
 
Description Twitter Q+A session: #fastertrials #fasterimpact 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Held a Twitter Q+A as part of the MRC Festival of medical research 2016. It focused on how to speed up both the conduct of trials and the impact of these trials' results.
Year(s) Of Engagement Activity 2016
URL http://www.ctu.mrc.ac.uk/news/2016/faster_trials_qa_230616