NERC/DEFRA Joint Water Evidence Group Evidence Review Fellow

Lead Research Organisation: Imperial College London
Department Name: Centre for Environmental Policy

Abstract

The project will enable a Policy Placement Fellow to work within the core Joint Water Evidence Group to further the development of their Evidence Review (ER) methodology. The Joint Water Evidence Group was established to enhance collaborative working on land and water issues across the Defra network to ensure the best use of resources and embrace innovative approaches to gathering evidence. Evidence Reviews search for, review and summarise the evidence available in response to a given question in order to communicate findings to end-users, making better use of existing research and ensuring that research has real impact The project will work to refine the guidance for ER methodology and practice, incorporating feedback and comments from end users and stakeholders, thereby improving the understanding of the science-policy interface. The Fellow will also work with policy and practice makers and other stakeholders in the water area to identify their objectives and evidence needs in order to develop the questions to be answered by ERs. The Fellow will be responsible for the project management of a number of ERs and will ensure a high standard of delivery. Through engagement with the Natural Environment Research Council and other Research Councils the Fellow will ensure that new and up to date research is fully incorporated into ERs, facilitating knowledge exchange. The project will also ensure effective and timely communication, in the appropriate format, to a wide range of stakeholders in order to maximise the benefits of the project to policy makers, businesses and practitioners.
 
Description The funding from NERC facilitated the exploration of methods to systematically gather and synthesise evidence and developed these so that they could be used to produce reviews of evidence for use in environmental decision making. The funding enabled a Fellow to work as a core member of the Joint Water Evidence Group (JWEG). JWEG work across Defra and its associated agencies to coordinate and collaborate on water and land research. This provided a deep understanding of the use of evidence to meet decision makers' needs, so that outputs produced were fit for purpose.

The work of the fellowship enable methods to be develop to aid:
•setting appropriate questions that will meet decision makers' needs
•unbiased searching and screening of evidence
•searching for evidence from a variety of sources, including software that can assist with this
•critically appraising the quality of evidence found
•quantitative and qualitative synthesis of evidence
•assessing and communicating uncertainty in an evidence base
•creating reports appropriate for decision makers, and
•reporting methods transparently
Exploitation Route The continued use of the How-to-Guide will lead to further improvement of methods to gather and synthesise evidence to meet decision maker's needs throughout Defra and other public bodies. This will result in further efficiencies and evidence-informed decisions being made.

Continued promotion, sharing of experiences and provision of training would enable this impact to be increased. Knowledge exchange would also enable lessons learned to be shared , thus increasing capabilities of external researchers and contractors to meet decision makers' evidence needs.

The open access paper regarding the application of lessons from systematic reviews to literature reviews also has the potential to have wider impact. This will demonstrate the benefits of methods to mitigate bias and outline practical methods of applying these to those conducting reviews in both policy and academic sectors. Engagement with the training of researchers such as with the Centres of Doctoral Training and further work with evidence specialists that support decision makers would increase this.

A number of decision makers, researchers and consultancies from around the world have expressed interest in the use of evidence reviews to meet decision makers' needs. Therefore there is potential for a number of international end-users to take up the methods and tools developed and for the UK to lead internationally in methods to provide evidence to inform environmental decision making.
Sectors Communities and Social Services/Policy,Energy,Environment,Government, Democracy and Justice

URL http://www.gov.uk/government/publications/the-production-of-quick-scoping-reviews-and-rapid-evidence-assessments
 
Description Research led to a How-to-Guide on the production of evidence reviews to be developed, outlining principles of systematic reviews and devising guidance on how these can be applied to more rapid methods (Rapid Evidence Assessments and Quick Scoping Reviews) that fit time-frames usually required in decision making contexts. A number of tools such as protocols, checklists and forms were also developed to aid the efficient production of reviews. The How-to-Guide has been adopted as guidance for the production of Rapid Evidence Assessments and Quick Scoping Reviews for Defra, the Environment Agency and Natural England. It has been used to set details of contracts and guide specifications for several research and development projects ensuring that robust methodologies were followed and outputs produced were of high quality and helped to inform policy and practice decisions. During the last year there has been continued interest in the work which has led to the US EPA using the methodology guidance for a number of projects as well as it being used by the Scientific Advisory Mechanism of the European Commission. Defra continue to promote and use the methodology and it has been used in a review of Antimicrobial Resistance. During 2017-2018 the work was used to develop methodology for Evidence Statements adopted by Defra's Chief Scientific Advisor and used in three Evidence Statements on air quality, upland management and the benefits of urban trees. A course On Evidence Synthesis for Environmental Policy (E-StEP) for Postgraduate students and funded by NERC was designed and delivered based on the work of the Fellowship. During 2018-2019 The work was used to produce more reviews and evidence statements by Defra. Additionally, work informed a paper on engaging policy makers with systematic reviews and helped with the successful proposal of a systematic map on green house gas emissions from arable land management. Work with researchers at the University of Melbourne and the US EPA led to proposals for evidence banking and an editorial in the Journal Freshwater Science. This continues to be trialled. Engagement has been undertaken with the IPCC working group 3 and British Ecological Society about how to apply systematic approaches to the work of these organsiations. During 2019 work continued on evidence statements at Defra and engagement continued with the University of Melbourne and the US EPA, we plan to host a joint conference session at the Environmental Evidence bi-annual conference in 2020, along with a workshop on rapid review methods in Washington in late 2020. The work of the fellowship was also used to develop a successful NERC proposal for a Systematic Map on GHG emissions from arable agriculture. Materials from the fellowship continue to be used in MSc and DTP research methods training
Sector Environment
Impact Types Policy & public services

 
Description Delivery of evidence reviews for the Joint Water Evidence Group and Water and Flood Risk Management directorate Defra
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact The work of the fellowship has contributed to the project management and delivery of a number of evidence reviews used to inform decision making in the water and flood risk management directorate of Defra
URL https://defra.bravosolution.co.uk/esop/toolkit/opportunity/opportunityDetail.do?opportunityId=19985&...
 
Description Developing methodologies to communicate uncertainty in summaries of evidence
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
Impact Research into methods to assess uncertainty in summarised bodies of evidence was provided to Defra. This will help in the production of evidence summaries which will be used to communicate evidence on politically important environmental topics.
 
Description Development of ER methodology for meeting decision makers needs within Environmental disciplines
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact The research conducted enabled a How to Guide on the Production of Rapid Evidence Assessments (REAs) and Quick Scoping Reviews (QSRs) to be produced for Defra. REAs and QSRs are methods that apply systematic approaches to searching for and synthesising evidence but with limited scope in order to reduce the resources and time frames required for their completion. The guide is being used a across Defra and its delivery partners in order to design and carryout evidence reviews that meet environmental policy and practices decision makers needs. The guide has been used for a number of contracts for individual projects as well as a managed service provider contract. Work on evidence reviews was recently mentioned in Defra's Evidence Strategy and work with the Chief Scientific Advisor and Defra's Strategic Evidence and Analysis team continues
URL https://defra.bravosolution.co.uk/esop/toolkit/opportunity/opportunityDetail.do?opportunityId=21877&...
 
Description Evidence Statement Methodology
Geographic Reach National 
Policy Influence Type Citation in other policy documents
Impact Work on systematic evidence reviews in environmental disciplines led to the development of Evidence Statement methodology adopted by Defra's Chief Scientific Advisor. Input has been provided for the development of 3 Evidence Statements and work has also been used in a Rapid Evidence Assessment on Risks to Human Health from Antimicrobial Resistance in the Water Environment (£31,684).
 
Description Evidence Statement on Air Quality
Geographic Reach National 
Policy Influence Type Citation in other policy documents
Impact Provide expert input to the creation of an Evidence Statement on Air Quality for Defra's Chief Scientific Advisor
 
Description Evidence Statement on Benefits of Urban Trees for Human Health
Geographic Reach National 
Policy Influence Type Citation in other policy documents
Impact Provide expert input to the creation of an Evidence Statement on Benefits of Urban Trees for Human Health for Defra's Chief Scientific Advisor
 
Description Evidence Statement on Upland Management
Geographic Reach National 
Policy Influence Type Citation in other policy documents
Impact Provide expert input to the creation of an Evidence Statement on Upland Management for Defra's Chief Scientific Advisor
 
Description Guidance on evidence reviews provided on Defra's intranet an EA's wiki site
Geographic Reach National 
Policy Influence Type Implementation circular/rapid advice/letter to e.g. Ministry of Health
Impact Sharing of guidance and best practice on Defra's intranet for evidence and analysis specialists, along with sharing on the EA's wiki enviropedia. Links can not be provided as they are only available internally.
 
Description Community Water Management for a Liveable London (CAMELLIA)
Amount £4,079,082 (GBP)
Funding ID NE/S003495/1 
Organisation Natural Environment Research Council 
Sector Public
Country United Kingdom
Start 09/2018 
End 09/2023
 
Description Nature Smart Cities across the 2 Seas https://www.interreg2seas.eu/nl/nsciti2s
Amount € 6,380,472 (EUR)
Funding ID nsciti2s 
Organisation European Union 
Sector Public
Country European Union (EU)
Start 09/2018 
End 02/2022
 
Description What are the impacts of agricultural soil and crop management on greenhouse gas fluxes? - Informing post Brexit agricultural subsidy policy
Amount £79,306 (GBP)
Funding ID NE/S015949/1 
Organisation Natural Environment Research Council 
Sector Public
Country United Kingdom
Start 01/2019 
End 01/2020
 
Description Defra Flood and Water Cluster 
Organisation Department For Environment, Food And Rural Affairs (DEFRA)
Country United Kingdom 
Sector Public 
PI Contribution The How-to guidance on the production of Quick Scoping Reviews and Rapid Evidence Assessments has been used by the cluster in developing contract specifications and the delivery of projects to meet evidence and analysis needs.
Collaborator Contribution Feedback on the development of the guidance and promotion of its use. Developing specifications and tender documents and processes
Impact Systematic Review of Environmental Needs to Support Water Abstraction Reform (EBPLW12044) Quick Scoping Review: Factors contributing to surface water flooding (FD2692) - http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None?ojectID=19252&FromSearch=Y&Publisher=1&SearchText=FD2692&SortString=ProjectCode&SortOrder=Asc&Paging10#Description Do On-farm Natural, Restored, Managed and Constructed Wetlands Mitigate Agricultural Pollution in Great Britain and Ireland? - Systematic Review (SR) (WT0989) - Outputs on Defra website The Health Risks of Bathing in Recreational Waters: a Rapid Evidence Assessment of Water Quality and Gastrointestinal illness (WT1530) (A JWEP assisted project) -http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None?ojectID=18942&FromSearch=Y&Publisher=1&SearchText=WT1530&SortString=ProjectCode&SortOrder=Asc&Paging=10#Description Impact of Contaminated Land on Controlled Waters (QSR) (WT1545) http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None?ojectID=19176&FromSearch=Y&Publisher=1&SearchText=WT1545&SortString=ProjectCode&SortOrder=Asc&Paging=10#Description The Impact of Pesticides Used for Amenity Purposes on Controlled Waters (QSR) (WT1546) - http://randd.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None?ojectID=19195&FromSearch=Y&Publisher=1&SearchText=quick%20scoping%20review&SortString=ProjectCode&SortOrder=Asc&Paging=10#Description Emerging tools and techniques to deliver timely and cost effective evidence reviews (WT1552) - http://sciencesearch.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None?ojectID=19265&FromSearch=Y&Publisher=1&SearchText=WT1552&SortString=ProjectCode&SortOrder=Asc&Paging=10#Description
Start Year 2014
 
Description Defra Strategic Evidence and Assessment/ Office of the Cheif Scientific Advisor 
Organisation Department For Environment, Food And Rural Affairs (DEFRA)
Country United Kingdom 
Sector Public 
PI Contribution Contributed to learning and development seminars run across the Department to promote evidence review methodologies Contributed to internal guidance that has been promoted by the department Contributed to the development of guidance on how to assess and communicate uncertainty in bodies of evidence. Developed evidence statement methodology to provide systeamtic methods to describe an evidence base for a high level topics and an evaluation of the uncertainties in these. Provided expert input to the creation of 3 evidence statements being commissioned by evidence teams in Defra and the Chief Scientific Advisor
Collaborator Contribution Supported the development of the guidance by providing input in terms of understanding of requirements Gave a platform to promote methods through organising training events. Organised the development of high level questions that could be addressed using an evidence statement methodology, these were supported by the Chief Scientific Advisor and his office
Impact Development of internal and external guidance on evidence reviews https://www.gov.uk/government/publications/the-production-of-quick-scoping-reviews-and-rapid-evidence-assessments. Three evidence statements using the methodology developed have been written and these will be published shortly. More evidence statements are in progress and being planned
Start Year 2013
 
Description US EPA 
Organisation United States Environmental Protection Agency
Department Office of Research and Development
Country United States 
Sector Public 
PI Contribution U.S. EPA's Office of Research and Development is beginning work on three systematic reviews to support nutrient and biological criteria work in the Office of Water. These reviews focus on the responses of macroinvertebrates, diatoms, and chlorophyll a to total nitrogen and total phosphorus in lotic freshwaters. We will compare systematic review processes and results to three rapid review methods that differ from systematic reviews in terms of 1) search comprehensiveness, 2) data extraction, and 3) study quality assessment. During one or more of these systematic reviews, detailed data on time requirements for searching, screening, data extraction, and quality assessment will be collect, allowing us to identify the most time-intensive parts of the process. We will also implement alternative quality assessment approaches as applied in Eco Evidence and Rapid Evidence Assessment methods, in collaboration with colleagues in Australia (S. Nichols, A. Webb) and the UK (A. Collins).
Collaborator Contribution The US EPA have set up the project and have committed to using the How-to-guide methodology developed. they have facilitated my participation in the project as an advisor and contributor.
Impact A scope of the work has been produced but no other outputs or outcomes have results as yet
Start Year 2017
 
Description Collaboration for Environemtnal Evidence Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Contributed as an invited speaker to the session 'Rapid review methods and their role in environmental management' as part of the Collaboration for Environmental Evidence annual conference. The presentation looked at The use of Rapid Evidence Reviews in Policy and Practice Decision-Making. The presentation was followed by a round table discussion and a number of breakout discussions
Year(s) Of Engagement Activity 2016
URL http://www.environmentalevidence.org/wp-content/uploads/2014/04/Conference-Booklet-for-web-1.pdf
 
Description Evidence Review Workshop for NERC Fellows 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact A 3 hour workshop was delivered as part of NERC's training for Research and Knowledge Exchanges Fellows. This introduced the concepts for evidence reviews and outlined how to deliver them as well as providing exercise to develop experience
Year(s) Of Engagement Activity 2015
 
Description Evidence Reviews Under a Managed Service Provider 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact A workshop to review evidence reviews produced under a Manages Service Provider for Defra. This shared experiences of how to manage reviews and to update on the developments of guidance to enable knowledge exchange about best practice processes
Year(s) Of Engagement Activity 2015
 
Description Evidence Reviews for the Marine Directorate Defra 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact The Marine Directorate at Defra were interested in the experiences of using Evidence Reviews and how this could help them to meet their evidence needs. A presentation of our findings and an introduction to the How-to-Guide was given.
Year(s) Of Engagement Activity 2015
 
Description Evidence Statement on Air Quality 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Inception meeting of the Evidence Statement on Air Quality 04/10/2017 and subsequent meetings to provide expert input to the completion of the statement
Year(s) Of Engagement Activity 2017,2018
 
Description Inception meeting of the Evidence Statement on Upland Land Use 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Inception meeting of the Evidence Statement on Upland Land Use 01/11/2017 and subsequent meetings to provide expert input to the completion of the Evidence Statement
Year(s) Of Engagement Activity 2017,2018
 
Description Intergovernmental Panel on Climate Chnage Working Group 3 and DFID 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Preliminary discussion on the use of systematic methods in the work of IPCC. I provide input about the benefits of this and how methods could be tailored for a range of resource requirements
Year(s) Of Engagement Activity 2018
 
Description Intergovernmental Panel on Climate Chnage Working Group 3 and DFID 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Ongoing discussion on the use of systematic methods in the work of IPCC. I provided input about the benefits of this and how methods could be tailored for a range of resource requirements and on the provision of training materials
Year(s) Of Engagement Activity 2019
 
Description NERC Knowledge Exchange Workshop on Evidence Reviews 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact 40 academics and decision makers attended a workshop to learn more about the use of evidence reviews in policy making

Increase knowledge regarding evidence reviews and policy making in the audience
Year(s) Of Engagement Activity 2014
 
Description Presentation at the European Environmental Evaluators Network 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A presentation on Providing Evidence Syntheses for Policy and Practice Decision Makers: Experiences Gained from the UK Department for Environment, Food and Rural Affairs
Year(s) Of Engagement Activity 2015
 
Description Presentation on communicating uncertainty in evidence syntheses to NCAS Policy Forum. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Presentation on communicating uncertainty in evidence syntheses to National Centre Atmospheric Science Policy Forum.
Year(s) Of Engagement Activity 2018
 
Description Presentation to the Air Land and Water Evidence Team at the Environment Agency 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact A presentation on the use of evidence reviews for decision making and the findings of the Fellowship
Year(s) Of Engagement Activity 2015
 
Description Presentation to the Evidence into Action Team, Department for International Development 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Thirty members of the Evidence into Action Team at the Department for International Development attended a presentation on evidence review methodologies, followed by a discussion of the How-to-Guide
Year(s) Of Engagement Activity 2016
 
Description Presentation to the Swedish Mistra Council for Evidence based Environmental Management 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I met with the Swedish Mistra Council for Evidence based Environmental Management to share experience of working to meet policy makers evidence needs and information regarding the methodologies that are used

The Swedish Mistra Council are considering using more rapid approaches to evidence reviews in their work and have taken on board our experience of communicating with policy makers, e.g. how best to communicate in briefings and summaries
Year(s) Of Engagement Activity 2014
 
Description Project advice for Defra's Flood and Water Cluster 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact I contributed to an expert panel assessing tender documents for a project on Antimicrobial Resistance
Year(s) Of Engagement Activity 2017
 
Description Project discussions with US EPA and University of Melbourne 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Contributed to a discussion formalising the scope of a project investigating alternative methodologies for reviewing evidence to inform water policy and practice
Year(s) Of Engagement Activity 2017,2018
 
Description Reviewing and Communicating Evidence Seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact A learning and skills seminar for Defra on reviewing and communicating evidence
Year(s) Of Engagement Activity 2015
 
Description Society of Fresh Water Science Annual Conference in Sacramento 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Gave a conference = presentation on "Using Evidence Reviews to Inform Water and Land Management: Experiences from Policy and Practice" at the Society of Fresh Water Science Annual Conference in Sacramento. This was followed by a panel discussion. The talk was well attended and generated a lot of interest. I subsequently held a number of meetings with representatives of the US EPA which led to pursuing further collaborations with them.
Year(s) Of Engagement Activity 2016
URL http://sfsannualmeeting.org/archive/2016/Schedule/grid_Topics.cfm?dtid=644,645,646,647,648,649&pdtid...
 
Description US EPA discussions on rapid reviews 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Ongoing discussions on the application of rapid review methods and an international workshop on best practice
Year(s) Of Engagement Activity 2018,2019
 
Description Using systeamtic approaches at the British Ecological Soceity 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Ongoing assistance on how to apply rapid review methods for the BES work
Year(s) Of Engagement Activity 2018,2019
 
Description Workshop at the Collaboration for Environemtnalk Evidence Annual Conference 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Organised and ran a training workshop on making research more reliable using lessons from systematic reviews
Year(s) Of Engagement Activity 2016
URL http://www.environmentalevidence.org/wp-content/uploads/2014/04/Conference-Booklet-for-web-1.pdf
 
Description • Inception meeting of the Evidence Statement on Benefits of Trees to human health 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Inception meeting of the Evidence Statement on Benefits of Trees to human health 10/01/2018 and subsequent meetings to provide expert input to the completion of the Evidence Statement
Year(s) Of Engagement Activity 2018