What are the impacts of user involvement in health and social care research and how can they be measured?

Lead Research Organisation: Lancaster University
Department Name: Division of Health Research

Abstract

People who use health and social care services can provide different perspectives on research from those of clinicians, policy makers and academic researchers. Involving service users in research is now a priority for government policy and is required by many organisations that fund research. User involvement (UI) can occur at all stages of research, including: identifying and prioritising research questions, designing and conducting research, considering the implications of findings for services and ensuring results influence practice and future research (Chalmers, 1995; Hanley, et. al. 2004). However, whilst there is considerable support in the research community some people argue that its impacts are limited or difficult to identify. Commentators have also argued that the involvement of service users in research can be associated with problems of representativeness, bias, quality and increased cost. There is also evidence from other fields that getting involved in official decision-making processes can have negative impacts on service users(Langston et al, 2005; O?Donnell & Entwistle, 2004; Ziersch & Baum, 2004). Robust methods to assess the impact of UI in research are therefore needed. This project aims to develop and test methods for doing this.

The project will review existing literature to describe different perspectives on the value of UI in research and identify what is already known about the impacts it can have. For example, involving service users in research could result in the outcomes that are measured being more relevant to patients, and help people develop new skills. We will also try to identify factors that seem to be causally linked to these impacts (for example, whether service users were asked which outcomes they valued) and ways in which these impacts have been measured and/or assessed in the past. We will then conduct a survey of funders of research, researchers and service users to explore in more depth areas of consensus and conflicts about the value of UI in research compared to other research values (for example, value for money) and consider how conflicts between values might be managed. We will then use all this information to develop guidance on how to assess the impact of UI in research and test this in a small number of pilot studies. The project will also produce standards against which approaches to user involvement in research can be assessed to help researchers maximise positive impacts of UI in research and avoid adverse effects.

Technical Summary

This research aims to advance understanding of the impact of user involvement (UI) in health and social care research and contribute to more robust approaches to the measurement of these impacts and to the development of evidence standards against which practice in UI can be assessed. Building on recent literature reviews our objectives will be to:
? Produce a framework identifying impacts of UI in research and factors associated with these;
? Identify different perspectives on the value of UI in research in relation to other values, explore areas of consensus and conflict and consider ways in which conflicts might be managed;
? On the basis of the above to develop and test guidance on the assessment and/or measurement of impacts and standards for assessing good practice.
The study design consists of five phases of work. During Phase 1 we will collate evidence, including recent systematic reviews, on (i) models of UI in research, impacts and factors associated with these; (ii) values and normative debates about UI in research; and (iii) evaluative approaches including specific tools and techniques for measurement and assessment of impacts. Phase 2 will consist of an adapted Delphi process that will consider the nature and extent of consensus and conflict around the value of UI in health and social care research and explore whether and how conflicts could be accommodated in policy/practice. The third phase of work will involve the development of a User Involvement Impact Assessment Framework (UI-IAF) and associated guidance on its use in different settings; piloting this in the context of existing research activity in three settings (priority setting; funding allocation; and research projects); and in light of these pilots, refinement of guidance and UI-IAF. A fourth phase of work will draw on the earlier phases to produce a report on standards for good practice in UI focusing on how positive impacts of UI in health and social care research might be maximised and adverse effects/negative impacts avoided. During the fifith and final phase we will consult with a wider group of stakeholders on the outputs of the research at a large deliberative event. The three main outputs ? a report on contemporary normative debates about UI in research, standards for good practice in UI and Guidance on the Assessment and Measurement of UI Impacts - will be revised in light of these discussions and plans for dissemination will be produced.

Publications

10 25 50
publication icon
Gradinger F (2015) Values associated with public involvement in health and social care research: a narrative review. in Health expectations : an international journal of public participation in health care and health policy

 
Description Jan 2013 Workshop NIHR wide approach to PI in research
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Use of PiiAF framework to shape evaluation of public and patient involvement in NIHR CLAHRCS
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Use of the PIIAF within Bristol University's BRU research http://www.uhbristol.nhs.uk/media/2637357/ppiannualreport2016final.pdf
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
URL http://www.uhbristol.nhs.uk/media/2637357/ppiannualreport2016final.pdf
 
Guideline Title Community Engagement: improving health and reducing health inequalities NICE Guidance 44
Description expert advice to NICE guidance committee revising guidance on community engagement in health improvement
Geographic Reach National 
Policy Influence Type Citation in clinical guidelines
URL https://www.nice.org.uk/guidance/NG44
 
Description reference to PiiAF in INVOLVE newsletter
Geographic Reach National 
Policy Influence Type Citation in other policy documents
 
Description CLAHRC infrastructure grants
Amount £9,000,000 (GBP)
Organisation NHS England 
Department NHS North West England
Sector Public
Country United Kingdom
Start 01/2014 
End 12/2018
 
Description NIHR school for Public Health Research - second phase funding
Amount £21,000,000 (GBP)
Organisation National Institute for Health Research 
Department NIHR Biomedical Research Centre
Sector Academic/University
Country United Kingdom
Start 04/2017 
End 03/2022
 
Description School for Public Health Research
Amount £2,200,000 (GBP)
Organisation National Institute for Health Research 
Department NIHR Central Commissioning Facility
Sector Public
Country United Kingdom
Start 04/2012 
End 03/2017
 
Title Public Involvement Impact Assessment Framework PiiAF 
Description this is the main output from the MRC grant. It is an online resource to support researchers develop plans to assess the impact of involving the public in their research. A set of resoures are also included to support the development process. The PiiAF can also be used for training researchers and members of the public/service users in issues and process for involving the public in research 
Type Of Material Improvements to research infrastructure 
Year Produced 2013 
Provided To Others? Yes  
Impact PiiAF was only launch this september so it is early days for impacts to be identified. However, several discussions are underway with research groups and funders about how PiiAF can be utilised in the future. 
URL http://piiaf.org.uk/
 
Title Tools/procedures for evaluating PI impact and Delphi survey of PI values 
Description our research team has designed an internal process and impact evaluation of public involvement in our own research and data collection instruments for use with members of the research team and members of the public involved in our research. we have also designed Delphi survey instruments to explore values associated with PI in research 
Type Of Material Improvements to research infrastructure 
Year Produced 2013 
Provided To Others? Yes  
Impact our approach to internally evaluating the process and outcomes of public involvement in research has been adapted for use by the MRC Hubs for Trials Methodology Research and by Dr Carrol Gamble in her NIHR HS&DR programme funded study: "An evidence base to optimise methods for involving patient and public representatives in clinical trials: a systematic investigation of a cohort of Health Technology Assessment funded clinical trials" 
 
Description PPI Ignite Award Ireland 
Organisation University of Limerick
Country Ireland 
Sector Academic/University 
PI Contribution Acting as expert adviser to Professor Anne MaCFarlane at the University of Limerick on their bid to the Irish Health Research Board to establish a programme of capacity building in public involvement for health research within and beyond their University
Collaborator Contribution The programme will be lead and delivered by Professor Anne MacFarlane and her colleagues in Limerick
Impact none yet
Start Year 2016
 
Description Supporting the use of PiiAF in an academic research unit 
Organisation Lancaster University
Department Spectrum Centre for Mental Health Research
Country United Kingdom 
Sector Academic/University 
PI Contribution Supporting the Spectrum Research Centre at Lancaster University to design and implement evaluation of their policies for public/service user involvement in their research
Collaborator Contribution Member of the PiiAF research team has worked closely with the Spectrum team supporting them through the PiiAF learning and development process
Impact Drafting paper on this experience should be submitted by Summer 2017
Start Year 2015
 
Description evaluation support for James Lind PSP on Kidney transplantation 
Organisation James Lind Alliance
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution we were asked to provide evaluation support using the PiiAF resource to help one of the James Lind PSP steering groups to develop an approach to evaluating public and patient involvement in the PSP process.
Collaborator Contribution they provided feedback on the support provided and the PiiAF in particular that will help us improve the online resource
Impact we will use the feedback from this collaboration to improve the online resource in due course
Start Year 2014
 
Title online resource - Public Involvement Impact Assessment Framework (PiiAF) 
Description our project will result in a conceptual framework to help people develop robust approaches to assessing the impact of public involvement in their research and associated guidance on how best to use this framework. we are currently developing these products in paper format but will be seeking additional funding to develop a web based resource 
Type Support Tool - For Fundamental Research
Current Stage Of Development Small-scale adoption
Year Development Stage Completed 2013
Development Status Actively seeking support
Impact there are none yet but eventually the impact should be more extensive and higher quality assessment of the impacts of public involvement in research and maximisation of positive impacts and avoidance of negative impacts. 
URL http://piiaf.org.uk/
 
Description April 2012 Qualitative Research Support Group seminar, Institute of Health Services Research, Peninsula Medical School, University of Exeter, 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Paper Presentation
Geographic Reach Local
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Dr Felix Gradinger a research fellow on the team gave a paper reporting on the results of our narrative review of literature entitled: 'Values and Normative debates associated with User and Public Involvement (UPI) in Health and Social Care Research - mixed-methods literature review employing narrative synthesis methods'

This was attended by around 20 academics who expressed considerable interest in the work and asked to be kept informed about project outputs.
Year(s) Of Engagement Activity 2012
 
Description Engaging adult, youth and professional stakeholders in patient-centred outcomes research: an iterative process 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This workshop was organised by the International Conference on Communication in Healthcare and took place in New Orleans, USA. The presentation focused on methods for engaging service users in decision making in health care research. It was an international conference involving academics health professionals, practitioners and service users which triggered a good discussion about the values of engagement of service users.
Year(s) Of Engagement Activity 2015
 
Description International conference Patient and Public Involvement in research: values, impact and theory 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach International
Primary Audience Health professionals
Results and Impact more than 100 academic nurses and methodologists attended the plenary presentation at the European Academy of Nursing Science Summer Conference at, Radboud University, Nijmegen Medical Centre, Nijmegen, The Netherlands.

Colleagues at the University of Nijmegen are now in discussions about using the MRC funded Public Involvement Impact Assessment Framework in their research
Year(s) Of Engagement Activity 2013
 
Description Jan 2012 Workshop with public in SW on Delphi study 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Workshop Facilitator
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact 10 people involved in Exeter University's public engagement with research group attended the meeting and there was much discussion of the aims and purpose of the delphi study

the sample to be developed and the Delphi survey instrument changed as a result of these discussions
Year(s) Of Engagement Activity 2012
 
Description July 2012 meeting of project national advisory network 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? Yes
Type Of Presentation Paper Presentation
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact The meeting involved a series of power point presentations updating participants on the aims and objectives of the research and progress on the different work strands. Five members of the research team were involved.

Eleven people attended this meeting from diverse backgrounds including representatives from the NIHR INVOLVE - the national organisation supporting public invovlement in health and social care research and PIs on NIHR funded research assessing the impact of public involvement. Very useful discussion ensued and there will be future meetings of this group.
Year(s) Of Engagement Activity 2012
 
Description July 2012 teleconference meeting with SW PAG 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? Yes
Type Of Presentation Workshop Facilitator
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact PAG members are also members of other public advisory groups in the NW and SW and hence report back on PAG discussions to others.

Continued advise on various aspects of the development of our guidance and impact assessment framework including refining the discussion resources we are developing
Year(s) Of Engagement Activity 2012
 
Description Launch of Public Involvement Impact Assessment Framework 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact A mixed audience of around 50 people including publics and patients involved in research, research funders, policy makers and academics attended the launch of the public involvement impact assessment framework - the final output of the research. There was considerable discussion about the value and potential uses of PiiAF.

Several people attending the launch have since made contact to explore ways in which they can use PiiAF in the future to help them improve the quality of their public invovlement. PiiAF was referenced in several publications as a result of the luanch including an editorial in Nature.
Year(s) Of Engagement Activity 2013
URL http://piiaf.org.uk/
 
Description May 2012 Medical Grand Round, Galway University Hospitals, National University of Ireland, Galway, 4 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach Local
Primary Audience Health professionals
Results and Impact Professor Nicky Britten gave a presentation entitled 'Patient and public involvement in research: values, mechanisms and outcomes' to an audience of around 50 clinicians and researchers and there was a rich discussion afterwards with all keen to improve the way they involve the public in their research.

General enthusiasm for the work and requests to be kept informed about the publication of outputs from the project
Year(s) Of Engagement Activity 2012
 
Description May 2012 Meeting of project Public Advisory Group 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? Yes
Type Of Presentation Workshop Facilitator
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact There are eight formal members of our project's public advisory group (PAG) and different numbers attended on different occasions ranging from all 8 to 4. The meetings are organised by our user investigators who are full members of the project research team. PAG members are also members of other public advisory groups in the NW and SW and hence report back on PAG discussions to others.

There were various impacts on the developing framework for assessing the impact of public involvement in research including identifying domains in the framework the research team had not thought about and elaborating other elements of the work.
Year(s) Of Engagement Activity 2012
 
Description May 2012 workshop with researchers 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Workshop Facilitator
Geographic Reach Local
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact 5 researchers involving members of the public/service users in their health related research attended and there was rich discussion of the proposed content and format of the guidance and impact assessment framework we are developing.

valuable suggestions for improving the format of the impact assessment framework and related guidance have resulted in changes.
Year(s) Of Engagement Activity 2012
 
Description Meeting of Project Public Advisory Group 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? Yes
Type Of Presentation Paper Presentation
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact There are eight formal members of our project's public advisory group (PAG) and different numbers attended on different occasions ranging from all 8 to 4. The meetings are organised by our user investigators who are full members of the project research team. PAG members are also members of other public advisory groups in the NW and SW and hence report back on PAG discussions to others.

The PAG was advising on the impact assessment framework and associated guidance. They have previously made useful contributions to the development of this framework. On this occasion they advised on the appropriateness of planned resources and made very useful suggestions about how these could be improved and extended particularly in terms of making the language we are using accessible to more diverse audiences.
Year(s) Of Engagement Activity 2012
 
Description Nov 2012 Workshop at INVOLVE conference 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Type Of Presentation Workshop Facilitator
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact The research team led by Professor Popay and Dr Collins. delivered an interactive workshop entitled: 'Developing guidance for a values based approach to assessing the impact of public involvement in research' at the INVOLVE national conference 'Putting People first in Research, 13th and 14th November Nottingham. http://www.profbriefings.co.uk/involve2012/


70 people attended this interactive workshop - the largest number attending any of the workshops at the conference. There were a number of very significant suggestions for refining our impact assessment framework and improving the associated guidance which we will be taking forward.
Year(s) Of Engagement Activity 2012
URL http://www.profbriefings.co.uk/involve2012/
 
Description Nov 2012 meeting of PIs involved in NIHR/MRC funded research on impact of PI 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Type Of Presentation Workshop Facilitator
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact around 20 researchers were involved in this meeting which explored opportunities for shared learning across projects all focusing on approaches to assessing the impact of public/service user involvement in health and social care research and policy

the group identified a number of topics/issues were researchers might collaborate on papers and these will be followed up.
Year(s) Of Engagement Activity 2012
 
Description Oct 2012 Public Advisory Group meeting 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? Yes
Type Of Presentation Workshop Facilitator
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact five PAG members attended and feed back to the other pubic involvement in research groups and networks they are involved in

The PAG gave useful feedback on the timing of the second round, of evaluation and suggestions for how the evaluation instruments could be made more relevant to them have led to changes.
Year(s) Of Engagement Activity 2012
 
Description PI in research international conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach International
Primary Audience Health professionals
Results and Impact over a hundred people attended this international conference New Developments in Public Involvement in Research, 13-15th November Exeter University. This was an invited keynote presentation on barriers and enablers to public involvement in research and policy and on the new Public Involvement Impact Assessment Framework

considerable interest in the online PiiAF resource and the results of our NIHR funded research on community engagement and health inequalities.
Year(s) Of Engagement Activity 2013
URL http://clahrc-peninsula.nihr.ac.uk/ppi-conference.php
 
Description Patient and public involvement in research: values, mechanisms and outcomes. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Keynote/Invited Speaker
Geographic Reach Local
Primary Audience Health professionals
Results and Impact Students and health professionals attended the presentation given as part of the Medical Grand Round, Galway University Hospitals, National University of Ireland, Galway. There was very lively discussion about the issues raised.



Very positive feedback on a stimulating and thought provodking presentation
Year(s) Of Engagement Activity 2013
 
Description Pen CLAHRC workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Type Of Presentation Paper Presentation
Geographic Reach National
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact 20 Patient and Public Involvement representatives from the CLAHRCs PPI group attended in London in Spring 2013 exploring areas of consensus and conflict in Public Involvement in Research

Representatives feed back that they would take away useful insights to improve their approach to the management of conflict around public involvement in CLAHRC research
Year(s) Of Engagement Activity 2013
 
Description Presentation on public and patient involvement in research: what, why and how? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This presentation was focused on methods for involving service users in research with a particular focus on values. It was delivered as a plenary presentation to an international conference "Researching Complex Interventions of Health: State of the Art" held at Exeter University. it stimulated a lot of interest and requests for details on how to access PiiAF
Year(s) Of Engagement Activity 2015
 
Description Service user Forum 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact informal presentation about the research to NW People in Research Forum a consortia of community/service user groups, universities and NHS organistions supporting public engagement with research sparked questions and discussion.

no impact other than that mentioned above.
Year(s) Of Engagement Activity 2011
 
Description repeated presentations to NHS services, third sector mental health charities, and user groups to recruit into our research across the UK 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Study participants or study members
Results and Impact Our research always requires extensive recruitment straetgies, and dissemination back to stakeholders and participants. Its not possible to list every activity

NA
Year(s) Of Engagement Activity Pre-2006,2006,2007,2008,
 
Description two presentations at a research focused symposium in Oxford UK 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Two presentations were delivered to a symposium in Oxford health service researcher and engaged public/service users exploring approaches to measuring the impact of public and service user involvement in health.
Year(s) Of Engagement Activity 2015
 
Description workshop on Public Involvement in Research conference, 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Type Of Presentation Paper Presentation
Geographic Reach Regional
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Meeting of academics, health professionals and members of the public involved in the NIHR Collaboration for Leadership in Applied Health Research and Care for the South West Peninsula, Exeter. lively discussion of the challenges of public involvement in research ensued.

considerable interest - invited to contribution key note to national conference on public involvement in Exeter in November 2013
Year(s) Of Engagement Activity 2012
 
Description workshop with University service users advisory group 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact about 25 people attended workshop - there was a lively discussion and request for further updates on the research

Members of the group have asked to be involved in future stages of the research consultation process
Year(s) Of Engagement Activity 2011