HRQ:Reevaluating the Remote: a realist evaluation of the practical, epistemological and ethical status of remote qualitative data collection in health
Lead Research Organisation:
University of Warwick
Department Name: Warwick Medical School
Abstract
The COVID-19 pandemic forced many qualitative researchers to collect their data remotely (for example, online or via phones). However, little is known about the effects of these methods on the quality of resulting data, nor its impacts on participants. It is not clear whose voices might be missed, and there is little existing guidance on which remote methods work best, and under what circumstances. This project explores the impacts of remote data collection, both on the data it produces, and the people who produce it (researchers and research participants).
Firstly, a review of existing research evidence will be conducted. This review will draw together, and assesses, previous studies that have used both remote and in-person methods within one study, and compared them. By assessing the data in these studies and synthesising them, it will be possible to build up a picture of the effects of remote collection on data quality across many studies to form an evidence base.
Once it is understood what effect remote data collection has on data quality, a second review of published studies will be conducted. This review will explore how, and under which circumstances good quality remote qualitative data is produced. To do this, the review will focus on health researchers' reflections and ideas, as well as recommendations for carrying out remote research that are published in research guidelines. Through the course of this review, theories will be developed to help answer the questions of how, and under which circumstances, the best quality data can be produced.
In order to test that the developing theories are correct, approximately thirty interviews will be conducted with researchers and research participants. Researchers who have previously done remote studies will be chosen for interview from a wide range of professional backgrounds and types of health research. People who have previously taken part in research (remotely or in-person) will also be interviewed to explore their perspectives on what makes data collection work well. To ensure a wide variety of people are included, the findings of the reviews will be looked at to see whose voices might sometimes get missed by using remote research methods (e.g. elderly people, or those who do not speak English). Special efforts will be made to also interview these people, to find out why they might not participate in remote research and how this could be changed, using community links and networks to find them.
The results of the interviews will be used to finalise the theories about what works, and what doesn't, in creating good quality data using remote methods. In conjunction with a panel of eight people from the general public, these theories will then be written up into a framework of guidance, to help future researchers make decisions about how to design and carry out their studies.
In the final stage of the project, a conference will be held with members of the public, research participants, researchers and organisations who fund research. This will be an opportunity to present the guidance, and to get feedback on it from a range of stakeholders. In small break-our groups, the participants will debate and discuss any recommendations that are not clear, or are controversial. After discussion, participants will reach a consensus on the best way to present the guidance to researchers to ensure the highest quality of research.
Following the conference, the final framework will be made available to researchers throughout the world via publications, a website resource and an online training module. It will be advertised on research methods training websites, and distributed to research funders to help them assess the quality of new applications. A whiteboard style animated video showing the development of the framework will also be created, posted on youtube and shared online to highlight the project, as well as the role of the general public in creating it.
Firstly, a review of existing research evidence will be conducted. This review will draw together, and assesses, previous studies that have used both remote and in-person methods within one study, and compared them. By assessing the data in these studies and synthesising them, it will be possible to build up a picture of the effects of remote collection on data quality across many studies to form an evidence base.
Once it is understood what effect remote data collection has on data quality, a second review of published studies will be conducted. This review will explore how, and under which circumstances good quality remote qualitative data is produced. To do this, the review will focus on health researchers' reflections and ideas, as well as recommendations for carrying out remote research that are published in research guidelines. Through the course of this review, theories will be developed to help answer the questions of how, and under which circumstances, the best quality data can be produced.
In order to test that the developing theories are correct, approximately thirty interviews will be conducted with researchers and research participants. Researchers who have previously done remote studies will be chosen for interview from a wide range of professional backgrounds and types of health research. People who have previously taken part in research (remotely or in-person) will also be interviewed to explore their perspectives on what makes data collection work well. To ensure a wide variety of people are included, the findings of the reviews will be looked at to see whose voices might sometimes get missed by using remote research methods (e.g. elderly people, or those who do not speak English). Special efforts will be made to also interview these people, to find out why they might not participate in remote research and how this could be changed, using community links and networks to find them.
The results of the interviews will be used to finalise the theories about what works, and what doesn't, in creating good quality data using remote methods. In conjunction with a panel of eight people from the general public, these theories will then be written up into a framework of guidance, to help future researchers make decisions about how to design and carry out their studies.
In the final stage of the project, a conference will be held with members of the public, research participants, researchers and organisations who fund research. This will be an opportunity to present the guidance, and to get feedback on it from a range of stakeholders. In small break-our groups, the participants will debate and discuss any recommendations that are not clear, or are controversial. After discussion, participants will reach a consensus on the best way to present the guidance to researchers to ensure the highest quality of research.
Following the conference, the final framework will be made available to researchers throughout the world via publications, a website resource and an online training module. It will be advertised on research methods training websites, and distributed to research funders to help them assess the quality of new applications. A whiteboard style animated video showing the development of the framework will also be created, posted on youtube and shared online to highlight the project, as well as the role of the general public in creating it.
Technical Summary
This project will use qualitative evidence synthesis combined with realist evaluation and embedded co-production to produce stakeholder informed programme theories capable of explaining the conditions and circumstances under which remote data collection methods produce high quality and trustworthy qualitative data. These programme theories will be transformed into a framework of principles and recommendations to guide researchers in the future design and implementation of remote qualitative studies.
A systematic evidence synthesis will firstly be conducted, of studies directly comparing in-person and remote qualitative methods. The studies will be appraised, and methods compared both within and between papers along core dimensions (structure, epistemology, ethics, interactional differences) that relate to data validity and trustworthiness. The quality of the resulting review will be assessed using the CERQual approach to ensure rigour.
From the evidence synthesis, initial programme theories will be iteratively developed and tested within a realist review of researcher experience literature, and published guidelines on remote data collection. Evidence will be included in this review based on relevance to emerging theories, and synthesised. The programme theories will be further interrogated and finalised through 30 remote interviews with qualitative researchers, research participants and seldom heard voices, purposively sampled for maximum diversity.
Finally, in co-production with a PPI panel, a framework of guidance will be developed from the final set of programme theories. This framework will be ratified through a stakeholder consensus conference, where complex or contentious recommendations will be deliberated. The final framework will be disseminated through teaching, training, health research networks, an animated video and a web resource. It will be used to guide the design, and conduct, of remote qualitative data collection for future research.
A systematic evidence synthesis will firstly be conducted, of studies directly comparing in-person and remote qualitative methods. The studies will be appraised, and methods compared both within and between papers along core dimensions (structure, epistemology, ethics, interactional differences) that relate to data validity and trustworthiness. The quality of the resulting review will be assessed using the CERQual approach to ensure rigour.
From the evidence synthesis, initial programme theories will be iteratively developed and tested within a realist review of researcher experience literature, and published guidelines on remote data collection. Evidence will be included in this review based on relevance to emerging theories, and synthesised. The programme theories will be further interrogated and finalised through 30 remote interviews with qualitative researchers, research participants and seldom heard voices, purposively sampled for maximum diversity.
Finally, in co-production with a PPI panel, a framework of guidance will be developed from the final set of programme theories. This framework will be ratified through a stakeholder consensus conference, where complex or contentious recommendations will be deliberated. The final framework will be disseminated through teaching, training, health research networks, an animated video and a web resource. It will be used to guide the design, and conduct, of remote qualitative data collection for future research.
