Comparing in-person to self-completion interviews: the example of the 2021 European Social Survey in the UK.
Lead Research Organisation:
City, University of London
Department Name: School of Social Sciences
Abstract
Face-to-face fieldwork is becoming more expensive in the UK whilst response rates have been declining. Many UK surveys have been considering whether a switch to using self-completion instead of interviewer-administered surveys offers a more efficient, digital, data collection future. This project funds the fielding of the 2021 ESS in the UK as a self-completion survey. That will be conducted in an overlapping period with the (already funded) face-to-face fieldwork. The dataset created will allow analysts to examine how similar the estimates from the self-completion survey are to those from the face-to-face survey.
Organisations
People |
ORCID iD |
| Rory Fitzgerald (Principal Investigator) | |
| Tim Hanson (Researcher) |
| Description | this project aimed to see if you got the same or different resutls when running a survey in self-completion mode compared to face to face. The data collection phase for the self-completion survey ran for the study ran from November 2021 to February 2022. This involved sending a postal invitation, two reminder letters, and a 'survey relaunch' letter to a random sample of GB addresses to ask them to complete a 50-minute online survey using the Round 10 European Social Survey questionnaire. A paper questionnaire was sent to non-responders with the second reminder. All sampled addresses were sent a £5 unconditional cash incentive. The study also involved a conditional incentive experiment: those who responded were sent either £0, £5 or £10 depending on their allocation to incentive group based on the experiment included in the study. A further £5 unconditional incentive was sent to non-respondents in the relaunch phase. An overall response rate of 36% was achieved, out of which 10% of the responses were completed on paper, while the rest completed the survey online. The total number of participants in this study was N=2908. This comfortably exceeded the predicted overall 25% for the study. The response rate varied somewhat between the conditional incentive groups: 34% where no conditional incentive was offered, 36% where £5 was offered, and 39% where £10 was offered. However, the response rate by incentive for the paper participants was stable across the three incentive groups (between 9.4% and 10.4%). The survey relaunch phase added around 3 percentage points to the response rate. The achieved sample composition was very similar to that achieved for the ESS Round 9 face-to-face survey in Great Britain (this was carried out in 2018; the data from the Round 10 face-to-face survey, carried out in 2021-22 are unfortunately not yet available). This showed that across a range of indicators, including sex, age and years of education, the samples produced from the two surveys were similar. The central objective of the study was to compare the results from this self-completion approach with the ESS Round 10 face-to-face survey in GB. Preparations for this are underway, with full analysis being carried out once the face-to-face data set is available (expected in April 2023). Preliminary analysis also investigated the data quality and results show that the data collected using the self-completion methods is of comparable quality to data collected in previous rounds using face-to-face interviews. Since the survey was completed online and on paper, we were able to closely investigate the breakdown of responses by mode and for the online responses, by device: 27% of the total sample completed the survey on paper; 23% used a mobile phone; 6% used a tablet; and about 43% used a computer. The study data set and documentation were deposited at the UK Data Archive in 2022 and is due to be published by UKDA as part of its curated collection in April 2023. |
| Exploitation Route | The results from this study will feed into the data colelction mode switch for ESS in 2027. They will help to guide users on the likely impact of the data coillection switch on time series analysis. The results will also feed into the ESRC Survey Data Collection Methods Collaboration (If fudned). |
| Sectors | Education,Healthcare,Government, Democracy and Justice,Other |
| Description | The findigns are feeding into survey practice and discussions in the survey industry |
| Sector | Other |
| Impact Types | Societal |