A UK web-based probability panel? Exploring the issues, opportunities and design considerations.

Lead Research Organisation: University College London
Department Name: Quantitative Social Science

Abstract

This project will look at whether the Economic and Social Research Council, which is a major funder of economic and social science research in the UK, should fund a major new research panel in the UK. The idea is recruit a large number of people (the 'panel'), representative of the UK population, and ask them to fill in questionnaires regularly over the internet. The questionnaires would cover lots of different topics, and researchers who wanted to know about a particular topic could ask for their questions to be included. This would mean that lots of researchers who don't have the time or money to set-up their own study would be able to get answers to the questions they are interested in quickly, and because the panel is already set-up and uses the internet, this would be relatively cheap too.

The purpose of this project is to look at whether it is a good idea to set-up such a panel, how best to do this and how much it would cost. We will speak to researchers in universities, government and the voluntary sector about whether they would use it, what they might use it for and what it would need to be like in order to meet their needs. We would also look at how similar panels in other countries are run and who uses them, and look at information about what kinds of studies the ESRC funds at the moment, to work out whether any of them could use the new internet panel instead. We will also look into how best to design such a panel, including how to identify the people to take part and how to encourage them to answer the surveys. Again, to do this we would speak to researchers who might use it, and look at how similar panels are designed and run.

The idea is that a panel like this could mean that UK social scientists are able to do as much (or more) high-quality research, but for less money. This would mean that UK taxpayers' money is used more efficiently and effectively.

Planned Impact

There are four key groups who will benefit from the proposed study. First, as discussed in the section on academic beneficiaries, researchers from a wide range of disciplines will benefit should this study lead to the creation of a major new economic and social science research resource. Secondly, government policy makers will potentially benefit from this research resource. The potential use of a UK web-based probability panel for government surveys will be explored as part of the exploratory study. Our current expectation is that there are likely to be a number of government surveys which could potentially be carried out using such a panel. It would potentially equip government policy-makers with a quick, relatively cheap and easy-to-use source of high-quality evidence on the UK general population, with which to inform policy. Thirdly, voluntary sector organisations would benefit from the creation of a web-based probability panel in the UK, as it would provide them with a robust and relatively inexpensive source of information on the general population. Fourthly, the general public would potentially benefit from the findings of the research carried out on the web-based panel, and also the panel could potentially lead to a more effective and efficient use of tax-payers money.

Publications

10 25 50
 
Description We have provided the Economic and Social Research Council (ESRC) with key evidence and information on which to base their decision regarding whether or not to commission a web-based probability panel in the UK. Such a panel would be a major new research resource for collecting high-quality survey data, stimulating new and innovative research opportunities as well as allowing more research ideas to be financially viable. In our project, we explored who would use the panel, how the panel should be designed and how much it would cost. We also considered strategic issues, such as governance, access and funding.

In relation to demand, we found that there was interest in using the panel among academics as well as government and voluntary sector researchers. The overall size of the panel and of key population sub-groups were important considerations for whether they would use the panel, with researchers in Wales, Scotland and Northern Ireland in particular stressing the importance of adequate sample sizes in these countries. The content of the core questionnaire was also viewed as key; researchers wanted this to be sufficiently broad for it to provide key contextual and background variables for their research. Government researchers in particular were attracted by the ability of the panel to provide rapid information, and for ad-hoc or emerging research needs. Overall, the panel was viewed as complementary and distinct from existing government and ESRC research resources. There was also a strong sense that the panel would advance research opportunities in the area of online data collection, and generate demand for new forms of data using innovative methods.

In relation to design, we produced a detailed set of design options and recommendations in relation to ten key design considerations. This was accompanied by an interactive costing tool showing how the costs of the panel change if different combinations of design options are chosen. We recommend aiming for a panel of 10,000+ active panelists from a target population of all adults aged 16+ in private households in the UK, with a minimum of 1,000 per country. Whole households would be sampled and the offline population should be included by offering a tablet and appropriate data subscription. We also recommend aiming for high recruitment, retention and study-specific co-operation rates, in part as this was viewed by many as crucial to bringing credibility to the panel. We propose starting with 30 minute questionnaires per month, but broken into separate modules that can be completed at any time during the month. Some modules should be more tightly date-bound to facilitate 'rapid response' research. A maximum of 25% of available data collection time should be ring-fenced for a cross-disciplinary core study. Given the radical nature of some of the possibilities with regard to new data collection, we recommend that a specialist group should be established to consider this further.

In relation to strategic issues, we made recommendations in relation to how users should pay for the panel, how data collected on the panel should be made available for secondary use, scientific and strategic governance, commissioning and maximising the potential for innovation and new forms of data collection.
Exploitation Route ESRC, and other potential funders, will be able to use this information to decide whether or not to commission a web-based probability panel, to develop a detailed invitation to tender with and to set aside appropriate budget.

More generally, the report provides the UK research community with detailed information on potential demand for such a panel, and on design options.
Sectors Digital/Communication/Information Technologies (including Software),Education,Government, Democracy and Justice,Other

URL http://www.cls.ioe.ac.uk/page.aspx?&sitesectionid=1352&sitesectiontitle=A+new+resource+for+social+science%3a+exploratory+study
 
Description The ESRC have not taken forward the recommendations in the report produced by this project. However, another agency - NatCen Social Research - have started their own probability panel, influenced indirectly by the scientific discussion, dialogue and mood changes generated by this project.
First Year Of Impact 2016
Sector Education,Healthcare
Impact Types Societal,Policy & public services