📣 Help Shape the Future of UKRI's Gateway to Research (GtR)

We're improving UKRI's Gateway to Research and are seeking your input! If you would be interested in being interviewed about the improvements we're making and to have your say about how we can make GtR more user-friendly, impactful, and effective for the Research and Innovation community, please email gateway@ukri.org.

Linking survey and experimental data: behavioural experiments in health and wellbeing

Lead Research Organisation: London School of Economics and Political Science
Department Name: Social Policy

Abstract

The aim of this project is to assess the validity of data collected through experimental methods, by linking them with survey data. While survey data allows large scale collection of a broad range of topics, they might be unable to capture individual preferences, and in particular preferences for risk and time which play a major role in human decisions and behaviour. The different ways in which risk is perceived, and future benefits are evaluated in relation to immediate costs, in fact, can explain a range of behaviours affecting health and wellbeing, from diet, physical activity and smoking, to savings, pensions and long-term care plans. Whilst some surveys ask questions about these preferences, none have used experimental methods ensuring that responses are incentive-compatible, i.e., motivated by real monetary rewards.
At the same time, experiments need to move out of traditionally student-centred labs to involve more general samples. The time is ripe for bringing together experimental methods and general population surveys. Crossing these two different methods of gathering data is crucial for the analysis of human behaviour, as it allows combining experimental evidence on risk and time preferences, with many other data that control for individual histories and backgrounds.
The research will be carried out in the context of health risky behaviour. In particular we will combine data for the same subjects collected by the Understanding Society longitudinal study (USoc), with data collected in experiments where we will assess individuals' attitudes towards risk and time. The research will encompass data collection in several points in time in order to analyse how subjects health risky behaviour changes over time and interact with changes in risk and time preferences.
The project will have a major academic, policy, and social impact. First, it will link, for the first time, a range of experimentally-derived measures with survey data for an active representative panel of the UK population, further enhancing the leadership of Understanding Society as a unique source of high quality and impact data for academic and policy research.
Second, the project will advance inter-disciplinary research and contribute to bridge the gap between the lab and the field, ultimately pushing forward the frontiers of experimental methods in behavioural and social sciences, and exploring the synergies between different research methods and areas, including experimental analysis, health research, econometric analysis of survey and experimental data, social policy, behavioural sciences.
Third, the focus on health and wellbeing will further enhance the policy relevance and impact of the research, allowing to better relate health behaviours and key behavioural attitudes such as risk and time preferences. The project will provide valuable insights to inform the design of policies to bring about changes in behaviour and wellbeing, ultimately enhancing our understanding of some of the foundations of human behaviour.
To ensure that the project will have the highest impact and will benefit the broadest community of users, decision-makers and researchers, a
range of active dissemination strategies will be pursued, including: publication in high impact journals, dissemination seminars for academics and policy makers, presentation of results in international conferences, policy briefings and public events, engagement with the media with a professional newsletter and producing regular media briefings, building a dedicated webpage at the LSE website where podcasts and research results will be regularly published.

Planned Impact

The project aims to combine survey data from the Innovation Panel (IP) of Understanding Society (USoc) with new experimental data on risk preferences (the desire for taking a gamble) and time preferences (the degree to which the present is valued more than the future). The project will explore the links between risk and time preferences and behaviour, with a focus on health and wellbeing. A sub-sample of the respondents within the IP will take part into two experimental sessions, taking place at an interval of 12 months, where risk and time preferences will be measured using incentive-compatible tests. Combined survey and experimental data will be used to empirically address three specific research questions related to the "validity" of experimental measures, namely: i) the cross-validation of survey and experimentally-derived measures; ii) the links between risk and time preferences and health-related behaviour; and iii) the stability over time of risk and time preferences and their response to life changes.
The project aims at benefiting a wide community of researchers - from economics to psychology, from social policy to health and behavioural sciences - in three distinct ways.
First, the project will elicit, for the first time, experimental measures for risk and time preferences for a representative sample of the UK population, and it will link them over time with current survey responses, which themselves are being linked with administrative data. USoc will become the first survey to contain a sub-sample of current panel members whose survey responses will be linked to, and broadly cross-validated by, incentive-compatible tests, which will be available to develop further applications and linkages.
This fits the ESRC priority themes in the call, encouraging research "involving secondary analysis of existing datasets and the application of innovative research methods", and will further enhance the leadership of USoc as a unique source of high impact data for academic and policy research. It will also advance the attempts to combine experimental and survey data (e.g. in the Netherlands, using Tilburg's CentERpanel), and to conduct field experiments with samples of the general population (e.g. in Germany and UK, replicating questions in the SOEP and BHPS; in Denmark).
Second, the project will bridge the gap between the lab and the field, ultimately pushing forward the frontiers of experimental methods in behavioural and social sciences, and exploring the synergies and cross-fertilisation between different research methods and inter-disciplinary areas, including experimental analysis, health research, econometric analysis of survey and experimental data, social policy, behavioural sciences.
Third, the focus on health and wellbeing will further enhance the impact of the research, allowing to better relate risk and time preferences and health behaviours, and to provide valuable insights to inform the design of policies to bring about changes in behaviour and wellbeing.
To achieve the highest impact, several dissemination strategies will be pursued, including: i) Involving world-leading experts within the advisory group; ii) Targeting high-impact peer-reviewed journals, e.g. the American Economic Review, Lancet, and Science; iii) Circulating results in working papers and policy briefings making them immediately available before publication; iv) Organising dissemination events: LSE and ISER have excellent research infrastructures for knowledge exchange and strong links with international academic and policy-making institutions; v) Presenting results at international conferences, e.g. the Royal Economic Society, and the American Economic Association; vi) Building a website for posting events, working papers, and uploading slides, videos and podcasts of presentations. vii) Engaging actively with the media, newsletters and websites (e.g. VoxEU) through regular media briefings with the help of knowledge translators.

Publications

10 25 50
 
Description Key findings

ESRC Future Research Leader Project
Linking Experimental and Survey Data: Behavioural Experiments in Health and Wellbeing
(ES/K001965/1, PI: MM Galizzi)

Dr Matteo M Galizzi
London School of Economics, m.m.galizzi@lse.ac.uk

There are three key findings of my ESRC FRL project. Each finding has been incorporated in (at least) one research paper that has been submitted to a leading peer-reviewed scientific journal. Each key finding is originally contributing to the existing literature and evidence both substantially and methodologically.

The first key finding and paper is about linking for the first time behavioural economics experiments with longitudinal survey data for a large representative sample of the UK population (Paper 1: Linking experimental and survey data for a UK representative panel: structural estimation of risk and time preferences). The paper documents for the first time with a representative sample of respondents within the Understanding Society panel (also known as UKHLS) some of the 'classic' behavioural economics anomalies that have previously been showed exclusively for small and specific samples of student subjects. This study is of interest to research organizations that run longitudinal surveys with representative samples of the population. It illustrates the case that integrating behavioural economics measures in ongoing surveys is feasible, and that promising insights can be gathered by systematically linking experimental, survey, administrative, and biomarkers data, and by running behavioural economics studies outside traditionally student-centred lab experiments. It has attracted attention from the research teams that are managing some other leading longitudinal surveys and epidemiological cohorts in the UK, France, and Finland. I have been invited into a policy workshop at Paris School of Economics in September 2015 to illustrate this case for data linking. I got involved in a series of knowledge exchange and impact activities between 2015 and 2016, culminating in the successful London Experimental Week (LEWeek) and in some high-level workshops at the LSE on the frontiers of 'behavioural data linking'.

The second key finding and paper is a randomized controlled experiment comparing the responses to behavioural economics tests across a traditional face-to-face computer assisted professional interview (CAPI) and a web-based interview mode (Paper 2: Estimates of risk and time preferences do not differ across web and face-to-face incentivised experiments: evidence from a UK representative survey). The experiment finds no statistically significant differences in the elicited responses across the two interview modes when the analysis adjusts for selection in the mode and controls for different sample composition. Respondents make significantly less 'errors' when responding to the web mode, compared to the CAPI mode. The study is of interest under two perspectives. First, there is currently a trend to 'migrate' large representative surveys to the web, for budget-saving reasons and to minimize attrition and non-response. Second, behavioural scientists are increasingly conducting experiments using online samples. The study shows that nothing in the 'structurally estimated' behavioural parameters would be lost by moving online, and if anything, web responses could benefit from higher precision in the estimates of behavioural parameters. The study has attracted the interest of organisations such as Science Europe, the 'umbrella' organization that represents all funders and research councils in Europe, that has invited me to a workshop in Bruxelles in May 2015 to present the project and contribute for making the case of integrating research infrastructures for the behavioural sciences, bringing closer together lab, field, and online experiments.

The third key finding consists of two 'parallel' papers. The first paper looks systematically at the 'cross-validity', 'temporal stability' and 'external validity' of three different measures of risk preferences, by focusing on risky health behaviours (Paper 3: Stability, cross-validity, and external validity of risk preferences measures: experimental evidence from a UK representative survey). To date, this is the most comprehensive and systematic exercise of this type to be conducted with a representative sample of the population. It has raised attention among behavioural and experimental economists in the UK and abroad and in networks such as the London Behavioural and Experimental Group (LBEG), and the Behavioural Experiments in Health Network, BEH-Net). A second paper has replicated the same research design looking at the time preferences measures, in particular by focusing on happiness and subjective wellbeing (Paper 4: Subjective wellbeing and 'inconsistent' time preferences: experimental evidence from a UK representative sample). The preliminary findings of have attracted the interest of a working committee at the Department of Health and the NICE that has invited me to join as 'expert' in a meeting in July 2015. Reckoning the mounting evidence against the assumption that time discounting is constant, the meeting aimed to discuss the introduction of new time discounting models to be used in the current cost-effectiveness analyses practices in the UK healthcare sector.

The key findings have been presented (or accepted) in international conferences:
• 2016: Royal Economic Society Conference, Sussex; FUR Conference on Foundations of Utility and Risk, Warwick; NIBS Conference in Behavioural Science and Policy, Norwich; EuHEA Conference in Health Economics, Hamburg.
• 2015: NIBS Conference in Behavioural Science and Policy, Nottingham; SPI-Hub Annual Conference, Chicago; London Experimental Workshop, London; UKHLS Biannual Users Conference, Colchester; HESG, Lancaster; ESA, Heidelberg; Hospinnomics Workshop, Paris; ExperiMetrix, Alicante; LBEG London Experimental Week, London.
• 2014: iHEA/ECHE, Dublin; ESA, Prague; IMEBESS Nuffield College, Oxford; NIBS Workshop in Behavioural Science, Nottingham; London Experimental Workshop, London.
• 2013: ESA, Zurich; London Health Economics Group Workshop, London.

The key findings have also been presented at:
Universities of: Aberdeen; Brescia; Ca' Foscari, Venice; City University; Cologne; Copenhagen; Durham; Erasmus University Rotterdam; Imperial College; Lancaster; Leeds; LSE; LUISS, Rome; Lund; Middlesex; Nottingham; Oxford; Paris School of Economics; Reading; Toulouse; UCL, London; Verona.
Invited talks at: Health Economics Global Congress 2015, London; 16th APES Conference of the Portuguese Association of Health Economists, Braga; 2nd UCL Centre for Behaviour Change Workshop, London; International Workshop on Smart Data and Methods in Health Economics, Paris; 11th 'BergamoScienza' Science Festival, Bergamo; Science Europe Workshop on Towards Networked Research Infrastructures for Behavioural Studies, Bruxelles.
Exploitation Route Paper 1 has attracted attention from the research teams that are running some other leading longitudinal surveys and epidemiological cohorts in Europe.

Paper 2 has attracted the interest of organisations such as Science Europe, the 'umbrella' organization that represents all funders and research councils in Europe. Science Europe has in fact invited me to a workshop in Bruxelles in May 2015 to present the design of and the main findings of the project, and has subsequently invited me to contribute to draft a new European policy brief for making the case of integrated research infrastructures for the behavioural sciences, bringing closer together lab, field, and online experiments.

Papers 3 and 4 have attracted the interest of a working committee at the Department of Health and the NICE that has invited me to join the committee as 'expert' in a meeting in July 2015. The meeting aimed to discuss the introduction of new time discounting models to be used in the current cost-effectiveness analyses practices in the UK healthcare sector. The committee, in fact, has reckoned that the current practice of using a constant discounting model may no longer be appropriate or accurate, given the mounting experimental evidence cumulating against the assumption that time discounting is indeed constant.
Sectors Creative Economy

Digital/Communication/Information Technologies (including Software)

Education

Healthcare

Government

Democracy and Justice

Pharmaceuticals and Medical Biotechnology

 
Description The findings of the project have already been used in four directions. First, the inherently interdisciplinary nature of the project has motivated the launch of a cross-disciplinary cross-university platform where experimental and behavioural researchers can meet and interact in London. The project, in fact, has directly fed into the foundation of the London Behavioural and Experimental Group (LBEG), coordinated by Matteo M Galizzi, which brings together, for the first time ever, experimental and behavioural scientists from all the disciplines (economics, psychology, neuroscience, philosophy, business), and from all the universities in London: LSE, UCL, LBS, Imperial Business School, King's College, Birkbeck, Queen Mary, City, SOAS, Royal Holloway, Middlesex, Westminster. The LBEG has set up a dedicated website: www.behavioural-experimental.london, which has greatly enhanced the visibility of the London behavioural research community and the profile of each LBEG member. In the last two years, the LBEG has animated several initiatives including: a series of very popular LBEG Workshops with world-leading behavioural scientists of the calibre of Colin Camerer (Caltech), Gary Charness (UCSB), John List (Chicago), George Loewenstein (Carnegie Mellon), Graham Loomes (Warwick), Matthias Sutter (Cologne), Peter Wakker (Rotterdam), among others; two London PhD Experimental Worskhops; a ground-breaking and highly successful London Experimental Week in December 2015 (LEWeek: http://leweek.co.uk/) involving a series of knowledge exchange and dissemination events taking place in the same week across all the London universities with leading speakers from the policy, business, and academic worlds, such as Orazio Attanasio (UCL), Daniel Berry (NHS England), Tim Chadborn (Public Health England), Stefan Dercon (Oxford, DFID), Benedikt Herrmann (Nottingham, European Commission), Aniol Llorente-Saguer (QMUL), Alison Mclean (Swiss Re), Rosemarie Nagel (Universitat Pompeu Fabra), Giovanni Ponti (Alicante, LUISS), Eyal Winter (Hebrew University of Jerusalem, Leicester); and two PhD training workshops at the LSE Behavioural Research Lab; two popular Jiscmail mailing lists regularly reaching thousands of researchers and practitioners interested in behavioural experiments (behaviour-and-experiments@jiscmail.ac.uk) and behavioural public policy (bpp@jiscmail.ac.uk). The LBEG has also established connections with other existing networks in the UK (e.g. Network for Integrated Behavioural Science, NIBS; Behavioural and Experimental Northeast Cluster, BENC) and internationally (e.g. Science of Philanthropy Initiative, SPI-Hub; Behavioural Experiments in Health Network, BEH-Net). The foundation of the LBEG is thus a long-standing legacy of the project that will foster and immensely facilitate any future dissemination and knowledge exchange strategies among the behavioural and experimental scientists in the London area, and the practitioners and decision-makers in the business, policy, and no-profit sectors. Second, the methodologically innovative design of the project has attracted attention from the research teams and organizations that run and manage some leading longitudinal panels and epidemiological cohorts in Europe, such as the TwinsUK Registry, the UK BioBank, and the Millennium Cohort Study in the UK, the KELA National Drugs Prescriptions Register in Finland, and the Constances Panel in France. Matteo M Galizzi has been invited into a high-level policy workshop at Paris in September 2015 to illustrate the case for data linking. He has also been awarded HEIF5 funding to organise a series of knowledge exchange and impact activities at the LSE, culminating in a high-level workshop on the frontiers of 'smart big data' and 'behavioural data linking', involving key decision-makers in leading business, policy, and research institutions such as the Financial Conduct Authority, Google, Dunnhumby, Ogilvy and Mathers, Prolific Academic, Pandora, NESTA, ESRC, Science Europe, Wellcome Trust, Health Foundation, among others. Third, the methodologically innovative design of the project has also attracted the interest of organisations such as Science Europe, the 'umbrella' organization that represents all funders and research councils in Europe. Science Europe has in fact invited Matteo M Galizzi to a workshop in Bruxelles in May 2015 to present the design of and the main findings of the project, and has subsequently invited him to contribute to draft a new European policy brief for making the case of integrated research infrastructures for the behavioural sciences, bringing closer together lab, field, and online experiments. Fourth, the preliminary findings of this project have attracted the interest of a working committee at the Department of Health and the National Institute of Health and Care Excellence (NICE) that has invited Matteo M Galizzi to join the committee as 'expert' in a meeting in July 2015. The meeting aimed to discuss the introduction of new time discounting models to be used in the current cost-effectiveness analyses practices in the UK healthcare sector. The committee, in fact, has reckoned that the current practice of using a constant discounting model may no longer be appropriate or accurate, given the mounting experimental evidence cumulating against the assumption that time discounting is constant. For the same reasons, Matteo M Galizzi has also been invited in February 2019 to a panel discussion at the British Embassy in Paris on how to use behavioural insights for policy applications, with a special emphasis on health, together with representatives of the behavioural unit within Public Health England, the Department of Health Policy Research Unit, the Behavioural Insights Team, as well as numerous counterparts in France.
First Year Of Impact 2015
Sector Communities and Social Services/Policy,Creative Economy,Education,Healthcare,Culture, Heritage, Museums and Collections,Pharmaceuticals and Medical Biotechnology,Retail
Impact Types Cultural

Societal

Policy & public services