How Much is Too Much? Leveraging Existing and Emerging Large-Scale Social Data to Build Robust Evidence-Based Policy for Children in the Digital Age

Lead Research Organisation: University of Oxford
Department Name: Oxford Internet Institute

Abstract

The amount of time British children and adolescents spend on digital technologies has more than doubled in the past decade, and debates about the possible impact of excessive screen use are prominent in the scientific and policy arenas, as well as in the public press. The UK House of Commons, DCMS, and Science and Technology Select Committees, the Royal Society for Paediatrics and Child Health, and the UK's Chief Medical Officer have recently investigated whether the Internet, smartphones, digital games, and social media influence the health of children and young people. These initiatives, however, have all acknowledged the lack of scientific knowledge in this area-which stymies the efforts of UK policy makers to craft effective advice, regulations, and interventions.

This project aims-for the first time-to use existing ESRC datasets to generate the science required to ground policy in this area. We aim to provide policy-makers, parents, teachers, and GPs with the evidence required to understand the role digital technologies play in the lives of British children, and to highlight potential risk and resilience factors that could be the focus of future interventions. We will use ESRC data assets, advanced statistical approaches, and robust open science methodologies to answer three pressing research questions:

1. What risk and resilience factors predispose adolescents to experiencing an effect of digital technology use on their psychological well-being?
2. What are the directional links between digital technology use and psychological well-being, and do the risk factors identified play a mediating role in this?
3. What are the causal pathways linking risk factors, digital technology use and psychological well-being that can inform future intervention?

Answers to these questions are currently elusive, due to the poor data quality and methodological shortcomings that restrain research on technology effects. We will leverage our extensive experience working with large-scale social datasets to examine the general effects of digital technologies and more technology-specific effects (e.g. social media and gaming). We will use machine learning, network modelling, and advanced longitudinal approaches to pinpoint potential risk and resilience factors (e.g. social support, economic deprivation) that alter children's reactions to digital technologies, and which could help guide future technology policy. This will create different profiles of children that we can use to investigate the uses and effects of digital technologies over the longer-term-determining which possible technology effects (e.g. social isolation) are currently unevidenced and over-hyped, and which (e.g. poor sleep) deserve a closer look.

We will thoroughly document our methodology and publish each detailed analysis behind our findings. This transparency is almost entirely missing from the current scientific and policy debate. Making our code freely available will enable other research teams in academia, the charity sector and government to build directly on our work, facilitating a new kind of incremental knowledge transfer which has hitherto been missing, and providing a template for empowering all sectors, including charities, to make more regular use of ESRC data. Crucially, this work will inform how our team continues to advise draft guidance under development at the UK Ministry of Health, ongoing deliberations at the UK Council of Internet Safety, UNICEF, OECD, social enterprises like the ParentZone, and children's charities including Barnardo's-with whom we already have a strong relationship. Our work will also provide a lens to understand the effects of novel technologies such as VR and AI, which will soon be measured by ESRC data. While VR and AI are still under the radar of most academic and policy debate, they will increasingly enter the discourse in coming years.

Planned Impact

Social scientists have not, to date, provided educators, caregivers, or policymakers with the scientific evidence they need to set a coherent agenda for protecting and empowering young people in the digital age. We will engage those working in the media, charity, and government sectors, as well as those in the technology industry over the course of the entire project, leveraging our network of UK and global stakeholders to ensure the project generates maximum domestic and international public and policy impact.

Popular Media. We will work with the London-based Science Media Centre so that each academic paper is the focus of a 'research roundup' or press briefing. These events bring independent experts as well as health and technology editors of major British newspapers together to discuss the work, with the aim of providing high quality science communication to the public. These events will help editors understand what sets robust science grounded on ESRC data apart from work of lower statistical or data quality. We will also write editorials for lay audiences on The Conversation UK and the science blogging network Cosmic Shambles.

UK Charities. We will build on our existing relationships with the children's charities sector: ranging from Barnardo's, The Diana Award, and the PSHE Association to health organisations including the Royal College of Paediatrics and Child Health. All four charities have expressed keen interest in using our findings to shape their advice on digital technology for young people, including those from disadvantaged social and economic backgrounds. Their experience with these populations will help us build statistical models for how these young people develop (or fail to develop) digital resiliencies to online behaviours such as bullying and social stigma.

International Policy. We will work with UNICEF's policy and research lead to draft and publish a white paper digesting our findings for global policy makers, including ministers, civil servants, and frontline services providers across the health, education, and development sectors. This will ensure our work-built on and citing ESRC data-will be widely read and fed directly into policy worldwide.

UK Government. The UK Government is currently undertaking multiple regulatory and statutory consultations aimed at safeguarding young Britons in the digital age. We already work closely with key stakeholders across these groups and have presented preliminary evidence using ESRC data to the evidence board of the UK Council of Internet Safety, as well as parliamentary committees for the DCMS, Department of Health, and directly to the UK's Chief Medical Officer. The CO-I has an ongoing relationship with political think-tanks and organisations like the Ditchley Foundation and the Adam Smith Institute; channels that will help us contact those politicians and policymakers interested in our work.

Technology Industry. The PI will build on his ties with major technology companies and trade groups to transmit project findings directly to the developers of technologies and online platforms used by young people. We will share our findings with the Safety and Well-Being teams at Facebook, the Policy and Well-Being teams at Google, and the Anti-Abuse and Community Management team at the gaming company Super Cell, and advocate data sharing between these technological platforms and UK research councils. We will deepen these existing relationships, meet with the executive team of the UK Interactive Entertainment trade group that represents UK video game developers, and will share our findings at the Annual Game Developers Conference to explore possibilities for data linkages between industry trace data and UK population cohort data. If successful, such a feat would greatly enhance both the scientific value and real-world impact of ESRC data resources for an expansive array of research questions.

Publications

10 25 50
publication icon
Anvari F (2023) Not All Effects Are Indispensable: Psychological Science Requires Verifiable Lines of Reasoning for Whether an Effect Matters. in Perspectives on psychological science : a journal of the Association for Psychological Science

publication icon
IJzerman H (2020) Use caution when applying behavioural science to policy. in Nature human behaviour

publication icon
Johannes N (2021) Video game play is positively correlated with well-being. in Royal Society open science

publication icon
Johannes N (2022) No effect of different types of media on well-being. in Scientific reports

 
Description Our results advance the field in two important ways. First, we show that collaborations with industry partners can be done to high academic standards in an ethical and transparent fashion. Second, we deliver much-needed evidence to policymakers on the link between play and mental health. Our work with secondary data in line with the proposal is ongoing. We have now delivered a series of high impact publications in the Nature Communications, Royal Society Open Science (2x)
Exploitation Route Too early to say yet, but we are hoping our collaborations might leverage the value of secondary data in coordination with industry data. We are also beginning to apply the advanced ML techniques to the data we identified in our proposal. The analysis code underlying our work in this space is currently hosted on github making our published work and forthcoming work fully computationally reproducible.

We have some summary statistics from GitHub and the like. The work has been accessed widely (22k downloads on https://doi.org/10.1098/rsos.220411; 68k downloads for https://doi.org/10.1098/rsos.202049) and widely cited.
Sectors Digital/Communication/Information Technologies (including Software)

 
Description In terms of partnerships our findings have been used to inform forthcoming data frameworks soon to be proposed by DSIT/DCMS and guidelines by the US Surgeon General for technology companies and mental health researchers. Our SDAI work has also been part of the conversation regarding the forthcoming Online Safety Bill and has been repeatedly cited in calls for evidence relating to this and DSIT/DCMS reports and working groups on games and mental health.
First Year Of Impact 2021
Sector Digital/Communication/Information Technologies (including Software),Healthcare
Impact Types Policy & public services

 
Description Aspen Digital - The Aspen Institute roundtable on digital well-being
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Became member of British Academy Public Policy Committee
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
 
Description Living online: the long-term impact on wellbeing (written evidence)
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL https://committees.parliament.uk/work/742/living-online-the-longterm-impact-on-wellbeing/
 
Description Member of the ESRC New and Emerging Forms of Data Leadership Group
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
 
Description Participation in Wellcome Symposium on private-sector data in mental health research
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Jacob's Foundation Isolationtracker study
Amount SFr. 240,000 (CHF)
Organisation Jacobs Foundation 
Sector Charity/Non Profit
Country Switzerland
Start 03/2021 
End 03/2021
 
Description Understanding Video Game Play and Mental Health
Amount £792,893 (GBP)
Funding ID ES/W012626/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 03/2023 
End 02/2025
 
Title Underlying data, code, and materials for Time spent playing video games is unlikely to impact well-being 
Description Data, code, and supplemental materials for https://doi.org/10.1098/rsos.220411. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact These data have been used in other papers. 
URL https://osf.io/fb38n/wiki/home/
 
Title Video game play is positively correlated with well being (data and materials) 
Description This is the OSF project page for "Video game play is positively correlated with well-being". The project holds the raw data, the formr survey, and the source code for data processing and analysis. The source code is on Github, but archived here. For a detailed project description, please go to the Github page (third bullet point below). 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact People have never played more video games and many stakeholders are worried that this activity might be bad for players. So far, research has not had adequate data to test whether these worries are justified and if policymakers should act to regulate video game play time. We attempt to provide much-needed evidence with adequate data. Whereas previous research had to rely on self-reported play behaviour, we collaborated with two games companies, Electronic Arts and Nintendo of America, to obtain players' actual play behaviour. We surveyed players of Plants vs. Zombies: Battle for Neighborville and Animal Crossing: New Horizons for their well-being, motivations, and need satisfaction during play and merged their responses with telemetry data (i.e., logged game play). Contrary to many fears that excessive game time will lead to addiction and poor mental health, we found a small positive relation between game play and well-being. Need satisfaction and motivations during play did not interact with game time but were instead independently related to well-being. Our results advance the field in two important ways. First, we show that collaborations with industry partners can be done to high academic standards in an ethical and transparent fashion. Second, we deliver much-needed evidence to policymakers on the link between play and mental health. 
URL https://osf.io/cjd6z/
 
Title Video game play is positively correlated with well-being (data wrangling and study code) 
Description Data analyses related to "Video game play is positively correlated with well-being" (Johannes, Vuorre, Przybylski, 2021) 
Type Of Material Data handling & control 
Year Produced 2020 
Provided To Others? Yes  
Impact The data and code from this project has been downloaded at least 100 times since time of publication. 
URL http://digital-wellbeing.github.io/gametime/
 
Description Colboration with US Surgeon General 
Organisation Department of Health and Human Services
Country United States 
Sector Public 
PI Contribution I have been working closely with the US Surgeon General and his team to set national guidelines for responsible data sharing access for academics interested in linking large scale social datasets and other forms of health data to online social media and games platforms data. This was reflected earlier in 2021 with the US SG Advisory on Youth Mental Health and in a forthcoming (March/April) call for technology company action.
Collaborator Contribution I have engaged with the US SG, his chief of staff, and their team to help shape the data governance questions and matrerial research asks of large tech firms in the games and social media space. The aim being to connect SDAI learnings with the possibilities afforded by linking industry data to existing NIH, NIMH, and URKI data infrastructure and cohort investments.
Impact The first outcome was the SG's advisory on youth mental health (late 2021). A follow up report with a call to action will be published either later this month or April 2023. The contents of this report are confidential (I have signed an NDA).
Start Year 2021
 
Description Collaboration with DSIT/DCMS CSA on Video Game Research Framework 
Organisation Department for Digital, Culture, Media & Sport
Country United Kingdom 
Sector Public 
PI Contribution Our work on primary and secondary data relating to technology has lead us to work with the DSIT/DCMS CSA and his office to the forthcoming (March 2023) Video Game Research Framework. This document will detail the DSIT/DCMS/Government position on transparent, ethical, and credible research between independent and industry scientists in the UK and the role of URKI and other bodies in facilitating high quality science on technology effects.
Collaborator Contribution I have participated in multiple consultations in person and via video link. I have line edited the draft documents working with the CSA and his team. Our SDAI supported research at the intersection of secondary and primary data on technology and the relevant expertise have been critical.
Impact The full report and framework should be published in March 2023 pending DSIT/DCMS ministerial sign off.
Start Year 2022
 
Description Understanding Society Innovation Panel on Satisfaction 
Organisation MiSOC - Institute for Social and Economic Research (ISER)
Country United Kingdom 
Sector Academic/University 
PI Contribution We have added a new question to the innovation panel of the understanding society project. This will help researchers calibrate the practical significance of technology effects on well-being.
Collaborator Contribution They have added an item that will let us calibrate practically significant effects.
Impact None yet but will principally be scientific in the first instance.
Start Year 2022
 
Description Creating a Path to a New Normal, virtual panel discussion 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Professor Przybylski joins an esteemed panel convened by the Family Online Safety Institute to discuss how the global pandemic and rise of online platforms interact to shape the mental health of young people. Professor Przybylski argues much could be learned from the data generated by online platforms including Roblox, Meta, and Tiktok.
Year(s) Of Engagement Activity 2021
URL https://www.oii.ox.ac.uk/news-events/videos/creating-a-path-to-a-new-normal/
 
Description Digital Technology and Youth youtube discussion 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Professor Przybylski speaks one-on-one with Professor Candice Odgers (UC Irvine) in a virtual fireside chat moderated by Professor John Horton (MIT). Their conversation spans questions of technology effects on young people, role of technology industry and online platforms in sharing data with independent researchers, and the future of social data science investigating global mental health.
Year(s) Of Engagement Activity 2022
URL https://www.oii.ox.ac.uk/news-events/videos/digital-technology-and-youth/
 
Description Video Game Play and Well-Being talk at Oxford Internet Institute 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact OII Director of Research, Professor Andrew Przybylski and Postdoctoral Researcher, Dr Niklas Johannes discuss their recent ground-breaking study which revealed time spent playing video games can be good for your wellbeing.

People have never played more video games and many stakeholders are worried that this activity might be bad for players. So far, research has not had adequate data to test whether these worries are justified and if policymakers should act to regulate video game playtime. Professor Andrew Przybylski with his co-author's Dr Niklas Johannes and Dr Matti Vourre in their report 'Video game play is positively correlated with well-being' attempt to provide much-needed evidence with adequate data. Whereas previous research had to rely on self-reported play behaviour, they collaborated with two games companies, Electronic Arts and Nintendo of America, to obtain players' actual play behaviour. They surveyed players of Plants vs. Zombies: Battle for Neighborville and Animal Crossing: New Horizons for their well-being, motivations, and need satisfaction during play and merged their responses with telemetry data (i.e., logged game play).

Contrary to many fears that excessive game time will lead to addiction and poor mental health, they found a small positive relationship between game play and well-being. Need satisfaction and motivations during play did not interact with game time but were instead independently related to well-being. Their results advance the field in two important ways. First, they show that collaborations with industry partners can be done to high academic standards in an ethical and transparent fashion. Second, they deliver much-needed evidence to policymakers on the link between play and mental health.
Year(s) Of Engagement Activity 2021
URL https://www.oii.ox.ac.uk/news-events/videos/oiis-wednesday-webinar-week-1-professor-andrew-przybylsk...