How Much is Too Much? Leveraging Existing and Emerging Large-Scale Social Data to Build Robust Evidence-Based Policy for Children in the Digital Age
Lead Research Organisation:
University of Oxford
Department Name: Oxford Internet Institute
Abstract
The amount of time British children and adolescents spend on digital technologies has more than doubled in the past decade, and debates about the possible impact of excessive screen use are prominent in the scientific and policy arenas, as well as in the public press. The UK House of Commons, DCMS, and Science and Technology Select Committees, the Royal Society for Paediatrics and Child Health, and the UK's Chief Medical Officer have recently investigated whether the Internet, smartphones, digital games, and social media influence the health of children and young people. These initiatives, however, have all acknowledged the lack of scientific knowledge in this area-which stymies the efforts of UK policy makers to craft effective advice, regulations, and interventions.
This project aims-for the first time-to use existing ESRC datasets to generate the science required to ground policy in this area. We aim to provide policy-makers, parents, teachers, and GPs with the evidence required to understand the role digital technologies play in the lives of British children, and to highlight potential risk and resilience factors that could be the focus of future interventions. We will use ESRC data assets, advanced statistical approaches, and robust open science methodologies to answer three pressing research questions:
1. What risk and resilience factors predispose adolescents to experiencing an effect of digital technology use on their psychological well-being?
2. What are the directional links between digital technology use and psychological well-being, and do the risk factors identified play a mediating role in this?
3. What are the causal pathways linking risk factors, digital technology use and psychological well-being that can inform future intervention?
Answers to these questions are currently elusive, due to the poor data quality and methodological shortcomings that restrain research on technology effects. We will leverage our extensive experience working with large-scale social datasets to examine the general effects of digital technologies and more technology-specific effects (e.g. social media and gaming). We will use machine learning, network modelling, and advanced longitudinal approaches to pinpoint potential risk and resilience factors (e.g. social support, economic deprivation) that alter children's reactions to digital technologies, and which could help guide future technology policy. This will create different profiles of children that we can use to investigate the uses and effects of digital technologies over the longer-term-determining which possible technology effects (e.g. social isolation) are currently unevidenced and over-hyped, and which (e.g. poor sleep) deserve a closer look.
We will thoroughly document our methodology and publish each detailed analysis behind our findings. This transparency is almost entirely missing from the current scientific and policy debate. Making our code freely available will enable other research teams in academia, the charity sector and government to build directly on our work, facilitating a new kind of incremental knowledge transfer which has hitherto been missing, and providing a template for empowering all sectors, including charities, to make more regular use of ESRC data. Crucially, this work will inform how our team continues to advise draft guidance under development at the UK Ministry of Health, ongoing deliberations at the UK Council of Internet Safety, UNICEF, OECD, social enterprises like the ParentZone, and children's charities including Barnardo's-with whom we already have a strong relationship. Our work will also provide a lens to understand the effects of novel technologies such as VR and AI, which will soon be measured by ESRC data. While VR and AI are still under the radar of most academic and policy debate, they will increasingly enter the discourse in coming years.
This project aims-for the first time-to use existing ESRC datasets to generate the science required to ground policy in this area. We aim to provide policy-makers, parents, teachers, and GPs with the evidence required to understand the role digital technologies play in the lives of British children, and to highlight potential risk and resilience factors that could be the focus of future interventions. We will use ESRC data assets, advanced statistical approaches, and robust open science methodologies to answer three pressing research questions:
1. What risk and resilience factors predispose adolescents to experiencing an effect of digital technology use on their psychological well-being?
2. What are the directional links between digital technology use and psychological well-being, and do the risk factors identified play a mediating role in this?
3. What are the causal pathways linking risk factors, digital technology use and psychological well-being that can inform future intervention?
Answers to these questions are currently elusive, due to the poor data quality and methodological shortcomings that restrain research on technology effects. We will leverage our extensive experience working with large-scale social datasets to examine the general effects of digital technologies and more technology-specific effects (e.g. social media and gaming). We will use machine learning, network modelling, and advanced longitudinal approaches to pinpoint potential risk and resilience factors (e.g. social support, economic deprivation) that alter children's reactions to digital technologies, and which could help guide future technology policy. This will create different profiles of children that we can use to investigate the uses and effects of digital technologies over the longer-term-determining which possible technology effects (e.g. social isolation) are currently unevidenced and over-hyped, and which (e.g. poor sleep) deserve a closer look.
We will thoroughly document our methodology and publish each detailed analysis behind our findings. This transparency is almost entirely missing from the current scientific and policy debate. Making our code freely available will enable other research teams in academia, the charity sector and government to build directly on our work, facilitating a new kind of incremental knowledge transfer which has hitherto been missing, and providing a template for empowering all sectors, including charities, to make more regular use of ESRC data. Crucially, this work will inform how our team continues to advise draft guidance under development at the UK Ministry of Health, ongoing deliberations at the UK Council of Internet Safety, UNICEF, OECD, social enterprises like the ParentZone, and children's charities including Barnardo's-with whom we already have a strong relationship. Our work will also provide a lens to understand the effects of novel technologies such as VR and AI, which will soon be measured by ESRC data. While VR and AI are still under the radar of most academic and policy debate, they will increasingly enter the discourse in coming years.
Planned Impact
Social scientists have not, to date, provided educators, caregivers, or policymakers with the scientific evidence they need to set a coherent agenda for protecting and empowering young people in the digital age. We will engage those working in the media, charity, and government sectors, as well as those in the technology industry over the course of the entire project, leveraging our network of UK and global stakeholders to ensure the project generates maximum domestic and international public and policy impact.
Popular Media. We will work with the London-based Science Media Centre so that each academic paper is the focus of a 'research roundup' or press briefing. These events bring independent experts as well as health and technology editors of major British newspapers together to discuss the work, with the aim of providing high quality science communication to the public. These events will help editors understand what sets robust science grounded on ESRC data apart from work of lower statistical or data quality. We will also write editorials for lay audiences on The Conversation UK and the science blogging network Cosmic Shambles.
UK Charities. We will build on our existing relationships with the children's charities sector: ranging from Barnardo's, The Diana Award, and the PSHE Association to health organisations including the Royal College of Paediatrics and Child Health. All four charities have expressed keen interest in using our findings to shape their advice on digital technology for young people, including those from disadvantaged social and economic backgrounds. Their experience with these populations will help us build statistical models for how these young people develop (or fail to develop) digital resiliencies to online behaviours such as bullying and social stigma.
International Policy. We will work with UNICEF's policy and research lead to draft and publish a white paper digesting our findings for global policy makers, including ministers, civil servants, and frontline services providers across the health, education, and development sectors. This will ensure our work-built on and citing ESRC data-will be widely read and fed directly into policy worldwide.
UK Government. The UK Government is currently undertaking multiple regulatory and statutory consultations aimed at safeguarding young Britons in the digital age. We already work closely with key stakeholders across these groups and have presented preliminary evidence using ESRC data to the evidence board of the UK Council of Internet Safety, as well as parliamentary committees for the DCMS, Department of Health, and directly to the UK's Chief Medical Officer. The CO-I has an ongoing relationship with political think-tanks and organisations like the Ditchley Foundation and the Adam Smith Institute; channels that will help us contact those politicians and policymakers interested in our work.
Technology Industry. The PI will build on his ties with major technology companies and trade groups to transmit project findings directly to the developers of technologies and online platforms used by young people. We will share our findings with the Safety and Well-Being teams at Facebook, the Policy and Well-Being teams at Google, and the Anti-Abuse and Community Management team at the gaming company Super Cell, and advocate data sharing between these technological platforms and UK research councils. We will deepen these existing relationships, meet with the executive team of the UK Interactive Entertainment trade group that represents UK video game developers, and will share our findings at the Annual Game Developers Conference to explore possibilities for data linkages between industry trace data and UK population cohort data. If successful, such a feat would greatly enhance both the scientific value and real-world impact of ESRC data resources for an expansive array of research questions.
Popular Media. We will work with the London-based Science Media Centre so that each academic paper is the focus of a 'research roundup' or press briefing. These events bring independent experts as well as health and technology editors of major British newspapers together to discuss the work, with the aim of providing high quality science communication to the public. These events will help editors understand what sets robust science grounded on ESRC data apart from work of lower statistical or data quality. We will also write editorials for lay audiences on The Conversation UK and the science blogging network Cosmic Shambles.
UK Charities. We will build on our existing relationships with the children's charities sector: ranging from Barnardo's, The Diana Award, and the PSHE Association to health organisations including the Royal College of Paediatrics and Child Health. All four charities have expressed keen interest in using our findings to shape their advice on digital technology for young people, including those from disadvantaged social and economic backgrounds. Their experience with these populations will help us build statistical models for how these young people develop (or fail to develop) digital resiliencies to online behaviours such as bullying and social stigma.
International Policy. We will work with UNICEF's policy and research lead to draft and publish a white paper digesting our findings for global policy makers, including ministers, civil servants, and frontline services providers across the health, education, and development sectors. This will ensure our work-built on and citing ESRC data-will be widely read and fed directly into policy worldwide.
UK Government. The UK Government is currently undertaking multiple regulatory and statutory consultations aimed at safeguarding young Britons in the digital age. We already work closely with key stakeholders across these groups and have presented preliminary evidence using ESRC data to the evidence board of the UK Council of Internet Safety, as well as parliamentary committees for the DCMS, Department of Health, and directly to the UK's Chief Medical Officer. The CO-I has an ongoing relationship with political think-tanks and organisations like the Ditchley Foundation and the Adam Smith Institute; channels that will help us contact those politicians and policymakers interested in our work.
Technology Industry. The PI will build on his ties with major technology companies and trade groups to transmit project findings directly to the developers of technologies and online platforms used by young people. We will share our findings with the Safety and Well-Being teams at Facebook, the Policy and Well-Being teams at Google, and the Anti-Abuse and Community Management team at the gaming company Super Cell, and advocate data sharing between these technological platforms and UK research councils. We will deepen these existing relationships, meet with the executive team of the UK Interactive Entertainment trade group that represents UK video game developers, and will share our findings at the Annual Game Developers Conference to explore possibilities for data linkages between industry trace data and UK population cohort data. If successful, such a feat would greatly enhance both the scientific value and real-world impact of ESRC data resources for an expansive array of research questions.
Publications
Anvari F
(2023)
Not All Effects Are Indispensable: Psychological Science Requires Verifiable Lines of Reasoning for Whether an Effect Matters.
in Perspectives on psychological science : a journal of the Association for Psychological Science
IJzerman H
(2020)
Use caution when applying behavioural science to policy.
in Nature human behaviour
Johannes N
(2022)
Time Spent Playing Two Online Shooters Has No Measurable Effect on Aggressive Affect
in Collabra: Psychology
Johannes N
(2022)
No effect of different types of media on well-being.
in Scientific reports
Johannes N
(2021)
Supplemental Material for Objective, subjective, and accurate reporting of social media use: No evidence that daily social media use correlates with personality traits, motivational states, or well-being.
in Technology, Mind, and Behavior
Johannes N
(2021)
Video game play is positively correlated with well-being.
in Royal Society open science
Johannes N
(2020)
Video game play is positively correlated with well-being
Johannes N
(2021)
No effect of different types of media on well-being
Description | Our results advance the field in two important ways. First, we show that collaborations with industry partners can be done to high academic standards in an ethical and transparent fashion. Second, we deliver much-needed evidence to policymakers on the link between play and mental health. Our work with secondary data in line with the proposal is ongoing. We have now delivered a series of high impact publications in the Nature Communications, Royal Society Open Science (2x). Our work has recently made a larger impact in both the USA (working with the office of the US Surgeon General) and here in the UK (DCMS Video Games Research Framework). We are continuing to work to use a combination of existing secondary data and novel data from large technology companies to gain insights into how technology influences influences users. |
Exploitation Route | We are hoping our collaborations might leverage the value of secondary data in coordination with industry data. We are also beginning to apply the advanced ML techniques to the data we identified in our proposal. The analysis code underlying our work in this space is currently hosted on GitHub making our published work and forthcoming work fully computationally reproducible. We have some summary statistics from GitHub and the like. The work has been accessed widely (22k downloads on https://doi.org/10.1098/rsos.220411; 68k downloads for https://doi.org/10.1098/rsos.202049) and widely cited. |
Sectors | Digital/Communication/Information Technologies (including Software) |
Description | Our work has significantly influenced the development of policies and frameworks regarding the relationship between digital technologies and mental health. The UK's Department for Digital, Culture, Media and Sport (DCMS) recently published the Video Games Research Framework, marking a significant milestone in understanding video games' societal and economic impacts. This framework sets the foundation for future high-quality, independent research by offering guidance on research priorities, methodologies, and data handling practices. It emphasises inclusivity, openness, transparency, and independence in research, aiming to foster collaboration across academia, industry, and the public to enhance our understanding of video games' multifaceted roles. In the US, our consultations contributed to the Surgeon General's advisory on youth mental health and social media in May 2023. This advisory highlights the nuanced effects of social media on adolescents, recognizing the complexity of its impacts on mental health. It underscores the importance of further research to understand these effects fully and to inform interventions that promote healthier digital environments for young people. The DCMS Video Games Research Framework and the US Surgeon General's advisory represent critical steps towards a more nuanced understanding of how digital technologies affect our lives. They reflect our commitment to evidence-based policy and the need for a balanced approach that recognizes the potential benefits and challenges of digital engagement. Our ongoing work with DSIT/DCMS, in shaping the Online Safety Bill and contributing to guidelines for technology companies and mental health researchers, demonstrates our pivotal role in shaping the conversation around technology, mental health, and society. |
First Year Of Impact | 2020 |
Sector | Digital/Communication/Information Technologies (including Software),Healthcare |
Impact Types | Policy & public services |
Description | Aspen Digital - The Aspen Institute roundtable on digital well-being |
Geographic Reach | Multiple continents/international |
Policy Influence Type | Participation in a guidance/advisory committee |
Description | Became member of British Academy Public Policy Committee |
Geographic Reach | National |
Policy Influence Type | Membership of a guideline committee |
Description | Living online: the long-term impact on wellbeing (written evidence) |
Geographic Reach | National |
Policy Influence Type | Contribution to a national consultation/review |
URL | https://committees.parliament.uk/work/742/living-online-the-longterm-impact-on-wellbeing/ |
Description | Member of the ESRC New and Emerging Forms of Data Leadership Group |
Geographic Reach | National |
Policy Influence Type | Membership of a guideline committee |
Description | Parliamentary Inquiry about Impacts of Living Online, House of Lords COVID Committee |
Geographic Reach | National |
Policy Influence Type | Contribution to a national consultation/review |
Description | Participation in Wellcome Symposium on private-sector data in mental health research |
Geographic Reach | Multiple continents/international |
Policy Influence Type | Participation in a guidance/advisory committee |
Description | Understanding Video Game Play and Mental Health |
Amount | £792,893 (GBP) |
Funding ID | ES/W012626/1 |
Organisation | Economic and Social Research Council |
Sector | Public |
Country | United Kingdom |
Start | 03/2023 |
End | 02/2025 |
Title | Underlying data, code, and materials for Time spent playing video games is unlikely to impact well-being |
Description | Data, code, and supplemental materials for https://doi.org/10.1098/rsos.220411. |
Type Of Material | Database/Collection of data |
Year Produced | 2022 |
Provided To Others? | Yes |
Impact | These data have been used in other papers. |
URL | https://osf.io/fb38n/wiki/home/ |
Title | Video game play is positively correlated with well being (data and materials) |
Description | This is the OSF project page for "Video game play is positively correlated with well-being". The project holds the raw data, the formr survey, and the source code for data processing and analysis. The source code is on Github, but archived here. For a detailed project description, please go to the Github page (third bullet point below). |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | People have never played more video games and many stakeholders are worried that this activity might be bad for players. So far, research has not had adequate data to test whether these worries are justified and if policymakers should act to regulate video game play time. We attempt to provide much-needed evidence with adequate data. Whereas previous research had to rely on self-reported play behaviour, we collaborated with two games companies, Electronic Arts and Nintendo of America, to obtain players' actual play behaviour. We surveyed players of Plants vs. Zombies: Battle for Neighborville and Animal Crossing: New Horizons for their well-being, motivations, and need satisfaction during play and merged their responses with telemetry data (i.e., logged game play). Contrary to many fears that excessive game time will lead to addiction and poor mental health, we found a small positive relation between game play and well-being. Need satisfaction and motivations during play did not interact with game time but were instead independently related to well-being. Our results advance the field in two important ways. First, we show that collaborations with industry partners can be done to high academic standards in an ethical and transparent fashion. Second, we deliver much-needed evidence to policymakers on the link between play and mental health. |
URL | https://osf.io/cjd6z/ |
Title | Video game play is positively correlated with well-being (data wrangling and study code) |
Description | Data analyses related to "Video game play is positively correlated with well-being" (Johannes, Vuorre, Przybylski, 2021) |
Type Of Material | Data handling & control |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | The data and code from this project has been downloaded at least 100 times since time of publication. |
URL | http://digital-wellbeing.github.io/gametime/ |
Description | Colboration with US Surgeon General |
Organisation | Department of Health and Human Services |
Country | United States |
Sector | Public |
PI Contribution | I have been working closely with the US Surgeon General and his team to set national guidelines for responsible data sharing access for academics interested in linking large scale social datasets and other forms of health data to online social media and games platforms data. This was reflected earlier in 2021 with the US SG Advisory on Youth Mental Health and in a forthcoming (March/April) call for technology company action. |
Collaborator Contribution | I have engaged with the US SG, his chief of staff, and their team to help shape the data governance questions and matrerial research asks of large tech firms in the games and social media space. The aim being to connect SDAI learnings with the possibilities afforded by linking industry data to existing NIH, NIMH, and URKI data infrastructure and cohort investments. |
Impact | The first outcome was the SG's advisory on youth mental health (late 2021). A follow up report with a call to action will be published either later this month or April 2023. The contents of this report are confidential (I have signed an NDA). |
Start Year | 2021 |
Description | Collaboration with DSIT/DCMS CSA on Video Game Research Framework |
Organisation | Department for Digital, Culture, Media & Sport |
Country | United Kingdom |
Sector | Public |
PI Contribution | Our work on primary and secondary data relating to technology has lead us to work with the DSIT/DCMS CSA and his office to the forthcoming (March 2023) Video Game Research Framework. This document will detail the DSIT/DCMS/Government position on transparent, ethical, and credible research between independent and industry scientists in the UK and the role of URKI and other bodies in facilitating high quality science on technology effects. |
Collaborator Contribution | I have participated in multiple consultations in person and via video link. I have line edited the draft documents working with the CSA and his team. Our SDAI supported research at the intersection of secondary and primary data on technology and the relevant expertise have been critical. |
Impact | The full report and framework should be published in March 2023 pending DSIT/DCMS ministerial sign off. |
Start Year | 2022 |
Description | Understanding Society Innovation Panel on Satisfaction |
Organisation | MiSOC - Institute for Social and Economic Research (ISER) |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | We have added a new question to the innovation panel of the understanding society project. This will help researchers calibrate the practical significance of technology effects on well-being. |
Collaborator Contribution | They have added an item that will let us calibrate practically significant effects. |
Impact | None yet but will principally be scientific in the first instance. |
Start Year | 2022 |
Description | 2023 FOSI European Forum - Regulating and Innovating Online Safety |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Around 250 working in EU/UK online internet and games safety and policy professionals. The panel was focused on Wellbeing in a New Digital Era, featured the industry perspective from both Tami Bhaumik of Roblox and Mindy Brooks of Google, the academic point of view from Andy Przybylski of OII, and the non-profit perspective from Áine Lynch of Ireland's National Parents Council. The discussion featured digital wellbeing best practices when thinking about the use of social media, immersive technologies, and AI. Panelists also shared their expertise and discussed ways in which both parents and kids can feel more empowered to take control of their relationship with technology in a healthy and balanced way. |
Year(s) Of Engagement Activity | 2023 |
URL | https://www.fosi.org/events/2023-european-forum#media |
Description | Creating a Path to a New Normal, virtual panel discussion |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Media (as a channel to the public) |
Results and Impact | Professor Przybylski joins an esteemed panel convened by the Family Online Safety Institute to discuss how the global pandemic and rise of online platforms interact to shape the mental health of young people. Professor Przybylski argues much could be learned from the data generated by online platforms including Roblox, Meta, and Tiktok. |
Year(s) Of Engagement Activity | 2021 |
URL | https://www.oii.ox.ac.uk/news-events/videos/creating-a-path-to-a-new-normal/ |
Description | Digital Technology and Youth youtube discussion |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Media (as a channel to the public) |
Results and Impact | Professor Przybylski speaks one-on-one with Professor Candice Odgers (UC Irvine) in a virtual fireside chat moderated by Professor John Horton (MIT). Their conversation spans questions of technology effects on young people, role of technology industry and online platforms in sharing data with independent researchers, and the future of social data science investigating global mental health. |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.oii.ox.ac.uk/news-events/videos/digital-technology-and-youth/ |
Description | Video Game Play and Well-Being talk at Oxford Internet Institute |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | OII Director of Research, Professor Andrew Przybylski and Postdoctoral Researcher, Dr Niklas Johannes discuss their recent ground-breaking study which revealed time spent playing video games can be good for your wellbeing. People have never played more video games and many stakeholders are worried that this activity might be bad for players. So far, research has not had adequate data to test whether these worries are justified and if policymakers should act to regulate video game playtime. Professor Andrew Przybylski with his co-author's Dr Niklas Johannes and Dr Matti Vourre in their report 'Video game play is positively correlated with well-being' attempt to provide much-needed evidence with adequate data. Whereas previous research had to rely on self-reported play behaviour, they collaborated with two games companies, Electronic Arts and Nintendo of America, to obtain players' actual play behaviour. They surveyed players of Plants vs. Zombies: Battle for Neighborville and Animal Crossing: New Horizons for their well-being, motivations, and need satisfaction during play and merged their responses with telemetry data (i.e., logged game play). Contrary to many fears that excessive game time will lead to addiction and poor mental health, they found a small positive relationship between game play and well-being. Need satisfaction and motivations during play did not interact with game time but were instead independently related to well-being. Their results advance the field in two important ways. First, they show that collaborations with industry partners can be done to high academic standards in an ethical and transparent fashion. Second, they deliver much-needed evidence to policymakers on the link between play and mental health. |
Year(s) Of Engagement Activity | 2021 |
URL | https://www.oii.ox.ac.uk/news-events/videos/oiis-wednesday-webinar-week-1-professor-andrew-przybylsk... |