XAIvsDisinfo: eXplainable AI Methods for Categorisation and Analysis of COVID-19 Vaccine Disinformation and Online Debates

Lead Research Organisation: University of Sheffield
Department Name: Computer Science

Abstract

UK vaccination rates are in decline and experts believe that vaccine disinformation, widely spread
in social media, may be one of the reasons. Recent surveys have established that vaccine
disinformation is impacting negatively citizen trust in COVID-19 vaccination specifically. As a
response, the UK Government agreed with Twitter, Facebook, and YouTube measures to limit the
spread of disinformation. However, simply removing disinformation from platforms is not enough,
as the government also needs to monitor and respond to the concerns of vaccine hesitant citizens.
Moreover, manual detection and tracking of disinformation, as currently practiced by many
journalists, is infeasible, given the scale of social media.

XAIvsDinfo aims to address these gaps through novel research on explainable AI-based models for
large-scale analysis of vaccine disinformation. Specifically, vaccine disinformation will be classified
automatically into the six narrative types defined by First Draft. A second model will categorise
vaccine statements as pro-vaccine, anti-vaccine, vaccine-hesitant, or other.
We will investigate explainable machine learning approaches that are human interpretable: both
in detecting errors and weaknesses of the models and in providing human-readable explanations
of the models' decisions.

XAIvsDisinfo will also create two new multi-platform datasets and organise a new community
research challenge on cross-platform analysis of vaccine disinformation, as follow-up from our
RumourEval one.

Our XAI models and tools will be integrated into the open-source InVID-WeVerify plugin, for take
up by journalists and fact-checkers. The project outputs will also contribute to evidence-based
policy activities by the UK government on improving citizen perception of COVID-19 vaccines.
 
Description We have studied the spread of vaccine narratives online, including vaccine hesitancy. The research is still ongoing and involved journalists and data scientists, as well as AI researchers developing intelligent methods for detection and classification of vaccine hesitant and anti-vaccine posts, as well as categorisation of vaccine narratives into 6 topical categories.
Exploitation Route The award is still ongoing, so this will be addressed at a later stage.
Sectors Digital/Communication/Information Technologies (including Software),Government, Democracy and Justice

 
Description Provided input to DCMS to help shape up the Online Harms bill. Used by journalists at First Draft for their work on vaccine narratives.
First Year Of Impact 2021
Sector Digital/Communication/Information Technologies (including Software),Government, Democracy and Justice
Impact Types Societal,Policy & public services

 
Description Participation in DCMS task force and virtual round table on tackling COVID-19 and vaccine misinformation
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
Impact Learnings from the task force have been used to inform the Online Harms bill going through Parliament.
URL https://www.gov.uk/government/news/social-media-giants-agree-package-of-measures-with-uk-government-...
 
Description Participation in the DCMS College of Experts
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
Impact Research on online disinformation and online abuse has played a key role in informing the latest Online Harms bill which is going to be put through Parliament.
 
Description vera.ai: Verification Assisted by Artificial Intelligence
Amount £900,000 (GBP)
Funding ID https://www.veraai.eu/home 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 09/2022 
End 09/2025
 
Title COVID-19 Vaccine Narrative Categoriser 
Description A machine learning classifier trained to categorise COVID-19 vaccine text into 6 categories. These are: Liberty/Freedom Development Provision and Access Safety Efficacy and Necessity Politics and Economics Conspiracy Morality Religiosity and Ethics 
Type Of Material Improvements to research infrastructure 
Year Produced 2021 
Provided To Others? Yes  
Impact Used by First Draft in their research on vaccine misinformation. 
URL https://cloud.gate.ac.uk/shopfront/displayItem/covid19-vaccine
 
Title Text Classification Web Annotation Tool 
Description A web application which enables a team of researchers to jointly annotate vaccine narratives for vaccine hesitancy and narrative category, for the purpose of training machine learning models. 
Type Of Technology Webtool/Application 
Year Produced 2021 
Impact The tool is still being developed and the award plans to use to for the development and release of two datasets.