Towards Understanding the Evolution and Narratives of Online Fringe and Extreme Communities and their Influence on Mainstream Online Communities

The modern Web suffers from various issues. Specifically, some of the most worrying issues are the evolution and dissemination of multiple conspiracy theories and disinformation, extreme and toxic communities targeting specific groups of particular sex, race, and religion. These communities often discuss politically incorrect and controversial topics. Since most mainstream online social networks (OSN) do not allow this content to be shared and discussed on their platforms, people that abide by these opinions tend to create communities in non-mainstream OSNs like 4chan, Voat, Gab, and Parler to communicate with like-minded people. The beliefs of these extremists often drive them to commit unspeakable crimes. A compelling case is Elliot Rodger, who killed six people in a stabbing and shooting spree in California in May 2014. The perpetrator was a member of the online subculture of Incels and was radicalized online. Other examples of radicalization online realized in the real world include the numerous shootings in schools, mosques, and other public spaces. These threats to human life highlight the need to study and understand these tightly-knit communities with the ultimate goal of mitigating the impact they have on the real world.
In addition to these extreme events, the Web buzzes with online chatter about evil cabals and secret organizations that control people and governments - conspiracy theories. Although suspicion has always been with the human race powered by our evolutionary drive to survive, it becomes extreme in the case of conspiracy theories and leads to apophenia. Apophenia refers to the notion of connecting previously unconnected ideas and theories with events without proof. Conspiracy theory adherents share their paranoid, unrealistic ideas that may cause panic, harm democracies, and gather other unsuspecting followers. Examples of dangerous conspiracy theories surround the COVID-19 global pandemic; pizzagate, which frames the 2016 US presidential candidate Hilary Clinton as the mastermind of global pedophile rings; and QAnon, a boldly defined super conspiracy theory, which integrates many conspiracy theories. Many studies and news press outlets explain the dangers and threats conspiracy theories pose to democracies and the general public. More specifically, conspiracy theories, like QAnon, proved to be extremely dangerous towards democracies as government officials and media often get involved in starting or promoting them to benefit their political agendas and interests. A prime example is January 6, 2020, when a pro-Qanon and pro-Trump mob stormed the US Capitol in a failed attempt to reinstate Donald Trump as the president of the US. Specifically, the FBI describes the QAnon movement as a domestic terror threat and its followers as "domestic extremists."
To this end, we focus our research interests on understanding and characterizing toxic, extreme, and fringe communities online. We are interested in analyzing and describing the content these communities create and how it is disseminated across the Web. Thus, shedding light on how unsuspecting communities and individuals get affected when exposed to such content, targeted raid attacks, and disinformation and conspiracy theories. We intend to analyze the content these communities create and share on mainstream and non-mainstream OSNs to investigate how they affect each other and how users are linked to various communities on different platforms. Notably, we intend to deploy state-of-the-art tools to characterize these communities: topic, sentiment, and stance detection, hoaxes processes, named entity recognition, word embeddings, latent analysis, toxicity, and advanced machine and deep learning and image processing techniques. We aim to understand and characterize the content these communities create, how they penetrate other online unsuspecting communities, and how they influence them. Last, we target to investigate possible mitigation mechanisms.

The EPSRC Centre for Doctoral Training in Cybersecurity will train over 55 experts in multi-disciplinary aspects of cybersecurity, from engineering to crime science and public policy.

Short term impacts are associated with the research outputs of the 55+ research projects that will be undertaken as part of the doctoral studies of CDT students. Each project will tackle an important cybersecurity problem, propose and evaluate solutions, interventions and policy options. Students will publish those in international peer-reviewed journals, but also disseminate those through blog posts and material geared towards decision makers and experts in adjacent fields. Through industry placements relating to their projects, all students will have the opportunity to implement and evaluate their ideas within real-world organizations, to achieve short term impact in solving cybersecurity problems.

In the longer term graduates of the CDT will assume leading positions within industry, goverment, law enforcement, the third sector and academia to increase the capacity of the UK in being a leader in cybersecurity. From those leadership positions they will assess options and formulate effective interventions to tackle cybercrime, secure the UK's infrastructure, establish norms of cooperation between industries and government to secure IT systems, and become leading researcher and scholars further increasing the UK's capacity in cybersecurity in the years to come. The last impact is likely to be significant give that currently many higher education training programs do not have capacity to provide cybersecurity training at undergraduate or graduate levels, particularly in non-technical fields.

The full details of our plan to achieve impact can be found in the "Pathways to Impact" document.