What does Artificial Intelligence Mean for the Future of Democratic Society? Examining the societal impact of AI and whether human rights can respond

Lead Research Organisation: University of Essex
Department Name: Human Rights Centre

Abstract

This research examines the impacts that States' use of artificial intelligence (AI) in decision making processes has on how individuals and societies evolve and develop and what this means for democratic society. Understanding these impacts is essential so that effective guidance can be developed that allows States to take advantage of the significant potential inherent in AI, while protecting those factors essential to a functioning democracy and preventing human rights harm.
AI has the power to radically transform State activity, redefining our understanding of how a State functions and delivers services, and how it interacts with its citizens. A key development in this regard is the incorporation of AI tools into State decision-making processes. To be effective, these tools are dependent upon significantly increased surveillance by State and non-State actors: the data obtained through surveillance is subject to analysis using AI in order to make individually-tailored decisions. This represents a step-change in terms of the level of insight the State has into individuals' day-to-day lives, and their ability to use this information to determine that individual's life choices. This may exert a profound impact on how individuals, and society as a whole, develops. Will individuals be afraid to experiment, or to seek out alternative ideas or ways of life, because they are worried that they will be categorised on this basis and their future life choices restricted? Will this in turn lead to the stagnation of democratic society?
AI has enormous potential. It can be used to transform how a State delivers services, and if used appropriately can make a real contribution to the development of society, and the protection of human rights. However, it is imperative that the broader impacts of AI on individuals and society be understood before AI becomes pervasive in decision-making processes, so that appropriate regulatory and policy responses can be developed, and human rights protections ensures.
This research focuses on the inadvertent, or unintentional, impacts associated with State adoption of AI technologies. There is, of course, clear potential for AI to be misused for repressive purposes. Of interest here, however, is States' use of AI when deployed in pursuit of legitimate objectives. The unintended consequences associated with States' uses of AI under these circumstances may be less visible but equally dramatic.
Human rights law provides the framework underpinning research. Although it must be reconceptualised to respond to the digital age (a key research objective) it provides the most effective means of identifying harm, resolving competing interests, and providing regulatory guidance.
The principal objective underpinning this inter-disciplinary research is the development of future-oriented human rights approaches to regulate States' use of AI in decision making processes, and to ensure that AI serves, rather than undermines, societal objectives. To do so will require in-depth research across law, human rights, philosophy, and sociology. Initial research will investigate factors essential to individual and societal development, how these relate to democratic functioning, and how they are impacted by States' use of AI. Human rights law itself must then be re-conceptualised, to ensure that it is capable of engaging with these factors, and protecting them in the digital age.
State agencies are beginning to incorporate AI technologies, and the utilisation of AI will increase exponentially over the coming years. Surveillance and AI-assisted analytical tools are deployed across all areas of State activity, from social welfare, to child protection, and healthcare. To examine the democratic effects where they are most visible in the short term, however, research will focus primarily on State activity related to law enforcement and counter-terrorism, examining the use of AI by police and intelligence agencies.

Planned Impact

The principal objective underpinning this research project is the production of world class research to inform human rights compliant policy and regulation with respect to the development and deployment of artificial intelligence (AI) tools by States. State agencies are just beginning to incorporate AI technologies, and it is expected that the utilisation of AI by public authorities will increase exponentially over the coming years. Indeed, AI has been put at the centre of the UK government's development and investment plans (see establishment of Centre for Data Ethics and Innovation, or Digital Catapult), reflecting global trends in this direction.
To-date, however, research has not addressed the effects that the use of AI in State decision-making processes has on how individuals and societies develop and evolve, and how this may affect the effective functioning of democratic society. This gap is significant as understanding these impacts is essential in order to identify if, and how, AI technologies may be appropriately deployed in decision-making contexts. This research intends to contribute directly to the development of policy guidance and regulation in this area.
Three principal pathways to impact will be pursued, addressing the United Nations, domestic State actors, and general awareness raising. As this is an emerging area of research, it is important that research findings are engaged with and adopted by standard setting bodies, or those with particular influence in the area, so that the frameworks developed during the research project can be taken up and receive authoritative endorsement.
United Nations
UN actors play a decisive role in developing human rights law, and the United Nations is arguably the most influential forum internationally in terms of informing international and domestic human rights approaches. This impact pathway is a means to ensure that research outputs inform mainstream human rights approaches. Engagement serves two purposes. It provides an opportunity to (a) influence thinking and policy development at a global level, and (b) contribute to the development of soft-law standards, capable of influencing domestic policy, legislation, and court decisions.
The applicant has pre-existing contacts with relevant actors, and a number of strategies for engagement will be employed. First, consultations will be held with the Office of the High Commissioner for Human Rights and relevant Special Procedures to discuss the research agenda, and to identify areas of mutual interest. Second, broad consultations will be held with all actors to identify areas of interest that would benefit from in-depth discussion. Workshops will then be organised in these areas, with outputs tailored to participants needs. Third, submissions will be made to various bodies, in response to calls in order to contribute directly to standard setting.
Domestic State actors
State agencies operating at the domestic level play a clear role in both the development and operationalisation of policy. The flexibility built into this funding scheme offers an opportunity to engage with these actors, and to tailor research to their specific needs thereby informing their practice and securing impact. This project provides a clear added value to State actors' work. It allows for academically rigorous research, provided by an independent third party, without placing additional strain on the actor's resources.
General Awareness Raising
The focus here is on distilling research findings into accessible public facing outputs across a variety of different formats. Using existing contacts, leading non-governmental organisations active in this area will be engaged. Similar to the first two pathways, efforts will be made to ensure that research can be tailored to these actors' needs, in order to inform their work. Outputs will also be posted regularly to the project website in a variety of non-academic forms tailored for different actors.

Publications

10 25 50
publication icon
Murray D (2021) Effective Oversight of Large-Scale Surveillance Activities: A Human Rights Perspective in Journal of National Security Law & Policy

 
Description To-date, work has focused on: understanding the impact of surveillance, and developing a human rights-based approached to the use of AI tools. Significant progress has been made in this regard, and part of the award's overall objectives may be considered met. Of course, we are only one year in and there is a lot of work left to do.

In terms of understanding the impact of surveillance, a major collaboration with the Centre for Constitutional Rights was undertaken. This involved interviewing a large number of individuals who have been subject to surveillance, in order to document their experiences, and how it affected their personal and professional (including activist) lives. This complements other research undertaken in Uganda and Zimbabwe. The impact of surveillance is not fully understood, and it has been difficult to quantify. However, by using a large-scale qualitative approach we adopted a methodology whereby common themes or consequences can be identified, helping to provide a more robust evidence base detailing the 'chilling effect' of surveillance. This work has never been done before and stands to make a significant impact and a novel contribution to academic thought. The findings indicate that surveillance exerts very similar impacts across different contexts, and gives rise to significant harm, particularly with respect to the right to freedom of expression, and the right to freedom of assembly. This work is being developed into a range of different outputs. Two articles, focusing on Uganda and Zimbabwe have been accepted for publication and others are in preparation. Discussions have been entered into with other entities to take this work forward, and the UN Special Rapporteur on Freedom of Association and Assembly has expressed an active interest in developing the work.

This work has produced two academic articles so far. One piece has been accepted for publication in Big Data & Society, and another has been re-submitted to the Journal of Human Rights Practice following minor revisions (final decision pending).

Work on developing a human rights-based approach to AI is also progressing. A key finding to-date has been that different entities, including government agencies (confidential) and national human rights commissions are interested in engaging with this work, and taking it forward. An initial framework has been developed, and this will be taken forward. A collaboration with a key government oversight agency was undertaken as part of this strand of work.
Exploitation Route The work on the chilling effect of surveillance stands to be taken up both academically and by non-academic actors. Academically, this work will provide a rich, first-of-its-kind, evidence base documenting the nuances of the chilling effect. This stands to make a positive contribution to existing theoretical research on the chilling effect, and may make a multi-disciplinary impact, particularly in law and human rights. Non-academically this work will be of particular use to civil society and other actors engaged in police use of surveillance technology. It stands to play an important role in litigation, and in advocacy. The UN Special Rapporteur on Freedom of Association and Assembly has already expressed interest in taking this work forward.

The human rights due diligence framework under preparation is intended to be taken up by a diverse range of government actors. Work is underway focusing on law enforcement, and the military. Over the coming months connections will be established in this regard.
Sectors Government, Democracy and Justice,Security and Diplomacy

 
Description The project is still in its early stages and so impact so far is necessarily limited. That said, there has been very promising engagement with the work strand dedicated to understanding how human rights can inform AI decision making processes. Some of this work is subject to confidentiality agreements. Work was conducted with one State agency to better understand the human rights considerations arising from their work, and to use this analysis to make concrete proposals for change. These proposals were presented to the board, and may be used to inform future work processes. Initial talks are also underway with a government body and an oversight agency to take this work forward. Work on the chilling effects of surveillance stands to make a landmark contribution to relevant academic fields. Although it is too early to tell, this may prompt significant interest in taking research forward, both empirically, and by utilising our research findings. This is a very exciting 'watch this space'. The UN Special Rapporteur on the Right to Freedom of Peaceful Assembly and of Association has expressed interest in taking this work forward. Impact also continues in relation to work specifically addressing facial recognition technologies. For example, research informed the Information Commissioner's guidance on the use of facial recognition in public spaces, and continues to generate engagement in the media.
First Year Of Impact 2021
Sector Government, Democracy and Justice,Security and Diplomacy
Impact Types Policy & public services

 
Description AHRC-UKRI Focus Group 4: Responsible AI Policy and Practice
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Citation by Green Party at European Parliament
Geographic Reach Europe 
Policy Influence Type Citation in other policy documents
URL https://www.greens-efa.eu/biometricsurveillance/
 
Description Contribution to UN Interregional Crime and Justice Research Institute Report
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to a national consultation/review
 
Description Engagement with Equality & Human Rights Commission
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Engagement with FCDO Digital Democracy Project
Geographic Reach Asia 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Engagement with Kent Police
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Engagement with Lord David Anderson
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
 
Description Engagement with OCDA Management Board
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Engagement with Review of Canadian Parliament's Use of Facial Recognition Technology
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
URL https://static1.squarespace.com/static/5fec97c81c227637fcd788af/t/62f2aeaade38217f6ea9459e/166007161...
 
Description Engagement with the EU Fundamental Rights Agency
Geographic Reach Europe 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Influence Australian Human Rights Commission
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://tech.humanrights.gov.au/sites/default/files/2021-05/AHRC_RightsTech_2021_Final_Report.pdf
 
Description Influence Irish Council for Civil Liberties
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://www.iccl.ie/wp-content/uploads/2022/11/221123_FRT_Letter_Minister_McEntee.pdf
 
Description Influence World Bank
Geographic Reach Multiple continents/international 
Policy Influence Type Citation in other policy documents
URL https://documents1.worldbank.org/curated/en/248201616598597113/pdf/World-Development-Report-2021-Dat...
 
Description Influence on German Marshall Fund of the United States
Geographic Reach Multiple continents/international 
Policy Influence Type Citation in other policy documents
URL https://www.gmfus.org/sites/default/files/2022-11/Goodman%20%26%20Trehu%20-%20Algorithmic%20Auditing...
 
Description Influenced Canadian review of parliament's use of facial recognition technology
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://static1.squarespace.com/static/5fec97c81c227637fcd788af/t/62f2aeaade38217f6ea9459e/166007161...
 
Description Influenced Tasmanian Liquor and Gaming Commission
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://www.treasury.tas.gov.au/Documents/Gambling%20Harm%20Minimisation%20Technologies%20-%20Resear...
 
Description Informed Information Commissioner's Office Approach to Facial Recognition
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://ico.org.uk/media/2619985/ico-opinion-the-use-of-lfr-in-public-places-20210618.pdf
 
Description Meeting with Markus Grübel (Christian Democratic Union)
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Meeting with UN SR Freedom of Assembly
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Submission to Justice and Home Affairs Committee
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL https://committees.parliament.uk/writtenevidence/38635/pdf/
 
Description Submission to UN SR Freedom of Assembly & Association Call for Inputs
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to a national consultation/review
 
Description UN Interregional Crime and Justice Research Institute Human Rights Stakeholder Meeting
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
 
Description AboutIntel Blog Post Re GCHQ AI Strategy 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Wrote blog addressing GCHQ's recently published AI Strategy. Subsequently in touch with GCHQ to discuss providing input on how human rights could more concretely inform their work.
Year(s) Of Engagement Activity 2021
URL https://aboutintel.eu/qchq-ethics-ai/
 
Description Engagement with GoodID Stories on Facial Recognition 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed by Good ID Stories, an instagram channel, on facial recognition and surveillance
Year(s) Of Engagement Activity 2022
URL https://www.instagram.com/p/CMuhiaAh03E/?igshid=YmMyMTA2M2Y%3D
 
Description Interview by L'Express on Facial Recognition London 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed by French newspaper L'Express on the use of facial recognition in London
Year(s) Of Engagement Activity 2021
URL https://www.lexpress.fr/economie/18-000-visages-identifies-par-minute-le-royaume-uni-l-autre-pays-de...
 
Description Interview for Chinese Language Media 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interview with Initium Media on police use of facial recognition in the UK
Year(s) Of Engagement Activity 2023
 
Description Interview for Declarations Podcast 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interview with Declarations Podcast discussing Live Facial Recognition. Season 6 Episode 3
Year(s) Of Engagement Activity 2022
URL https://declarationspod.com/season-6-episode-3-live-facial-recognition/
 
Description Interview for Wired 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interview with WIRED magazine for story on expansion of facial recognition in London
Year(s) Of Engagement Activity 2021
URL https://www.wired.co.uk/article/met-police-facial-recognition-new
 
Description Interview with Associated Press re AI and Instagram 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed by the Associated Press for story on Instagram's use of AI tools to determine individuals age
Year(s) Of Engagement Activity 2022
URL https://apnews.com/article/technology-artificial-intelligence-e7a5583ccfe7e1b284081db40cb2ea7c
 
Description Interview with TechMonitor re Facial Recognition 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interview with TechMonitor on UK public authorities' use of Chinese surveillance equipment, including facial recognition
Year(s) Of Engagement Activity 2022
URL https://techmonitor.ai/policy/privacy-and-data-protection/hikvision-dahua-cctv-uk-ban
 
Description Interview with The Guardian on GCHQ use of AI 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interview with the Guardian on GCHQ's use of AI.
Year(s) Of Engagement Activity 2021
URL https://www.theguardian.com/technology/2021/feb/24/ai-could-have-profound-effect-on-way-gchq-works-s...
 
Description Interview with civil society organisation 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Interview with Mark Wilding, from Liberty Investigates, an investigative reporting branch of the civil society organisation, Liberty. The interview was intended to inform Liberty's investigation into police use of facial recognition technology, this investigation is just beginning.
Year(s) Of Engagement Activity 2023
 
Description Louder Than Words Podcast - Facial Recognition 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed by Louder Than Words Podcast on the topic of facial recognition.
Year(s) Of Engagement Activity 2021
URL https://louder-than-words.castos.com/episodes/is-big-brother-watching-you
 
Description Media Interview Financial Times 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Interview with Madhumita Murgia, Financial Times. Interview based on research findings to discuss possible FT investigation into police use of FT.
Year(s) Of Engagement Activity 2022
 
Description Panel Facial Recognition and Emerging Technologies in Law Enforcement 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Participated on expert panel examining 'Facial Recognition and Emerging Technologies in Law Enforcement'. Other panellists: Prof. Pete Fussey (Essex), Dr. Marion Oswald (Northumbria).

Recent step changes in technologies available to law enforcement and security agencies, including uses of AI, predictive technologies, merged data hubs, algorithmic decision-making and advanced biometric video analytics such as facial recognition have generated significant attention and commentary. Focusing on remote biometric identification technologies in general, and live facial recognition in particular, and drawing on empirical research of their operational uses, this panel analyses how such developments have engaged multiple rights concerns, influenced operational practices and generated pressing questions for oversight and regulation.
Year(s) Of Engagement Activity 2022
URL https://www.eventbrite.co.uk/e/human-rights-and-emerging-technologies-tickets-368630873867
 
Description Panel Participation: Matrix on Point: Humanitarian Technologies 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Participated on panel discussing the use of AI in the humanitarian space. Presented on a human rights due diligence framework to facilitate the 'Do No Harm' Agenda.
Year(s) Of Engagement Activity 2022
URL https://matrix.berkeley.edu/events/matrix-on-point-humanitarian-technologies/
 
Description Participation TMC Asset Winter School on AI and Human Rights 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Participated in TMC Asser Winter School on AI and Human Rights, engaging (primarily with practitioners) on better understanding the impact that AI may exert on human rights protections, and how to respond.
Year(s) Of Engagement Activity 2021
URL https://www.asser.nl/media/795820/ar-annex-2021-def2.pdf
 
Description Participation on 'Biometrics, border tech and human rights' Panel 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Participated on 'Biometrics, border tech and human rights' panel, organised by the JustTech Coalition and the Institute for Humanities and Social Sciences. This is an initiative developed by Amnesty International, and participants included policy makers, government officials, and practitioners.
Year(s) Of Engagement Activity 2023
 
Description Presentation to Hertie School on how Human Rights can inform AI 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Presentation focused on how human rights law can inform our approach to AI. Particular emphasis was placed on moving from an after the fact approach (typical of human rights law) to a before the fact approach (which would offer significant benefits in terms of informing policy decisions).
Year(s) Of Engagement Activity 2021
URL https://www.hertie-school.org/en/events/event-detail/event/re-thinking-human-rights-for-the-ai-revol...