What does Artificial Intelligence Mean for the Future of Democratic Society? Examining the societal impact of AI and whether human rights can respond

Lead Research Organisation: Queen Mary University of London
Department Name: Sch of Law

Abstract

This research examines the impacts that States' use of artificial intelligence (AI) in decision making processes has on how individuals and societies evolve and develop and what this means for democratic society. Understanding these impacts is essential so that effective guidance can be developed that allows States to take advantage of the significant potential inherent in AI, while protecting those factors essential to a functioning democracy and preventing human rights harm.
AI has the power to radically transform State activity, redefining our understanding of how a State functions and delivers services, and how it interacts with its citizens. A key development in this regard is the incorporation of AI tools into State decision-making processes. To be effective, these tools are dependent upon significantly increased surveillance by State and non-State actors: the data obtained through surveillance is subject to analysis using AI in order to make individually-tailored decisions. This represents a step-change in terms of the level of insight the State has into individuals' day-to-day lives, and their ability to use this information to determine that individual's life choices. This may exert a profound impact on how individuals, and society as a whole, develops. Will individuals be afraid to experiment, or to seek out alternative ideas or ways of life, because they are worried that they will be categorised on this basis and their future life choices restricted? Will this in turn lead to the stagnation of democratic society?
AI has enormous potential. It can be used to transform how a State delivers services, and if used appropriately can make a real contribution to the development of society, and the protection of human rights. However, it is imperative that the broader impacts of AI on individuals and society be understood before AI becomes pervasive in decision-making processes, so that appropriate regulatory and policy responses can be developed, and human rights protections ensures.
This research focuses on the inadvertent, or unintentional, impacts associated with State adoption of AI technologies. There is, of course, clear potential for AI to be misused for repressive purposes. Of interest here, however, is States' use of AI when deployed in pursuit of legitimate objectives. The unintended consequences associated with States' uses of AI under these circumstances may be less visible but equally dramatic.
Human rights law provides the framework underpinning research. Although it must be reconceptualised to respond to the digital age (a key research objective) it provides the most effective means of identifying harm, resolving competing interests, and providing regulatory guidance.
The principal objective underpinning this inter-disciplinary research is the development of future-oriented human rights approaches to regulate States' use of AI in decision making processes, and to ensure that AI serves, rather than undermines, societal objectives. To do so will require in-depth research across law, human rights, philosophy, and sociology. Initial research will investigate factors essential to individual and societal development, how these relate to democratic functioning, and how they are impacted by States' use of AI. Human rights law itself must then be re-conceptualised, to ensure that it is capable of engaging with these factors, and protecting them in the digital age.
State agencies are beginning to incorporate AI technologies, and the utilisation of AI will increase exponentially over the coming years. Surveillance and AI-assisted analytical tools are deployed across all areas of State activity, from social welfare, to child protection, and healthcare. To examine the democratic effects where they are most visible in the short term, however, research will focus primarily on State activity related to law enforcement and counter-terrorism, examining the use of AI by police and intelligence agencies.

Planned Impact

The principal objective underpinning this research project is the production of world class research to inform human rights compliant policy and regulation with respect to the development and deployment of artificial intelligence (AI) tools by States. State agencies are just beginning to incorporate AI technologies, and it is expected that the utilisation of AI by public authorities will increase exponentially over the coming years. Indeed, AI has been put at the centre of the UK government's development and investment plans (see establishment of Centre for Data Ethics and Innovation, or Digital Catapult), reflecting global trends in this direction.
To-date, however, research has not addressed the effects that the use of AI in State decision-making processes has on how individuals and societies develop and evolve, and how this may affect the effective functioning of democratic society. This gap is significant as understanding these impacts is essential in order to identify if, and how, AI technologies may be appropriately deployed in decision-making contexts. This research intends to contribute directly to the development of policy guidance and regulation in this area.
Three principal pathways to impact will be pursued, addressing the United Nations, domestic State actors, and general awareness raising. As this is an emerging area of research, it is important that research findings are engaged with and adopted by standard setting bodies, or those with particular influence in the area, so that the frameworks developed during the research project can be taken up and receive authoritative endorsement.
United Nations
UN actors play a decisive role in developing human rights law, and the United Nations is arguably the most influential forum internationally in terms of informing international and domestic human rights approaches. This impact pathway is a means to ensure that research outputs inform mainstream human rights approaches. Engagement serves two purposes. It provides an opportunity to (a) influence thinking and policy development at a global level, and (b) contribute to the development of soft-law standards, capable of influencing domestic policy, legislation, and court decisions.
The applicant has pre-existing contacts with relevant actors, and a number of strategies for engagement will be employed. First, consultations will be held with the Office of the High Commissioner for Human Rights and relevant Special Procedures to discuss the research agenda, and to identify areas of mutual interest. Second, broad consultations will be held with all actors to identify areas of interest that would benefit from in-depth discussion. Workshops will then be organised in these areas, with outputs tailored to participants needs. Third, submissions will be made to various bodies, in response to calls in order to contribute directly to standard setting.
Domestic State actors
State agencies operating at the domestic level play a clear role in both the development and operationalisation of policy. The flexibility built into this funding scheme offers an opportunity to engage with these actors, and to tailor research to their specific needs thereby informing their practice and securing impact. This project provides a clear added value to State actors' work. It allows for academically rigorous research, provided by an independent third party, without placing additional strain on the actor's resources.
General Awareness Raising
The focus here is on distilling research findings into accessible public facing outputs across a variety of different formats. Using existing contacts, leading non-governmental organisations active in this area will be engaged. Similar to the first two pathways, efforts will be made to ensure that research can be tailored to these actors' needs, in order to inform their work. Outputs will also be posted regularly to the project website in a variety of non-academic forms tailored for different actors.
 
Description To-date, this project has resulted in three initial findings. These preliminary findings will be explored, and developed, over the life of the project.

First, the chilling effects of surveillance are real, and exert a profound impact on individuals personal and professional lives. This is likely to severely limit individuals ability to freely develop their identity, and to engage politically. Further research is underway, to explore these initial findings in more detail, and to develop a more robust evidence base. This finding has potentially significant impact vis-a-vis police use of surveillance technology.

Second, there is an appetite within policy to explore a human rights-based approach to AI tools, Initial work has engaged with various police constabularies, and with the Equality & Human Rights Commission. Work is underway to add content to this human rights-based approach, so that this can be of more practicable use to different State actors. This also paves the way for significant academic contribution going forward.

Third, existing research on police use of digital technologies and work conducted over the first two years of the project resulted in a collaboration with various UN actors. The outcome of this collaboration is the 'Model Protocol for Law Enforcement Officials to Promote and Protect Human Rights in the Context of Peaceful Protests', and an accompanying document focused specifically on the use of digital technologies. This project played a leading role in the drafting of the digital technologies component of the Protocol and accompanying document, and informed the Protocol as a whole. This is the first international document to address police use of modern surveillance techniques in the context of protest, and is a standard setting document.
Exploitation Route The Model Protocol is a standard setting document, issued by a number of different UN agencies. In the days since its release it has been welcomed by numerous States and NGOs. As a standard setting document it serves as a point of reference both for police forces, so that it may inform their use of digital technologies in the context of protests, and for other actors, as a frame of reference to evaluate police action by.
Sectors Government

Democracy and Justice

URL https://freeassemblyandassociation.net/tools-for-law-enforcement/
 
Description Research findings have contributed to two key non-academic impacts. First, research played a key role in the development, and drafting, of the digital technologies component of the UN 'Model Protocol for Law Enforcement Officials to Promote and Protect Human Rights in the Context of Peaceful Protests'. Our team led on the digital technologies strand of the work. As part of this we produced background papers, convened and hosted two international workshops, and led on the drafting of the digital technologies elements of the Model Protocol. We also played a key role in drafting the accompanying document which establishes principled guidance focused specifically on police use of digital technologies. Second, research played a key role in informing the Equality & Human Rights Commission's policy in relation to police use of facial recognition technology, and their use of new technologies generally. The Equality & Human Rights Commission is the UK's National Human Rights Institution, and plays a key role in ensuring human rights compliance in relation to State actors. Our research, and direct engagement with the organisation (an ongoing collaboration) has played a key role in informing their policy, shaping their engagement with different police constabularies and the Home Office.
First Year Of Impact 2023
Sector Government, Democracy and Justice
 
Description Cited by Australian Human Rights Commission on Need for Human Rights re AI
Geographic Reach National 
Policy Influence Type Citation in other policy documents
Impact Research informed work of Australian Human Rights Commission in submission to department of Industry, Science and Resources, emphasising need for Human Rights Based Approach to AI.
URL https://humanrights.gov.au/sites/default/files/the_need_for_human_rights-centred_artificial_intellig...
 
Description Contributed to Joint Declaration on Freedom of Peaceful Assembly and of Association and Misuse of Digital Technologies
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to a national consultation/review
Impact Contributed to the drafting of Joint Declaration on Freedom of Peaceful Assembly and of Association and Misuse of Digital Technologies. This declaration is one of the first to address the human rights impacts linked to the use of digital technologies, specifically in a protest context. It is a reference point for others working on this issue.
URL https://www.ohchr.org/sites/default/files/documents/issues/trafficking/statements/20230915-jd-foaa-d...
 
Description Informed Report on Pre-Legislative Scrutiny of the General Scheme of the Garda Síochána (Recording Devices) (Amendment) Bill 2023
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
Impact This is a pre-legislative scrutiny report, which will inform any drafting decisions.
URL https://data.oireachtas.ie/ie/oireachtas/committee/dail/33/joint_committee_on_justice/reports/2024/2...
 
Description Informed the work of the UN Special Rapporteur on the Rights to Freedom of Assembly and to Association, and other UN actors, in the development of a Model Protocol for Law Enforcement Officials to Promote and Protect Human Rights in the Context of Peaceful Protests, and accompanying documents
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
Impact It is too early to note full impact, as the Model Protocol was only released on 6 March. However, the Model Protocol was welcomed by numerous States during the UN Human Rights Council Session where the Protocol was presented to the Council. It has also been welcomed by civil society organisations.
URL https://undocs.org/en/A/HRC/55/60
 
Description Meeting with UN SR Freedom of Assembly
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Oral Evidence Committee on Justice (Ireland) - Draft Facial Recognition Bill
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
Impact This oral evidence fed into a pre-legislative report addressing the proposed new Bill on police use of facial recognition technology.
URL https://www.oireachtas.ie/en/press-centre/press-releases/20240212-the-joint-committee-on-justice-to-...
 
Description Oral Evidence House of Lords AI and Weapons Systems Inquiry
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL https://committees.parliament.uk/event/17788/formal-meeting-oral-evidence-session/
 
Description Quoted in 'Biometric Britain' report by Big Brother Watch
Geographic Reach National 
Policy Influence Type Citation in other policy documents
Impact Informed substantive findings in Big Brother Watch's report. Big Brother Watch are one of the leading civil society organisations active in the surveillance space in the UK.
URL https://bigbrotherwatch.org.uk/wp-content/uploads/2023/05/Biometric-Britain.pdf
 
Description Referred to in the 'Independent Review of the Investigatory Powers Act 2016'
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://www.gov.uk/government/publications/independent-review-of-the-investigatory-powers-act-2016--...
 
Description Submission to UN SR Freedom of Assembly & Association Call for Inputs
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to a national consultation/review
 
Description Written Submission to Committee on Justice (Ireland) regarding proposed facial recognition Bill
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
Impact Contributed to a pre-legislative report by the Committee.
 
Description Collaboration with EHRC on police adoption of new technologies 
Organisation Equality and Human Rights Commission
Country United Kingdom 
Sector Public 
PI Contribution Collaborated with the Equality and Human Rights Commission to assist in the development of their policy regarding police adoption of new technologies. Convened a range of police officers, academics, and police and crime commissioners to discuss issues relating to police adoption of new technologies. The objective of this workshop was to work towards a human rights compliant process to oversee the potential adoption of new technologies. The workshop was held at Queen May University of London. My team and I prepared a background paper for this workshop, hosted the workshop, and helped to chair and facilitate the event itself.
Collaborator Contribution The aim of this collaboration was to inform the EHRC's development of policy in this area.
Impact This collaboration is multi-disciplinary involving law, human rights law, and sociality. There are no public outputs available at the moment. However, this workshop has informed the EHRC's internal policy regarding facial recognition, and other policy is under development.
Start Year 2023
 
Description Collaboration with UN agencies to develop Model Protocol for Policing of Protests 
Organisation United Nations (UN)
Department Office of the United Nations High Commissioner for Human Rights
Country Switzerland 
Sector Charity/Non Profit 
PI Contribution Collaborated with the UN Special Rapporteur, Office of the High Commissioner for Human Rights, and UN Office of Drugs and Crime to develop the 'Model Protocol for Law Enforcement Officials to Promote and Protect Human Rights in the Context of Peaceful Protests'. I led on the strand of this collaboration dealing with the use of digital tools. This work informed the content of the Model Protocol itself. With a colleague I was also the co-author of an associated Annex, dealing specifically with police use of digital tools in the context of policing protests. I also organised a two day workshop in Geneva, with police and civil society from around the world, to specifically address the use of digital tools in the context of protests.
Collaborator Contribution The Human Rights Council requested that the UN Special Rapporteur, Office of the High Commissioner for Human Rights, and UN Office of Drugs and Crime develop a model protocol addressing the policing of protests. These three agencies all contributed to the overall process, and were involved in the drafting of the document.
Impact Model Protocol for Law Enforcement Officials to Promote and Protect Human Rights in the Context of Peaceful Protests Annex: 'Human rights compliant uses of digital technologies for the facilitation of peaceful protests' This collaboration is multi-disciplinary involving law, human rights law, and sociology.
Start Year 2023
 
Description Collaboration with UN agencies to develop Model Protocol for Policing of Protests 
Organisation United Nations Office on Drugs and Crime
Country Global 
Sector Public 
PI Contribution Collaborated with the UN Special Rapporteur, Office of the High Commissioner for Human Rights, and UN Office of Drugs and Crime to develop the 'Model Protocol for Law Enforcement Officials to Promote and Protect Human Rights in the Context of Peaceful Protests'. I led on the strand of this collaboration dealing with the use of digital tools. This work informed the content of the Model Protocol itself. With a colleague I was also the co-author of an associated Annex, dealing specifically with police use of digital tools in the context of policing protests. I also organised a two day workshop in Geneva, with police and civil society from around the world, to specifically address the use of digital tools in the context of protests.
Collaborator Contribution The Human Rights Council requested that the UN Special Rapporteur, Office of the High Commissioner for Human Rights, and UN Office of Drugs and Crime develop a model protocol addressing the policing of protests. These three agencies all contributed to the overall process, and were involved in the drafting of the document.
Impact Model Protocol for Law Enforcement Officials to Promote and Protect Human Rights in the Context of Peaceful Protests Annex: 'Human rights compliant uses of digital technologies for the facilitation of peaceful protests' This collaboration is multi-disciplinary involving law, human rights law, and sociology.
Start Year 2023
 
Description Interview with New York Times reporter on facial recognition 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interview for Intium Media - Chinese independent media organisation - on the use of facial recognition in the UK
Year(s) Of Engagement Activity 2023
URL https://theinitium.com/article/20230322-international-surveillance-in-uk/
 
Description Interview with national radio station, DriveTime (Ireland) 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed on RTE DriveTime by Margaret Curley in relation to proposals to introduce police use of facial recognition technology in Ireland.
Year(s) Of Engagement Activity 2024
 
Description Interviewed on 'Declarations' podcast re police use of facial recognition 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Interviewed for Declarations podcast, on the police's use of live facial recognition.
Year(s) Of Engagement Activity 2024
URL https://declarationspod.com/2022/02/10/season-2-episode-3-fortress-europe/
 
Description Participated in talk hosted by the 'Just Tech and Migration Steering Committee' 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Participated in talk hosted by 'Just Tech and Migration Steering Committee' focused on the use of AI and other new technologies in a migration context.
Year(s) Of Engagement Activity 2023
URL https://www.youtube.com/watch?v=un7koljzsBc
 
Description Presented at event on 'Police Surveillance North and South' (Ireland) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Participated in event organised by the Committee on the Administration of Justice and the Irish Council for Civil Liberties, focusing on 'Police Surveillance North and South'
Year(s) Of Engagement Activity 2024
URL https://caj.org.uk/latest/police-surveillance-conference/
 
Description Quoted in Army Technology on Autonomous Weapons Systems 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Quoted in Army Technology piece regarding the use of autonomous weapons systems, or AI more generally, by the military.
Year(s) Of Engagement Activity 2023
URL https://www.army-technology.com/features/precision-weapons-and-preventing-collateral-damage/?cf-view
 
Description Quoted in BiometricUpdate article on facial recognition 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Quoted in Biometric Update article regarding legal basis (or need for a legal basis) for facial recognition technology.
Year(s) Of Engagement Activity 2024
URL https://www.biometricupdate.com/202312/uk-police-use-of-facial-recognition-probed-by-lawmakers
 
Description Quoted in Drone Wars article 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Third sector organisations
Results and Impact Quoted by Drone Wars (a UK NGO focused on the use of military drones) in relation to evidence given to the House of Lords Committee on AI and Weapons Systems.
Year(s) Of Engagement Activity 2024
URL https://dronewars.net/2023/04/13/lords-committee-on-ai-in-weapons-systems-ai-harms-humans-vs-compute...
 
Description Quoted in EuroNews Article 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Quoted in media regarding Irish proposal to introduce new police facial recognition legislation.
Year(s) Of Engagement Activity 2024
URL https://www.euronews.com/next/2024/03/01/irish-police-facial-recognition-bill-has-fundamental-defect...
 
Description Quoted in Liberty Investigates 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Third sector organisations
Results and Impact Quoted in Liberty Investigates (the leading British human rights organisation) piece concerning police use of facial recognition.
Year(s) Of Engagement Activity 2024
URL https://libertyinvestigates.org.uk/articles/hundreds-of-thousands-of-innocent-people-on-police-datab...
 
Description Quoted in NME 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Quoted in NME piece concerning the use of facial recognition at a Beyonce concert.
Year(s) Of Engagement Activity 2024
URL https://www.nme.com/news/music/police-criticised-for-using-live-facial-recognition-in-cardiff-during...
 
Description Quoted in The Irish Times 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Gave expert evidence to Irish Parliamentary Committee examining the use of facial recognition technology. This article quotes my contribution.
Year(s) Of Engagement Activity 2024
URL https://www.irishtimes.com/politics/2024/02/13/academics-warn-of-facial-recognition-technologys-chil...
 
Description Quoted in The New York Times 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Quoted in the New York Times on 'A.I., Brain Scans and Cameras: The Spread of Police Surveillance Tech'
Year(s) Of Engagement Activity 2024
URL https://www.nytimes.com/2023/03/30/technology/police-surveillance-tech-dubai.html
 
Description Quoted in the Guardian newspaper 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Quoted in Guardian article relating to the use of facial recognition technology at Beyonce concert in Cardiff.
Year(s) Of Engagement Activity 2024
URL https://www.theguardian.com/technology/2023/may/17/police-to-use-facial-recognition-technology-in-ca...
 
Description Quoted in the Journal.ie 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Quoted in the Journal.ie regarding comments made in relation to police use of facial recognition, and the potential that they may be used in Ireland.
Year(s) Of Engagement Activity 2024
URL https://www.thejournal.ie/racial-bias-in-facial-tracking-gardai-6205331-Oct2023/
 
Description Quoted on Irish Council of Civil Liberties website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Quoted in Irish Council for Civil Liberties website update, regarding 'Leading experts warn against Garda use of FRT'
Year(s) Of Engagement Activity 2024
URL https://www.iccl.ie/digital-data/leading-facial-recognition-technology-experts-have-warned-against-g...
 
Description Taught Summer School Module on Human Rights in the Digital Age 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Taught on the Bonavero Institute for Human Rights (Oxford) and George Washington University Law School summer school. Lead module on human rights and AI.
Year(s) Of Engagement Activity 2023
URL https://www.law.ox.ac.uk/content/news/new-summer-school-international-human-rights-law
 
Description Work referred to in Computer Weekly Magazine, re AI and weapons systems 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Quoted extensively in computer weekly magazine article on AI and Weapons Systems, in light of House of Lords inquiry;
Year(s) Of Engagement Activity 2023
URL https://www.computerweekly.com/news/365533962/Lords-AI-weapons-committee-holds-first-evidence-sessio...