People Powered Algorithms for Desirable Social Outcomes

Lead Research Organisation: CRANFIELD UNIVERSITY
Department Name: Cranfield Defence and Security

Abstract

Algorithms increasingly govern interactions between state and citizen and as the 'digital by default' model of government-citizen interaction spreads this will increase. This increase, combined with the value of data science and how AI and machine learning is embraced as a way to achieve efficiency and carry out public policy we need to consider how algorithms mediate real-world relationships between the state and individuals.

Without confidence in the legitimacy and credibility of the algorithms the trust between government and citizens will dramatically degrade. Our research will therefore focus on algorithmic interactions between the citizen and the state and examine how we form productive and trusted relationships between those designing, deploying and using the algorithmic interactions and the communities affected by the decisions.

We will examine three key public policy areas where algorithmic decision-making is used for aspects of policy deployment: refugee resettlement, welfare and healthcare provision. These three areas have been selected as they are at the forefront of services that developed as part of digital by default, where issues of cost are addressed, in part, by algorithmic decision making to evaluate legitimate service access and use. Additionally, these are areas of significant public spending where the intended users of these services are more likely feel excluded and disenfranchised from mainstream society. Our research will examine how the re-designing of the system interactions and the communication of the political and economic logic will enhance the security and well-being of individuals, protects the security of the state and increases the confidence in digital service design.

Planned Impact

Our proposed research will synthesise the outputs of market design research with information security research and, by visualising and abstracting the outputs, facilitate new dialogues across a range of stakeholders from individuals through to policy makers. The resulting dialogues will:
1. Promote discussions within communities about fairness and legitimacy in data-driven decision making;
2. Facilitate inter-community conversations between system designers, policy makers and communities that discuss the impact of informal information flows and the conceptual models on which systems are built;
3. Diversify the security dialogue so that a more nuanced discussion can be had about the connection between personal security, state security and technical security.
4. Allow us to engage with market design researchers to explore how an understanding of informal information flows might impact on their research.

We will drive impact in four beneficiary groups: User communities; Government Policy makers; Data scientists, system designers and security architects and finally academia.

User communities: This societal impact will ensure that our research not only involves the user communities but also genuinely supports the increased well being of these communities. This will be achieved through the improved social outcomes from better algorithmic policy making and through the increased agency and awareness from discussing these important topics. We will work across a range of policy domains to reach particular communities. For example our work with algorithmically implemented refugee policy will reach Refugees, Hosting communities, Communities in zones of first asylum and Communities in zones of transit. While in the UK our research in healthcare and welfare will consider a number of community groups and local government in Sunderland (see letter of support), communities with which members of the team have already engaged.

Government policy makers: It is intended that this work will assist in supporting and developing algorithm design at a national level as well as a local government level. Note that all members of the team have been active in supporting policy development at government (see also letter of support from GCHQ). In the specific policy domains our research will benefit multiples stakeholders. In the domain of refugee policy the following government beneficiaries will be reached: State ministries (e.g. UKHO, German BAMF, etc) Intergovernmental agencies involved in coordinating resettlement (e.g. EASO, IOM, UNHCR resettlement office), Local government involved in meeting resettlement needs (councils, municipalities, quangos such as JOMAST), Humanitarian workers and CSOs and Migration management ministries and border. In the domain of welfare policy, the following government beneficiaries will be reached: Department of Work and Pensions both centrally and regionally.

Data scientists, systems designers and security architects: There is a significant increase in the use of data-led decision-making both in government and industry. Driving awareness and of the issues raised in this research in these practitioners through their education and skills pipeline will ensure that our research will have a long-lasting impact on their professional practice, not only in their current posts but also their future influence.

Academia:Our inter-university consortium will support the training of highly skilled researchers and facilitate the generation and transfer of new methods of research synthesis. This will be supplemented by a network of visiting academics from fields including Political Science, Economics, Computer Science and Cyber-Psychology - this will result in an energetic and vibrant community. Further engagement with projects such as DAPM, UnBias and Ox-Chain in addition to the Orbit and TIPS1 funded projects will ensure that the exciting research is disseminated widely amongst the UK and International communities.

Publications

10 25 50
 
Title Digital and Physical games to support discussions around algorithmic decision making 
Description Both digital and physical games have been built to take a player through a journey that is under the control of algorithmic decision-making, it tackles a number of key elements around agency and transparency and make these issues salient. 
Type Of Art Artefact (including digital) 
Year Produced 2022 
Impact The games have been played at a number of regional science festivals and generated significant levels of interest, both from players who reported an increased awareness of their interaction with algorithmic decision-making, and from media and the science outreach community regarding the approach to outreach. 
URL https://crestresearch.ac.uk/comment/rolling-the-dice-on-algorithms-increasing-understanding-through-...
 
Description Our work has focussed on the application of Automated Decision Making (ADM) across a variety of sectors. We have concentrated on the fundamental effects of the 'logics' used to build these models rather than the mechanisms used to construct the artefacts. Hence, rather than focussing on 'bias' or 'explainability', we have considered the effects of the social processes that occur during the design, implementation (and to a lesser degree) operation of ADM systems.

We have focussed on domains such as refugee resettlement and health and social care, working to expose the logics associated with the ADM, provide stakeholders and participants mechanisms to engage with logic and identify the unexpected outcomes from some of these fundamental logics.
Exploitation Route There are three main areas where this work could be exploited by others. The first is to support policymakers' understanding of the fundamental effects their choice of logic has on the outcomes. This is a complex domain-specific problem, and we have demonstrated the effect of this choice of logic within the domain of refugee resettlement and welfare provision. Articulating the transitive effect of this logic choice on the outcomes of those affected by ADM is an essential part of this work.

The second area for exploitation is to expose the logics of ADM to the general public. We have produced both real-world physical games and computer-based games to enable citizens to explore the effect of these logics on those affected in an engaging manner. Finally, we have also identified the value of using an art competition to examine the impact of ADM on citizens; utilising this mechanism to understand the effect on citizens allowed a talented group of individuals to articulate their interaction with ADM in the manner in which they were most able.

The third area is to develop the technique of "worldbuilding" as a means to i) examine the context in which ADM is to be used ii) to evaluate the impact of ADM in a given situation. Worldbuilding is a technique used to identify and understand how different stakeholders and the tasks they carry out interact with each other. It is a technique that enables algorithm designers to not only understand what types of ADM are necessary but also a technique to explore possible ADM futures.
Sectors Aerospace

Defence and Marine

Communities and Social Services/Policy

Digital/Communication/Information Technologies (including Software)

Education

Healthcare

Government

Democracy and Justice

Security and Diplomacy

URL http://peoplepoweredalogrithms.org/
 
Description The research synthesised in this grant continues to grow, the work on refugee resettlement and welfare provision identified the ways in which ADM is used. In collaboration with RHUL's StoryFutures speculative design techniques have been developed to explore how we might model the impacts of different types of ADM (these techniques can be seen here: https://voices-of-tomorrow.com/ ). This collection of speculative design techniques has been introduced to policymakers in both the US and the UK to stimulate discussion about what a people-powered ADM might mean. The games that were created to explore refugee resettlement are continuing to be developed and funded through impact accelerator awards, these will be both be released to the wider community as well as assessments made as to their efficacy as engagement tools. In addition, the lead-PI on this grant has taken up a role within Government and continues to be involved in related work.
First Year Of Impact 2022
Impact Types Cultural

Societal

Economic

Policy & public services

 
Description Collaboration with Norwich University of Arts to support Final year projects 
Organisation Norwich University of the Arts
Country United Kingdom 
Sector Academic/University 
PI Contribution The team support the conception, critique and presentation of a collaborative project as part of the students final year projects. This will lead to a number of mixed-media pieces being created to explore and represent the effects of algorithmic decision making. This was started in 2020, but due to COVID the students were unable to complete their projects, however, in 2021 we have continued this relationship and anticipate the students delivering their pieces in the summer of 2021.
Collaborator Contribution The students at Norwich University of Arts create a number of submissions exploring the effect of algorithms on the everyday. These pieces are across the range of media, including illustration, fine art and digital media .
Impact This collaboration joins the technical components of People Powered Algorithms for Desirable Social Outcomes with the arts discipline. The pieces will be delivered in the Summer 2021 and will form part of an exhibition and the project will retain the pieces.
Start Year 2019
 
Description Digital identity in asylum and migration management 
Organisation Boston University
Country United States 
Sector Academic/University 
PI Contribution Will Jones and Nick Micinski have conducted in depth interviews with civil servants, lawyers, activists, non-state welfare providers and politicians to understand the decisions of the Swedish state as to which forms of evidence they will require to be digitised, allowed to be digitised, and use automation to authenticate. They have then supplemented this with a small set of focus groups with refugees to find out the extent to which refugees are internalising these new requirements and altering their behaviour and strategies accordingly. We have found that in contrast to Germany and Denmark (the exemplars of the new digitised evidence regime for refugees) the Swedish state remains surprisingly low-tech on this point. We use this to reflect on the limits placed on the ability of states to use the digital to automate by constitutional and legal culture. This work will close out during 2020. Digital refugee identity. UNHCR is currently helming a high-level inter-governmental dialogue to produce a global digital identity package for refugees which will permit a degree of automated management and algorithmic decision-making (for the confirmation of evidence, for the dissemination of aid, for the governance of camps, etc). Will Jones and Nick Micinski have conducted preliminary interviews with the UNHCR digital identity team in Copenhagen with a view to producing research seeking to explain why UNHCR has embarked on this now, what perceived benefit has led states to participate, and the consequences for the lives of refugees. Our initial research suggests a hypothesis that this is driven by developmental agendas in the global south, and this will have seriously deleterious consequences for refugees in the West, which have largely been written off as the cost of doing business. This work will close out in 2021.
Collaborator Contribution This is co-conducted research
Impact One article in the Journal of Refugee Studies (done). One further article for a digital politics journal. We plan to do a dissemination activity this summer in London and Stockholm (travel permitting) for refugee advocates and resettlement officials.
Start Year 2019
 
Description Robotic Process Automation in Government 
Organisation University of Gothenburg
Country Sweden 
Sector Academic/University 
PI Contribution Collaborating with Agneta Ranerup at Gothenburg University, Will is examining how third party technology companies shape and direct the policy of using robotic process automation in UK and Swedish government services. Ranerup is an authoritative voice on the Trelleborg RPA programme (Sweden) and Will has interviewed a UK civil servant leading the UK programme. In this work we build on Margett's work (Information Technology in Government) to unpack how the drive to introduce RPA has created a new set of relationships between central government, implementing agencies, and private companies. The drive to replace human-conducted processes with algorithmic bots has ultimately had counterintuitive logics for the capacity of the state to digitise in the future, to reform its services, or to understand the strategic logic of its own programming. This work will be closed out during 2020.
Collaborator Contribution Dr Ranerup is leading the Swedish side of the research
Impact TB determined, but it will likely involve whatever we decide is the most effective way to assist governments engaging in RPA with understanding the wider political impacts of their work.
Start Year 2018
 
Description AI for a safe and secure UK - AI Frenzy at Barclays Eagle Lab 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Around 80 SMEs and data science practitioners attended an AI frenzy hosted by the Barclay Eagle Lab, the project led a session on ethical and secure AI. The feedback forms indicated that the session was thought provoking and 82% of respondents would be 'Likely' or 'Very Likely' to take lessons back to their workplace.
Year(s) Of Engagement Activity 2019
 
Description April 2021: Panel presentation: Migrant Belongings: Digital Practices and the Everyday Conference, Utrecht University, Netherlands. [Virtual event] 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact This panel presentation provoked discussion amongst researchers and practitioners about the use of digital technology and algorithms to implement migration policies.
Year(s) Of Engagement Activity 2021
 
Description Blog post on the topic of Refugees and Digital Exclusion. 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Blog post that has been used to encourage debate and reflection.
Year(s) Of Engagement Activity 2021
URL https://www.rethinkingrefuge.org/articles/refugees-and-digital-exclusion
 
Description Informal ONS meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Presentation of People Powered Algorithm project to members of the inclusive Data Taskforce
Year(s) Of Engagement Activity 2021
 
Description March 2021: Panel presentation: How to Make Urban Refugees Count: Reflections on data gaps and data collection during COVID-19. 2nd IGAD Scientific Conference on Migration and Displacement in the Context of COVID-19. IGAD (Intergovernmental Authority on Development). [Virtual event] 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We used data from East Africa and focused on digital data collection and algorithms, including remote sensing in order to encourage debate on the use of algorithms and sensors as part of refugee management.
Year(s) Of Engagement Activity 2021
 
Description Meeting with the Office for Artificial Intelligence 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Policymakers/politicians
Results and Impact The project was introduced to the Office for AI along with the research process and how the various streams of research fit together. The individuals were interested in the project and agreed to help identify relevant projects which could be approached as required.
Year(s) Of Engagement Activity 2019
 
Description Norwich Science Festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Both variants of the game (see Creative Outputs) were played during the festival, with significant interest from all attendees. There was also significant regional media interest.
Year(s) Of Engagement Activity 2022
 
Description Not Equal Network + summer school session 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Members delivered a session to a summer school which was run as part of the UKRI funded NotEqual Network+. The session was aimed at helping the participants begin to unpack some of the issues around algorithmic decision making in governments. The participants were very engaged during the session and associated tasks and were beginning to articulate an appreciation for some of the effects of an increased use of ADM.
Year(s) Of Engagement Activity 2019
 
Description Panel presentation: Refugee Integration in a Sharing Economy: Collective Action, Organizational Communication and Digital Technologies; International Association for Media and Communication Research (IAMCR) Conference. [Virtual event] 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact We took part in a panel discussion at a prestigious conference on media and communication. The panel discussion provoked debate on the use of algorithms and the use of digital monitoring in refugee resettlement and migration.
Year(s) Of Engagement Activity 2021
 
Description Paper presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Presented a paper at the Australian Social Policy conference. This was an aural presentation that presented an examination of digital welfare and identified the different types of security necessary in this case of automated decision making.
Year(s) Of Engagement Activity 2019
 
Description Presentation to HEAR Network [London-based civil society organisations working with marginalised populations] on research: Digital technology, integration, and refugees. [Virtual event] 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Dissemination about the project activities which sparked debate about the use of algorithms in refugee resetllement.
Year(s) Of Engagement Activity 2021