UnBias: Emancipating Users Against Algorithmic Biases for a Trusted Digital Economy

Lead Research Organisation: University of Nottingham
Department Name: School of Computer Science

Abstract

Contrary to public opinion, young people care about their personal data and want a digital world more transparent, a digital world they can trust. For example, little is known about how Amazon is able to tailor advertisements and recommend products that are actually interesting for potential online customers, or how Facebook decides which news Facebook users may be more inclined to read. All the mechanisms that support this filtering of information and products is obscure and internet users would like to know more about it, such as possible bias in their behaviour and, more importantly, have some control over these recommender systems.

This project aims to closely work with young people to further understand how aware 'digital natives' are about algorithm bias, their attitudes and main concerns and recommendations when interacting with such systems. This information will help us to better understand the way young people interact with such systems and identify youth-led solutions for teaching critical thinking toward digital information systems. We will apply different engagement tools and methodologies including focus groups, workshops and youth 'juries' to facilitate discussion, reflection and a deeper understanding of youth online behaviour and youth-lead software solutions.

Relevant for this project is the development of 'fairer' algorithms with young people or non-experts as well as with experts. To do that, we intend to run a series of hackathons or workshops in which expert programmers and young people will work together to produce and evaluate new fairer algorithms that will produce more transparent and less creepy outputs. We will also run an 'ethicon'; an innovative approach in which scientists and ethicists work together to produce ethically justifiable algorithm design. During this process the project will be able to identify a series of principles that will contribute to the development of a platform that will allow users to have more control over the existing online filtering systems. In other words, this project will provide citizens with the skills and tools to better judge when and how much to trust the information they are given, understand the digital identity that algorithms create from user's personal data as well as protecting the users' privacy and online security.

This project will provide policy recommendations, ethical guidelines and a 'fairness toolkit' co-produced with young people and other stakeholders that will include educational materials and resources co-designed to support youth understanding about online environments as well as raise awareness among online providers about the concerns and rights of young internet users. This project is relevant for young people as well as society as a whole to ensure trust and transparency are not missing from the internet.

The results will be widely disseminated to a variety of audiences ranging from academic peer-review journals to community groups of interest such as secondary schools and youth clubs.

Planned Impact

*General Public:*
The public is at the centre of this project from the initial co-design of scenarios/tasks, through the 'Youth Juries' workshops and focus group sessions that map public concerns, to co-design Hackathons and prototype evaluation sessions.

Educational materials produced as part of the project will be made available to educators, guardians and other stakeholders. These materials will help to empower citizens with skills and tools to reflect upon the behaviour of social media algorithms so that they will be better positioned to know which online services to trust, manage their online identity and privacy and be more secure from unwanted manipulation.

In addition workshop with young people, their guardians and educators, the project will also include dedicated public engagement and communications activities, such as presentations associated at:
- ESRC Research Methods Festival.
- ESRC Festival of Social Science.
- Nesta's Future Fest [futurefest.org], a weekend dedicated to "radical ideas, compelling talks and immersive experiences to inspire, excite and challenge perceptions of the future".
- MozFest in London, an annual hackathon organized by the Mozilla foundation.

*Policy-makers:*
Our findings regarding the cumulative impact of filtering/recommender algorithms on users will be communicated to the Information Commissioner's Office, to facilitate evidence based policy review related to the UK Data Protection Act's 'principle 6' on the "protection of individuals against having significant decisions made about them by wholly automated mean".
A report on teaching critical inquiry of algorithm mediated information will be prepared for presentation to the minister for internet safety and security for contributing towards evidence based policy making.
A report, summarizing our findings on subjective experiences of algorithm 'fairness' and related design guidelines for 'fair' information control algorithms will be produced for industry bodies, to support the development of improved industry guidelines on transparency and responsible innovation.

*Industry:*
There will be a series of workshops for software developers and service providers to encourage thinking about implicit and explicit biases that may affect information control algorithms. It is the hope that these workshops will stimulate innovations that will boost the reputation of the UK Digital Economy as leader in trustworthy, reliable systems.
The project will tap into an existing network of over 200 commercial partners at the Horizon Digital Economy Research institute to disseminate findings and encourage participation in relevant events. One route to engagement with SMEs will be through Continuing Professional Development workshops. Efforts will be particularly focused on sectors where understanding human behaviour and 'fair' unbiased information provision is business-critical, such as market analysis and product evaluation.
Beyond the immediate benefits to citizens, the outcomes will also contribute towards a fairer digital economy by helping to expose anticompetitive practices that might be embedded in algorithm behaviours and which harm SMEs especially.

*Third Sector:*
The educational materials produced by this project will be made available to support programmes by NGOs and Charities, like the Mozilla foundation, in teaching young people how to develop critically inquiring attitudes towards digital systems. Starting from the first year the project will engage with these organizations as an important part of the stakeholders workshops.
The 'Youth Juries' workshops will build on existing links between the Horizon Digital Economy Research institute and the iRights coalition which promotes the rights of young people online.
 
Title AI for Decision Makers toolkit 
Description AI for Decision Makers toolkit . AI4DM is published under the same Creative Commons license as before UnBias Toolkit (Attribution, Non Commercial, Share Alike). The "DIY version" is open and free to download and make up with commonly available stationery . There is a physical box set of cards which can be ordered "print on demand" just like the previous set too: https://www.makeplayingcards.com/sell/marketplace/unbias-ai-for-decision-makers.html Dr Ansgar is aiming to talk about the toolkit at various industry orientated conferences during 2020 - TBC - potentials being: International Conference on Robotics, Automation & Artificial intelligence Systems Conference https://icraais.com/ MyData2020 https://online2020.mydata.org/ Digital Leaders Week https://week.digileaders.com/talks/submit-suggestion/# 
Type Of Art Artefact (including digital) 
Year Produced 2020 
Impact Production of this toolkit was commissioned to Proboscis for Ernst Young under CC BY SA licence and launched by Proboscis: https://www.indiegogo.com/projects/unbias-ai-for-decision-makers/ The toolkit is open source and launched on Sept 15th with a crowdfunding campaign to raise funds to manufacture a print run and make the kit more affordable that the print-on-demand option (through scale of economy). The campaign offers the original UnBias Fairness Toolkit and thus increases the impact of this to reach new audiences who may not have had access to it previously. The toolkit was used in a workshop led by Ansgar Koene to over 130 registered guests and video produced by Giles Lane supported this event (Autumn 2020) - link to video: https://youtu.be/aIFVo3xCSOw 
URL https://www.makeplayingcards.com/sell/marketplace/unbias-ai-for-decision-makers.html
 
Title Our Future Internet: Free And Fair For All! 
Description This is a short video animation (under 3 minutes) which explains what algorithms are, how they shape our online browsing and how they can create risks of bias. It also describes how the UnBias project seeks to promote a future Internet that is free and fair for all. It was commissioned to a external company but co-created with the UnBias team. 
Type Of Art Film/Video/Animation 
Year Produced 2018 
Impact Since it was launched in May 2018, it has been disseminated by our team in a wide range of activities including: UnBias competition, national and international presentations, the UnBias Showcase event, V&A Digital Design weekend, ESRC and Impact Exploration Grant workshops, and other UnBias related workshops/public engagement events. The feedback we have received from this video animation has been extremely positive. In particular, workshop participants including young people and/or older adults (65 years of age and older) with or without previous knowledge of the concept of algorithms said the content of the video was very clear, fun and educational! 
URL http://bit.ly/OurFutureInternet
 
Title Roots 
Description This was an artistic exhibition organised at the Institute of Mental Health. The purpose of this exhibition was to share the artists' work and inspire others to try art as a means of helping them through difficult times by finding ways to express their emotions and be in the "here and now". 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2017 
Impact Outcomes of "Roots" Exhibition. Artist statement: People have expressly stated how inspired they have been by the art and staff at IMH have stopped and thanked me stating that the light and life the artwork has brought into the building has been inspiring. Many people have stated that they have felt emotional connections to the artworks. Contributions to others and the community: Provided a talk with a group of MA students of Trauma and used the displayed artwork as a talking point for growth and resilience through trauma. Followed up with an individual student interview to assist with dissertation studies Held live discussion with BBC Radio Nottingham (Alan Clifford show) Developed links with the Nottingham Peace Project and planned provision of and gained council funding to lead "expression through art" workshops for young people in Basford (2018) Roots Exhibition Book - my story of Trauma, Resilience and Growth with accompanying paintings 1. Charitable financial benefits: Sales of paintings have raised £800 to provide a Remembrance Bench for Joel Cooke and £300 for Young Minds Trust. A total of £1,100 sales was made between the 10th October 2017 and 5th of January 2018. 2. New friendships and New opportunities: I have had a number of people ask me to show them how I paint different styles and I am starting small workshops at home on the 10th Jan 2018 I am contributing to Prof Stephen Regel's Stories and Narratives Project (2018) I am working with Su Ansell (De Montford University) on producing a short film about resilience, change and growth (2018) 3. Personal Growth, Empowerment, Health and Well-Being: The exhibition has encouraged me to get out and about, meet new people, socialise and converse. Exposure to the environment, particularly travelling on buses was my goal for my trauma treatment. I now have no fear of travelling on buses or passing through Nottingham. My confidence and communication skills are much improved. When I began planning g the exhibition with Elvira I had to take my daughter with me as I struggled with conversation skills. That is no longer the case. I am sure Elvira has noted the differences since we first met. My family and friends often comment on the progress I have made in overcoming my fear of public places and people. I intend to continue to raise funds for charity, raise awareness of how we can rebuild our resilience and growth through art expression. I will continue to inspire others by sharing my story and art. 
URL http://www.institutemh.org.uk/x-about-us-x/art-at-the-institute
 
Title UnBias Fairness Toolkit 
Description The Fairness Toolkit has been developed for UnBias by Giles Lane and his team at Proboscis, with the input of young people and stakeholders. It aims to promote awareness and stimulate a public civic dialogue about how algorithms shape online experiences and to reflect on possible changes to address issues of online unfairness. The toolkit contains the following elements: Handbook, Awareness Cards, TrustScape, MetaMap and Value Perception Worksheets.For more detail about these components please visit https://unbias.wp.horizon.ac.uk/fairness-toolkit/. All components of the Toolkit are freely available to download and print from the UnBias website under Creative Commons license (CC BY-NC-SA 4.0). This output has elicited lots of interest from different sectors National and International: education, third sectors organisations, NGOs, public and private sector, academia, policy development, regulatory bodies, media, online industry and the general public. By the 12th February 2019, 194 decks of Awareness Cards have been given away and/or requested from a wide range of organisations including UNICEF, Mozilla, Microsoft, IBM, Welcome Trust, House of Lords, ISOC UK, national and international NGOs and more than 20 national and international universities, and British schools and academies, etc. 
Type Of Art Artefact (including digital) 
Year Produced 2018 
Impact The toolkit has been co-created with young people- so right from the outset of its development, it helped to stimulate discussion surround fairness and safety online. We have also had the opportunity to revisit some of the young people who were involved in the creation of the fairness toolkit, and host workshops with them to allow them to interact with the fairness cards. This has been received well, and again there has been evidence of good work. The most popular components of this Toolkit so far have been the TrustScapes and the Awareness Cards. We have run more than 20 workshops using these elements and have been very welcomed by participants from different sectors, backgrounds and interests as well as different age groups ranging from 13 to 65 years of age and older. Even though this output was designed to raise awareness and stimulate debate, discussion and critical thinking on issues of online algorithm fairness, they were also design for civic thinking around more general issues of fairness/ethics beyond the online sphere. Feedback we have received so far from people that have used this tool it's been extremely positive. In particular, our research teams working in the UnBias and currently in the ReEnTrust projects (both EPSRC funded) have been using the TrustScapes and Awareness Cards in a series of different workshops and public engagement events. Participants have found them fun and very useful as an educational tool as well as to aid debate in very topical issues related to online fairness and to reflect on the impact of algorithm bias on internet users' experiences. In addition to the extensive programme of user focus groups we have run within UnBias and will run in our current project ReEnTrust, to reach broader audiences, raise awareness and elicit public views on online fairness as well as trust in algorithms across a broad range of individuals, we have and will continue using this Toolkit in public engagement events such as MozFest, the ESRC Festival of Social Science, STEMcity Festival of Science and Curiosity, the annual Women in Computer Science day at Oxford, and other local science and technology festivals in the UK. We believe it is critical that we investigate issues of online fairness giving internet users a supportive space to share their views and provide recommendations on what changes should be done to have better user's online experiences. In particular we are very pleased to make vulnerable users voices - from young people and older adults- heard! and promote change on online policy and regulatory affairs. There is great interest and potential for future use and deployment for the toolkit. 
URL https://unbias.wp.horizon.ac.uk/fairness-toolkit/
 
Description The UnBias Youth Juries methodology introduced to promote digital citizenship led to identifying:
1. The existence of concerns over the contemporary prevalence algorithm driven online platforms
2. A desire for change to improve the user experience of these platforms
3. Opportunities for change - we identified capacity to address unfairness in algorithmic systems through various means, in particular via education, engagement and policy change.
Exploitation Route Our contributions towards policy recommendations include the co-authoring of a report for the European Parliament Science Technology Options Assessment to support a governance framework for algorithmic accountability and transparency. Submitting research findings to the UK Information Commissioners Office Call for Age Appropriate Design Code led to supporting Revealing Reality (the Agency commissioned with developing the code).The code has now been published by the ICO (Jan 2020) setting out standards expected of those responsible for designing, developing or providing online services.

We have submitted research evidence to UK Parliamentary inquiries.

Our contributions towards standards setting involve chairing the IEEE P7003 Standard for Algorithmic Bias Considerations Working Group and participating (through the British Standards Institute) in work on setting Standards for Artificial Intelligence by ISO/IEC.

Open source resources - Youth Jury Open Educational Resources, Fairness Toolkit, Video (Our Future Internet - free and fair for all) could be put to use by IT teachers and parents to help understand algorithmic processes in daily life . Requests for resources have been received by a wide range of global organisations including UNICEF, IBM, Internet Society UK, House of Lords etc.
Sectors Digital/Communication/Information Technologies (including Software),Education,Government, Democracy and Justice,Security and Diplomacy
URL http://unbias.wp.horizon.ac.uk/
 
Description The project has led to input to UK and EU government inquiries on topics related to the role of algorithmic accountability, transparency and in decision making processes, the Internet and whether to regulate or not, the impact of algorithmic UnBias and we are engaged in discussions around Baroness Kidron's amendments to the UK DPA around age appropriate design, whilesocial media and screen use on young peoples mental health and well-being and fake news. Evidence submitted in response to the Information Commissioner Office relating to an Age Appropriate Design Code Call led to supporting the work of Baroness Kidron and the 5Rights Foundation to develop the Code. The ICO published the age appropriate design code - as set of 15 standards that online services should meet to protect children's privacy in January 2020. This work is ongoing. Reserach findings fed into three successful grant applications - Telling Tales of Engagement (2017) , Impact Exploration award (2018) and ReEnTrust (2018) to further work in this area. This research project presented opportunities to network and build a strong stakeholder representation with expertise crossing professional, industry, policy, education, NGOs and legislative institutions. This led to an opportunity to Chair the IEEE P7003 working group developing a new standard for algorithmic bias considerations. This work is ongoing. Open source resources have been used at a number of out reach events among varied audiences, have been shared and made publicly available for sustainability beyond the lifecycle of the project.
First Year Of Impact 2017
Sector Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Education,Government, Democracy and Justice
Impact Types Societal,Policy & public services
 
Description AlgoAware workshop. Ansgar Koene, Brussels
Geographic Reach Europe 
Policy Influence Type Participation in a guidance/advisory committee
Impact Contribution to high-level expert discussion with representatives from the European Parliament, the European Commission as well as Civil Society. The recent "State of the Art" report on automated decision-making by AlgorithmWatch (https://www.algoaware.eu/state-of-the-art-report/) was launched on this event. The UnBias project gets mentioned as examples of an initiative by academic research groups to develop Policy and technical tools (page 10), in particular with the UnBias "Fairness Toolkit" as a "clear example". The report also includes a summary of the aims of the UnBias project (page 57) and the elements of the "Fairness Toolkit" (with references to the relevant webpages).
URL https://www.algoaware.eu/2018/11/14/29-jan-2019-automating-society-taking-stock-of-automated-decisio...
 
Description CDEI Landscape Summary: Bias in Algorithmic Decision Making cited in Royal United Services Institute for Defence and Security Studies Briefing Paper
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://rusi.org/sites/default/files/20190916_data_analytics_and_algorithmic_bias_in_policing_web.pd...
 
Description Citation of "A governance framework for algorithmic accountability and transparency" in draft EU Committee on Legal Affairs report
Geographic Reach Europe 
Policy Influence Type Citation in other policy documents
URL https://www.europarl.europa.eu/doceo/document/JURI-PR-650508_EN.pdf
 
Description Comments on the European Data Protection Board's Guidelines 4/2019 on Article 25 Data Protection by Design and by Default
Geographic Reach Europe 
Policy Influence Type Contribution to a national consultation/review
URL https://nottingham-repository.worktribe.com/preview/3774957/comments_on_edpb_guidelines_on_a_25_dpbd...
 
Description Dr Ansgar Koene invited as Chair for IEEE Computer Society AI Standards Committee
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
URL https://www.computer.org/publications/technews/insider/ai-standards-kick-off
 
Description Dr Ansgar Koene invited by UNESCO to contribut towards the Virtual Regional Consultation on the 1st draft of Recommendation on the Ethics of Artificial Intelligence (AI) on 27028 July 2020
Geographic Reach Europe 
Policy Influence Type Contribution to a national consultation/review
URL https://events.unesco.org/event?id=2763453844&lang=1033
 
Description Dr Ansgar Koene invited by the Chartered Governance Institute UK & Ireland to participate in online webinar contributing towards CPD
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
URL https://www.cgi.org.uk/events/networking-and-cpd-events/cpd-events/ai-and-boardroom-technology
 
Description Dr Ansgar Koene invited to join an Ethics Committee at MOSIP
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
URL https://www.mosip.io/
 
Description Dr Ansgar Koene invited to present to the IE Big Data & AI Club, Spain
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
URL https://ieconnects.ie.edu/iebigdata/blog/
 
Description Dr Ansgar's report for the EU entitled Governance frameworks for Algorithmic Accountability and Transparency is cited in the European Parliament briefing report EU guidelines on ethics in artificial intelligence: Context and Implementation
Geographic Reach Europe 
Policy Influence Type Citation in other policy documents
URL https://www.europarl.europa.eu/thinktank/en/document.html?reference=EPRS_BRI(2019)640163
 
Description Dr Koene invited to be on the Advisory Committee of the proposal FINDER, WSSC
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
URL https://www.upf.edu/web/wssc/
 
Description Dr Koene invited to contribute towards UK Government funded AI R&D project
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
URL https://www.gov.uk/government/news/understanding-the-uk-artificial-intelligence-commercialisation
 
Description Dr Koene invited to participate in a Digital Regulation Cooperation Forum Workshop
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
URL https://www.gov.uk/government/collections/the-digital-regulation-cooperation-forum
 
Description Dr Koene invited to participate in an online workshop on December 14th 2020 to Bath University CDT students entitled AI related standards for Bath's CDT in Accountable Responsible and Transparent AI. Dr Koene presented: • Algorithmic Bias with Ansgar Koene, Working group chair for IEEE Standard on Algorithm Bias Considerations (P7003)
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
URL https://cdt-art-ai.ac.uk/news/events/the-global-ai-standards-landscape-an-extended-seminar/
 
Description Dr Koene invited to participate in the Open Ethics Series - Human in the loop AI - governance agency and oversight. Open Ethics Series is the chain of educational events around impact of artificial intelligence. The first Series aims to cover the main principles emerging from the broad range of discussions on requirements for development and deployment of AI. The topics will include those that are part of the Ethics guidelines for trustworthy AI by European Commission (Privacy & data governance, Human agency & oversight, Robustness & safety, Transparency, Fairness, Societal & environmental well-being, Accountability), as well as additional topics proposed by the community.
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
URL https://www.eventbrite.com/e/human-in-the-loop-ai-gency-oversight-open-ethics-series-s01e06-registra...
 
Description Dr Koene presented 'AI ethics and regulation for a trustworthy AI icosystem' to the Institute of Engineering and Technology IET (Feedback: Ansgar Koene was a superb speaker. His presentation was very well structured and very interesting. Following every word from beginning to end was easy. I work now in Financial Markets with interest in regulation. All of the content was relevant to the challenges faced by regulators and regulated firms in Financial Services. An excellent presentation from a very knowledgeable expert. I learnt a lot from this lecture, and took away plenty of sources of further research.
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
URL https://events.theiet.org/events/ai-ethics-and-regulation-for-a-trustworthy-ai-ecosystem/
 
Description EPSRC - UKRI Artificial Intelligence and Public Engagement workshop
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Enhancing DCMS public consultation on reforms to the UK's data protection - Policy Connect Roundtable (Dr Ansgar Koene)
Geographic Reach National 
Policy Influence Type Participation in a guidance/advisory committee
 
Description Evidence submitted to CMA market study interim report on Online Platforms and Digital Advertising
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL https://www.gov.uk/cma-cases/online-platforms-and-digital-advertising-market-study
 
Description Evidence to UK Parliament inquiry. Fake News - Koene A.
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/culture-media-and-s...
 
Description Input of research evidence towards a POSTNote Online Safety Education
Geographic Reach National 
Policy Influence Type Implementation circular/rapid advice/letter to e.g. Ministry of Health
URL https://researchbriefings.parliament.uk/ResearchBriefing/Summary/POST-PN-0608
 
Description Input of research evidence towards a POSTNote Online Safety Education, UK Parliament
Geographic Reach National 
Policy Influence Type Implementation circular/rapid advice/letter to e.g. Ministry of Health
URL https://researchbriefings.parliament.uk/ResearchBriefing/Summary/POST-PN-0608
 
Description Joint letter of response (2ith 5 Rights) to FTC regarding COPPA Rule
Geographic Reach Multiple continents/international 
Policy Influence Type Implementation circular/rapid advice/letter to e.g. Ministry of Health
Impact Children's Online Privacy Protection Act TAGS: Consumer Protection MISSION: Consumer Protection LAW: 15 U.S.C. §§ 6501-6506 LINK: http://uscode.house.gov/view.xhtml?req=granuleid%3AUSC-prelim-title15-section650... This Act protects children's privacy by giving parents tools to control what information is collected from their children online. The Act requires the Commission to promulgate regulations requiring operators of commercial websites and online services directed to children under 13 or knowingly collecting personal information from children under 13 to: (a) notify parents of their information practices; (b) obtain verifiable parental consent for the collection, use, or disclosure of children's personal information; (c) let parents prevent further maintenance or use or future collection of their child's personal information; (d) provide parents access to their child's personal information; (e) not require a child to provide more personal information than is reasonably necessary to participate in an activity; and (f) maintain reasonable procedures to protect the confidentiality, security, and integrity of the personal information. In order to encourage active industry self-regulation, the Act also includes a "safe harbor" provision allowing industry groups and others to request Commission approval of self-regulatory guidelines to govern participating websites' compliance with the Rule. Link to call for public comment by FTC: https://www.regulations.gov/document?D=FTC-2019-0054-0001
URL https://uscode.house.gov/view.xhtml?req=granuleid%3AUSC-prelim-title15-section6501&edition=prelim
 
Description Letter of response to the UN Human Rights Council Advisory Committee, on behalf of the Advisory Committee -"New and emerging digital technologies and human rights". call for information from Member States and other stakeholders, including international organizations, national human rights institutions, non-governmental organizations, private sector, technical community and academic institutions on new and emerging digital technologies and human rights.
Geographic Reach Multiple continents/international 
Policy Influence Type Implementation circular/rapid advice/letter to e.g. Ministry of Health
URL https://www.ohchr.org/EN/HRBodies/HRC/AdvisoryCommittee/Pages/HRCACIndex.aspx
 
Description Marina Jirotka invited to sit on the Advisory Board of The Society for Computers and Law
Geographic Reach Multiple continents/international 
Policy Influence Type Membership of a guideline committee
 
Description Marina Jirotka invited to sit on the advisory board of APPG on Data Analytics - Data and Technology Ethics Inquiry
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
 
Description NSF future scoping workshop on Trustworthy Algorithmic Decision-Making
Geographic Reach North America 
Policy Influence Type Contribution to a national consultation/review
URL http://trustworthy-algorithms.org/whitepapers/
 
Description Orbit publication "Editorial responsibilities arising from personalisation algorithms"
Geographic Reach Europe 
Policy Influence Type Citation in other policy documents
URL https://www.europarl.europa.eu/stoa/en/document/EPRS_STU(2019)624262
 
Description Response to Commons Science and Technology Committee "Impact of social media and screen-use on young people's health" inquiry (SMH0131). McAuley D., Koene A., Perez E-V., Portillo V., Creswick H., Dowthwaite L. and Cano M-G.
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/science-and-technol...
 
Description Response to House of Commons inquiry. Algorithms in Decision Making - McAuley D., Koene A., Perez E-V., Jirotka M., Patel M. & Webb H.
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/science-and-technol...
 
Description Response to House of Lords Select Communications Committee "The Internet - To Regulate or Not To Regulate?" inquire (IRN0038). McAuley D, Koene A, and Urquhart L.
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/communications-comm...
 
Description Response to House of Lords inquiry - what are the implications of Artificial Intelligence
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/artificial-intellig...
 
Description Response to Information Commissioner's Office (ICO) Age Appropriate Design Code call for evidence.
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL https://ico.org.uk/media/2260169/horizon-digital-economy-research-insitute-university-of-nottingham....
 
Description Science Technology Options Assessment report for the European Parliament: A Governance Framework for Algorithmic Accountability and Transparency
Geographic Reach Europe 
Policy Influence Type Contribution to a national consultation/review
URL http://www.europarl.europa.eu/ep-live/en/other-events/video?event=20181025-0930-SPECIAL
 
Description Submission to UK Government call for evidence - The right to privacy: digital data
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
URL https://nottingham-repository.worktribe.com/output/7466129
 
Description USACM Panel on Algorithmic Transparency and Accountability
Geographic Reach North America 
Policy Influence Type Membership of a guideline committee
URL https://www.acm.org/public-policy/algorithmic-panel
 
Description UnBias Fairness toolkit cited as a resource to raise awareness and facilitate dialogue around algorithmic bias and fairness (pages 29 & 30)
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/8190...
 
Description UnBias project cited in policy brief for the All Party Parliamentary Group on Data Analytics: rebuilding and enhancing trust in algorithms
Geographic Reach National 
Policy Influence Type Citation in other policy documents
URL https://www.policyconnect.org.uk/appgda/news/apgda-and-reentrust-host-webinar-rebuilding-and-enhanci...
 
Description Biomedica Research Centre fuding
Amount £24,322 (GBP)
Organisation University of Leicester 
Department NIHR Biomedical Research Centre
Sector Hospitals
Country United Kingdom
Start 01/2018 
End 01/2019
 
Description Digital Rights
Amount £57,600 (GBP)
Funding ID TLEF 2017 001045 
Organisation The Legal Education Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2018 
End 04/2019
 
Description EP STOA
Amount € 14,948 (EUR)
Funding ID EPRS/STOA/SER/18/201N 
Organisation European Union 
Department European Parliament Research Services
Sector Public
Country Belgium
Start 03/2018 
End 10/2018
 
Description EPSRC TIPS Network+
Amount £1,298,811 (GBP)
Funding ID EP/R045178/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2018 
End 08/2021
 
Description EPSRC- Impact Exploration Grant
Amount £10,000 (GBP)
Funding ID EP/R511730/1 
Organisation University of Nottingham 
Sector Academic/University
Country United Kingdom
Start 09/2018 
End 07/2019
 
Description Horizon Service Campaign
Amount £90,000 (GBP)
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 09/2018 
End 09/2019
 
Description Investigating novel methods and best practices for service design and patient safety in the Mexican healthcare service,
Amount £50,000 (GBP)
Organisation University of Nottingham 
Sector Academic/University
Country United Kingdom
Start 05/2018 
End 08/2019
 
Description QR funding fro evidence-based policy making
Amount £4,000 (GBP)
Organisation University of Nottingham 
Sector Academic/University
Country United Kingdom
Start 12/2019 
End 03/2020
 
Description RPA Digital Futures About Algorithms and beyond
Amount £30,000 (GBP)
Organisation University of Nottingham 
Sector Academic/University
Country United Kingdom
Start 04/2017 
End 05/2018
 
Description RPA Rights and Justice Access to justice in a digital age: suspects' legal rights
Amount £15,000 (GBP)
Organisation University of Nottingham 
Sector Academic/University
Country United Kingdom
Start 04/2017 
End 09/2018
 
Description Smart City Project
Amount £8,000 (GBP)
Organisation University of Nottingham 
Sector Academic/University
Country United Kingdom
Start 05/2017 
End 05/2018
 
Description Telling Tales of Engagement 2017 award.
Amount £10,000 (GBP)
Organisation RCUK Digital Economy Theme (DET) 
Sector Public
Country United Kingdom
Start 05/2018 
End 11/2019
 
Description UnBias Fairness Toolkit: Usage and Impact
Amount £10,000 (GBP)
Organisation University of Nottingham 
Sector Academic/University
Country United Kingdom
Start 01/2018 
End 08/2018
 
Description eNurture Network+ Promoting Young People's Mental Health in a Digital World
Amount £1,249,969 (GBP)
Funding ID ES/S004467/1 
Organisation United Kingdom Research and Innovation 
Sector Public
Country United Kingdom
Start 09/2018 
End 08/2022
 
Description Chair of IEEE-SA P7003 Standard: Algorithmic Bias Considerations - Dr Ansgar Koene 
Organisation Institute of Electrical and Electronics Engineers (IEEE)
Country United States 
Sector Learned Society 
PI Contribution I proposed the P7003 standard for Algorithmic Bias Considerations as part of the IEEE Global Initiative for Ethics of Autonomous and Intelligent systems and now chair the IEEE working group for the development of the P7003 Standard for Algorithmic Bias Considerations. The work involves facilitating a monthly conference call, promoting the work to attract working group participants and providing the general outline for the standards documents. Liz Dowthwait from the UnBias/ReEnTrust project is the P7003 secretary, supporting my work by helping with the monthly agenda and conference minutes.
Collaborator Contribution The IEEE Standards Association (IEEE-SA provides the liaison officer to assist with coordination of the P7003 working group with the wider IEEE Standards activities. IEEE-SA also provides the web-conferencing facilities (Join.me) and online document hosting/collaboration space (iMeet). The IEEE Global Initiative has been facilitating media engagement and policy engagement activities (e.g. ACM/IEEE panel on Algorithmic Transparency and Accountability in Washington DC where I was a panel member). From September 30 to October 3rd, IEEE-SA hosted a workshop for the P70xx standards working groups in Berlin (IEEE-SA paid for travel and accommodation).
Impact Dr Ansgar Koene was contacted by the ACM to participate in a panel discussion on Algorithmic Transparency and Accountability in Washington DC on 14th September 2017. https://www.acm.org/public-policy/algorithmic-panel. There was a write-up of the panel discussion, with follow-up interview with Ansgar "IEEE and ACM Collaborations on ATA", published in AI Matters: A Newsletter of ACM SIGAI, on 1 October 2017. Ansgar's role as chair and a summary of the P7003 Standard were mentioned in the article: "The Ethics of Artificial Intelligence for Business Leaders - Should Anyone Care?", in TechEmergence, on 9th December 2017. Ansgar was invited to publish a short paper on the IEEE P7003 activity in IEEE Technology and Society Magazine, "Algorithmic Bias: Addressing Growing Concerns", IEEE Technology and Society Magazine, Volume: 36, Issue: 2, June 2017. [DOI: 10.1109/MTS.2017.2697080]. An interview with Ansgar about P7003 was published in The Institute (The IEEE news source), on 12 September 2017, "Keeping Bias From Creeping Into Code".
Start Year 2017
 
Description ISCO-UK "User Trust" 
Organisation Internet Society (ISOC)
Country United States 
Sector Charity/Non Profit 
PI Contribution In collaboration with ISOC-UK the UnBias team has run two workshop hosted by ISCO-UK. The first workshop was a panel discussion on "Multi-Sided Trust in Multi-Sided platforms" https://unbias.wp.horizon.ac.uk/2018/04/13/isoc-uk-horizon-der-panel-for-multi-sided-trust-on-multi-sided-platforms/ The second workshop was a demonstration of the UnBias "Fairness Toolkit" with a session using the Awareness Cards https://unbias.wp.horizon.ac.uk/2018/11/23/workshop-on-algorithmic-awareness-building-for-user-trust-in-online-platforms/
Collaborator Contribution The ISOC-UK partner published the workshop to its members, and hosted the workshop events.
Impact The collaboration is multi-disciplinary due to the nature of ISCO as a multi-disciplinary membership organization with members from a wide range of industry, civil-society and academic groups who are interested in an open and free internet
Start Year 2018
 
Description Secretary of IEEE-SA P7003 Standard: Algorithmic Bias Considerations 
Organisation Institute of Electrical and Electronics Engineers (IEEE)
Country United States 
Sector Learned Society 
PI Contribution I support the work of the P7003 standard as part of the IEEE Global Initiative for Ethics of Autonomous and Intelligent systems by organising and minuting the monthly working group meetings. The group (EEE working group for the development of the P7003 Standard for Algorithmic Bias Considerations) is chaired by Ansgar Koene, also a member of the UnBias project.
Collaborator Contribution The IEEE Standards Association (IEEE-SA) provides the liaison officer to assist with coordination of the P7003 working group with the wider IEEE Standards activities. IEEE-SA also provides the web-conferencing facilities (WebEx/join.me) and online document hosting/collaboration space (iMeet). The IEEE Global Initiative has been facilitating media engagement and policy engagement activities.
Impact Minutes of all meetings are available at http://sites.ieee.org/sagroups-7003/
Start Year 2017
 
Description "Civic Tech 101": civic engagement collaboration with Nottingham City Council 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Policymakers/politicians
Results and Impact About 25 people attended a pre-session meeting followed by a live Skype and Q&A conference with Christopher Whitaker, the organiser of Chi Hack Night (Chicago's weekly event to build, share and learn about civic technology). The purpose of the event was to provide some insights into how civic technology could be used in Nottingham to produce online resources that will make complex and important information accessible to Nottingham citizens.

This event brought a wide range of stakeholders together (see "who were the audiences" section, above) aiming to create a multidisciplinary collaborative team. The members for the "Civic Tech 101" Steering Group will be decided on the next meeting (20th March 2017).
Year(s) Of Engagement Activity 2017
 
Description "Interacting with different web browsers & search engines - Do you care?" (Explorers Fair - Festival of Science and Curiosity, Nottingham) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Seventy pupils attended this drop-in activity at the Explorers Fair Expo, as part of Nottingham Festival of Science and Curiosity (10th February).
Sessions involved hands-on exercises where students, working in pairs, compared results obtained when using different browsers and/or search engines. Participants engaged and discussed about their online preferences and/or concerns regarding: algorithm bias, filtering systems, fairness and privacy issues.

At the end of the sessions, networking with teachers led to establish links with schools to promote other current research activities within our research team (e.g., UnBias Youth Juries). Also, this activity contributed to develop strong links with the organisers of the Science and Curiosity Festival, which takes place annually in Nottingham, and to express our interest to take part in future events.
Year(s) Of Engagement Activity 2017
URL http://unbias.wp.horizon.ac.uk/event/explorers-fair-expo/
 
Description "Logan Paul's dead body video reinforces call for better child protection online", in the Conversation 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Elvira to add
Year(s) Of Engagement Activity 2017
URL https://theconversation.com/logan-pauls-dead-body-video-reinforces-call-for-better-child-protection-...
 
Description "Machine gaydar: AI is reinforcing stereotypes that liberal societies are trying to get rid of", in the Conversation 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Impact: Reposted on Digital Leaders blog, and Medium page on 29th November 2017.
Year(s) Of Engagement Activity 2017
URL https://theconversation.com/machine-gaydar-ai-is-reinforcing-stereotypes-that-liberal-societies-are-...
 
Description "What kind of internet do children and young people want?" NSPCC annual conference, London. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Third sector organisations
Results and Impact Poster presentation to the "How safe are our children?" conference. Our results sparked interesting questions and discussion during the poster session. Networking with other researchers in the field as well as members of third sectors organisations was relevant for our research project. It was a very inspiring conference whose participants included children sharing their online experiences with the audience, as well as key researchers and members of the public sector including Prof. Sonia Livingstone (LSE), Alexandra Evans (5Rights foundation) and Matt Hancock (Secretary of State for Digital, Culture, Media and Sport).
Year(s) Of Engagement Activity 2018
URL https://unbias.wp.horizon.ac.uk/tag/conference
 
Description "Who is in charge? You or the algorithm?" A. Koene (Explorers Fair- Festival of Science and Curiosity, Nottingham) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact This was a general talk to reflect about how search engines, social networks, news recommender systems, etc., select which information is relevant for the user. The main purpose of this event was to raise awareness of how the internet works in a "language" for all ages. Being part of a Science Festival event for families, the audience was quite broad (from 9 year olds to teenagers, accompanied by their parents).

The audience participated answering and making questions during and at the end of the presentation.
Year(s) Of Engagement Activity 2017
URL http://unbias.wp.horizon.ac.uk/2017/02/19/unbias-public-engagement-at-the-explorers-fair/#more-368
 
Description "they don't really listen to people". Young people's concerns and recommendations for improving online experiences.Helen Creswick, Liz Dowthwaite, Ansgar Koene, Elvira Perez Vallejos, Virginia Portillo, Monica Cano and Christopher Woodard 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This presentation highlighted young people (13-17 years old) concerns in relation to online issues of algorithm bias. In particular users' lack of agency and feelings of disempowerment in young people's internet use, exacerbated by their experiences of online terms and conditions.The audience engaged with interesting questions about the applicability of these findings to the adult population.
Researchers were invited to submit an enhanced version of the research paper to a special issue of the Journal of Information, Communication and Ethics in Society, which has been recently accepted for publication but is not available online yet.
Year(s) Of Engagement Activity 2018
URL https://easychair.org/smart-program/ETHICOMP2018/2018-09-24.html#talk:78107
 
Description A second Workshop to develop UnBias Facilitator Toolkit 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact This workshop was run with 18-29 years, 30-50 years and the over 65s as part of an EPSRC Impact Exploration Grant that was awarded to the UnBias team in Nottingham. The workshops enabled the team to run sessions using the cards and to receive feedback on them towards development of the Facilitator booklet. Each group was also asked to co-design games that could be used to help people to interact with the Awareness Cards and to learn about issues of online fairness.
Year(s) Of Engagement Activity 2019
URL https://unbias.wp.horizon.ac.uk/2019/07/21/unbias-awareness-cards-facilitator-booklet/
 
Description ALGORITHMIC DISCRIMINATION: ARE YOU IN OR OUT? 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Blog about algorithm bias and possible discrimination to the users, posted in UnBias website.
Year(s) Of Engagement Activity 2016
URL http://unbias.wp.horizon.ac.uk/2016/10/21/algorithmic-discrimination-are-you-in-or-out/
 
Description Accessible Fun Day- Festival of Science and Curiosity, Nottingham 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Other audiences
Results and Impact Hands-on activity for the general public but in particular for a SEND audience, aimed to bring awareness and to reflect on how personal data is used by online platforms. It was the first time UnBias took part of an activity SEND audience-friendly. Children from different ages and learning abilities, parents, grandparents and careers engaged with creative activities which sparked very interesting discussions and genuine interest from the public about our research topic. This activity received good feedback by online social networking (e.g., on twitter).
Year(s) Of Engagement Activity 2017
 
Description Accountability and regulation of algorithms. EuroDIG2017 conference, Estonia. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact As part of the EuroDIG (European Dialogue on Internet Governance) conference UnBias organized a 30minute 'flash session' on Accountability and Regulation of Algorithms. This session was held on the first day of the conference. EuroDig is a multi-stakeholder dialogue forum bringing together civil-society, government, academic and industry to discuss issued related to the future on the internet. In keeping with the spirit of this conference the UnBias led session consisted of a brief introduction of the issues of algorithmic bias followed by a lively discussion involving representatives from UNICEF, Google, Internet Society, European Commission and various others. Following this session a member of the DG connect team at the European Commission requested to join the UnBias stakeholder group.
Year(s) Of Engagement Activity 2017
URL https://eurodigwiki.org/wiki/Accountability_and_regulation_of_algorithms_-_Flash_02_2017
 
Description Algorithm Mediated Online Information Access (AMOIA) - user trust, transparency, control and responsibility. WebSci17, NY. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This activity was part of an ongoing series of stakeholder workshops we are running for UnBias to discuss fairness, and other ethics issues, in algorithmic decision making. This particular instance of the workshop series was a workshop help on the first day of the WebScience conference in Troy, New York, USA. Participants were other academic attendees of the Webscience conference. The event involved a lively discussion between participants, which was voice-recorded for future analysis (participants gave written consent) and provided an interest US perspective on the issues we had previously discusses with participants in the UK. Following the workshop a number of participants joined the UnBias stakeholder group to keep up to date on future activities of the project.
Year(s) Of Engagement Activity 2017
URL https://unbias.wp.horizon.ac.uk/event/amoia-workshop-at-acm-web-science-2017/
 
Description Algorithm Workshop, Held at the TIC, Strathclyde, February 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Lilian Edwards organised an Algorithm Workshop on 27th February at the Technology Innovation Centre (TIC) at Stathclyde for the benefit of Internet Law and Policy Masters students, and Internet law Honours students, as well as invited guests from law, technology and business and public attendees. The event was a sell out with 300 attending and created much interest on social media.
The workshop considered the legal, social and technical dimensions of algorithms and their role in governance. As our everyday lives are more and more determined by decisions made by automated algorithmic processes, the status of algorithms and how they can be regulated , queried and countermanded, becomes evermore important. To give just a few examples, algorithms now often determine how we are assessed as risks for policing and terrorism surveillance: how we get hired, fired and promoted; and what news and political opinion we are exposed to. Both personal and societal welfare and arguably, the future of democracy itself, thus depend on being able to regulate and make accountable the algorithm society. In particular a major current question is if a right to explanation of what an algorithm has decided exists in data protection law? Lilian spoke on this in her contribution: ""Slave to the Algorithm: War Stories and Legal Remedies".
We hope the event will be a "starter seed" for CILP and partners (including Edinburgh and Horizon DEC, Nottingham) to investigate this fascinating topic further and we will be seeking major grant funding. ax 120000 characters)
Year(s) Of Engagement Activity 2017
URL http://www.strath.ac.uk/humanities/lawschool/newsevents/algorithmworkshopfebruary2017/
 
Description Algorithmic Decision-Making. European Big Data Value Forum, Versailles. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Ansgar Koene presented an invited talk on the ongoing activities of the UnBias project, and the related work of the IEEE P7003 Standard for Algorithmic Bias Considerations working group he chairs. This presentation was part of the Data & Society panel session on the 2n day of the conference. The session was attended by about 200 people consisting of industry, academia, government and NGO attendeed of the European Big Data Values forum. Following the presentation there was a panel discussion. At the end of the session Ansgar was approached by a number of audience members to request future contact by a professor from Télécom Ecole de Management and a person from the New York based NGO NYSci [https://nysci.org/NYSci] regarding our work of engaging with young people on their experience and recommendations related to online algorithmic services.
Year(s) Of Engagement Activity 2017
URL http://www.european-big-data-value-forum.eu/program/data-society/#algorithmicdecisionmaking
 
Description Algorithmic awareness building for User Trust in online platforms. ISOC-UK England, London 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Third sector organisations
Results and Impact Interactive session aimed at exploring awareness building around the use of algorithms in online platforms, through the use of the UnBias Awareness Cards. Activities resulted in critical and civic thinking for exploring how decisions are made by algorithms, and the impact that these decisions may have on our lives and the lives of others. Participants engaged in interesting discussions and extra decks of cards were requested by some attendees.
Year(s) Of Engagement Activity 2018
URL https://unbias.wp.horizon.ac.uk/2018/11/23/workshop-on-algorithmic-awareness-building-for-user-trust...
 
Description Algorithmically Mediated Online Information Access (AMOIA) discussion workshop at Mozfest 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Third sector organisations
Results and Impact This activity was a discussion dialogue, with participants of the Mozfest event in London, on the topic of fairness in Algorithmic decisions making related to Algorithmically Mediated Access to Online Content (e.g. search engines, social media recommendation feeds etc). Participants included civil-society NGO from Germany (Algorithm Watch), Liberians from US and Sweden, London based data activists and general audience. Activity mentioned by Mozfest organisers in the Medium blog.
Year(s) Of Engagement Activity 2017
URL https://medium.com/mozilla-festival/decentralisation-at-the-mozilla-festival-aad3e33e1443
 
Description Ansgar Koene featured speaker at Purdue conference: "Policies for Progress" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact The Purdue Policy Research Centre at Purdue University, held to "Policies for Progress" conference exploring ways to bring together policymakers, industry leaders, not-for-profits, and academics to bring their collective expertise to bear to address Wicked problems."Policies for Progress" was the capstone event for the Breaking Through: Developing Multidisciplinary Solutions to Global Grand Challenges research project funded by The Andrew W. Mellon Foundation.

Experts from four multidisciplinary teams shared findings and results of their groundbreaking work on the Breaking Through research project. Stakeholders who were integrated into these projects discussed the successes, benefits, as well as challenges in partnering with academia.

In recognition of our extensive work on multi-stakeholder engagement in our past and current projects Ansgar Koene was invited to speak about the work we did on the UnBias project, the associated IEEE P7003 Standard for Algorithmic Bias Considerations development and our current activities for ReEnTrust.
Year(s) Of Engagement Activity 2019
URL https://www.purdue.edu/breaking-through/
 
Description Approach from BBC Bitesize to contribute research findings/resources to a Fake News Campaign 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact We received a request from the Assistant Producer of BBC Bitesize Campaigns:Hi Helen - I'm emailing from BBC Bitesize where I am part of a team developing a new campaign on Fake News for secondary school pupils. I was reading about the multitude of projects in this area hailing from Nottingham and would love to talk to you about them. In particular, I was keen to know more about your work aimed at 3-13 yr olds and if this was to be expanded maybe? As we plan content for the campaign it would be great to talk to you about possibly getting involved in some way given your experience and expertise.

A teleconference and then a further f/f meeting took place with the Producer of the Campaign who was interested in our work on algorithms and the psychological effects of social media. Campaign due to run from March 2020 for 6 months
Year(s) Of Engagement Activity 2020
 
Description Approached to provide input towards a media intervention from Yvette (March 2020) being led by the Fabian Society (regarding their Commission on Workers and Technology 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Dr Koene was approached by Fabian Society regarding their Commission on Workers and Technology, a 2-year policy inquiry looking at how best policy can ensure technology change proves to be an opportunity rather than a threat for typical workers over the next decade. The Commission is chaired by Yvette Cooper MP.
They were preparing a media intervention from Yvette (March 2020) which specifically addressed technology driven discrimination in the world of work including the incidence of algorithmic bias in various parts of the employment relationship. .

"As one of the authors of CDEI's 'landscape summary' of bias in algorithmic decision-making, I was wondering whether you would be happy to take a look at our draft note, so we can get your thoughts about whether we are on the right track in our thinking when it comes to recommendations"
Year(s) Of Engagement Activity 2020
 
Description Article in UK media: We asked young people what they want from the internet of the future - here is what they said 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Impacts: Reposted on the Independent website on April 1st 2017 and on the Digital Leaders blog and Medium page on November 28th 2017
Year(s) Of Engagement Activity 2017
URL https://uk.news.yahoo.com/asked-young-people-want-internet-112933237.html
 
Description BBC Click - 'It's Been Emotional' (about Digiplay workshop at Techtopia, Polka Theatre London) by Liz Dowthwaite 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Interview to UnBias researcher that helped to create and run a workshop at the Techtopia Festival. The aim was through play, teaching kids about data and how algorithms shape the world online.
Year(s) Of Engagement Activity 2018
URL https://www.bbc.co.uk/iplayer/episode/b0b4kg9d/click-its-been-emotional
 
Description BBC Nottingham Drive Time Radio interview on NSPCC proposal for algorithmic detection of online grooming 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact While visiting the BBC Nottingham studio for the TV interview regarding the proposal by the NSPCC that the government should mandate that Social Media companies use algorithms to detect if a child is being approached online as part of sexual grooming, Ansgar was asked to also provide expert commentary to contribute to the BBC Nottingham Drive Time Radio broadcast at 3:30pm on January 29th.
Year(s) Of Engagement Activity 2018
 
Description BBC Salford symposium 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Horizon and Databox presentation to BBC around privacy preserving personalised services
Year(s) Of Engagement Activity 2018
 
Description BBC1 News (East Midlands) - TV interview regarding NSPCC proposal for algorithmic detection of online grooming 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Following the proposal by the NSPCC that the government should mandate that Social Media companies use algorithms to detect if a child is being approached online as part of sexual grooming, the BBC East Midlands contacted Ansgar to ask for expert commentary regarding the technical feasibility of this proposal. The interview was broadcast as part of the 6:30pm and 10:30pm BBC1 regional news for East Midlands on January 29th.
Year(s) Of Engagement Activity 2018
URL https://www.bbc.co.uk/iplayer/episode/b09pr6zm/east-midlands-today-evening-news-29012018
 
Description Blog post by Liz Dowthwaite 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Towards the end of July we had another paper published from the results of the UnBias project in the Journal of Information, Communication and Ethics in Society, entitled "They don't really listen to people": Young people's concerns and recommendations for improving online experiences: https://dowthwaite.wordpress.com/2019/08/12/belated-new-paper-alert-they-dont-really-listen-to-people/
Year(s) Of Engagement Activity 2019
 
Description CNBC Arabia TV News commentary on NHS and global Wannacry randsomeware incident 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Following the nationwide disruption of NHS services, and subsequent problems in other countries, due to a 'WannaCry' ransomware attack CNBC Arabia contacted Ansgar Koene to provide 'expert commentary' on the possible origins, impacts and technical safeguards that should be put in place. The interview was broadcast on 12 May 2017 as a segment of the regular news show.
Year(s) Of Engagement Activity 2017
URL https://www.cnbcarabia.com/
 
Description CaSMa blog 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The CaSMa blog disseminates and publicize work and events designed to address the ethical challenges around social media.
To date, the blog has attracted more than 5,000 views and over 400 subscribers. During the last year, the CaSMa blog has posted more than 70 blogs, received many comments from followers (n>39) and several requests for guest posts (e.g. World Wide Web Foundation (webfoundation.org).
The blog is a valuable tool to publicize events, showcase current research projects and inform the public about current topics on social media research as well as provide a platform for the public to engage and communicate with the CaSMa team. For example, after the last public engagement event organised at Nottingham Contemporary, one of the attendees posted:

'Hello, I attended the most recent event. The event was organised masterfully and conducted with the utmost dignity and professionalism yet remained safe, supportive, engaging and thought provoking, it was "real".

At the end of the debate I posed the question, "is the emerging phenomenon of digital interaction a window into Jung's collective unconscious." This could be refined into a debate/exploration of virtual social groups and crowd consensus or dissension on the internet and perhaps their effects on the physical world.

please email me if you for further thoughts.'

Evidence that the CaSMa blog is reaching policy makers is evident from two occasions:
On December 7th 2015, following a blog post regarding the House of Lords inquiry into online platforms we were contacted:
Dear Dr Koene,
I just noticed your short blog on the Lords inquiry into online platforms, particularly commenting on the question relating to algorithms.
Would you care to respond to our inquiry, on that one question? You clearly have a lot of expertise in this area. A page or two would be sufficient and would certainly enrich the debate.
The only issue is that we will stop accepting submissions on the 10th of this month.
Nevertheless, given that you have already more or less written it
Best,
Kilian
Kilian Bourke
Policy Analyst
EU Internal Market Sub-Committee
Committee Office, House of Lords

On January 5th 2016, following blog posts on the draft Investigative Powers Bill we were contacted:
Dear Ansgar,
I work with the team leading on the Investigatory Powers Bill in the Home Office.
I hope you don't mind my approach, but we saw that you'd produced an analysis of the IP Bill on the CaSMa project blog site, and wondered if you might be interested in an informal discussion with officials working on the Bill. This could either be in London, or via phone if that were more convenient.
If you are interested in setting something up please do get in touch.
Best regards
Rebecca

Rebecca Underhill
National Security Directorate
Home Office
Year(s) Of Engagement Activity 2014,2015,2016
URL http://casma.wp.horizon.ac.uk/
 
Description Can we trust what we see online? Futurum online article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Schools
Results and Impact This article was produced by Futurum, a magazine and online platform aimed at inspiring young people to follow a career in the sciences, research and technology. For more information, teaching resources, and course and career guides, see www.futurumcareers.com
Year(s) Of Engagement Activity 2019
URL https://futurumcareers.com/can-we-trust-what-we-see-online
 
Description Children's and Young People's Rights in the Digital Age: An IAMCR 2016 pre-conference 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Third sector organisations
Results and Impact Are children's rights being newly infringed or enhanced in the digital age? Such questions concern researchers, child rights' advocates, and internet governance experts. Yet children's needs and experiences in the digital age are often neglected in high-level debates about global internet provision and governance and children's rights are treated as a minority interest and seen as demanding exceptional treatment from adult society. Further, current debates frequently emphasise the risks children potentially face online and underline their right to protection, but much less attention is gven to children's provision and participation rights online.

The conference will foster an international debate on key issues related to children's and young people's rights in the digital age, including: child and youth participation rights; online opportunities and risks; inequalities and digital exclusion; policy and multi-stakeholder governance; the role of peers and peer culture; participatory research; e-learning, health promotion and creativity in the digital environment; cross-generational dynamics of online engagement.
The conference includes a plenary presentation from Global Kids Online. It will begin at midday on 26 July and close at midday on 27 July to enable delegates to make the journey to Leicester for the opening of IAMCR 2016 (approximately one-hour train journey).
Year(s) Of Engagement Activity 2016
URL http://www.lse.ac.uk/media@lse/events/CYPR-IAMCR-2016.aspx
 
Description Contribution of expertise knowledge to Digital Future Society - Spanish Ministry of Economy/Mobile World Capital 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Response to a request from Digital Future Society (Interprofit) Spain who engage with experts, policymakers, civic organisations, and entrepreneurs to explore, experiment, and exchange knowledge in four key areas: public innovation, digital trust and security, inclusion and citizen empowerment and equitable growth- to contribute towards a programme (which includes a series of interviews), supported by the Spanish Ministry of Economy and Business in collaboration with Mobile Capital Barcelona.

Response content:
Interview questions for Digital Future Society - Spanish Ministry of Economy/Mobile World Capital
By Ansgar Koene, Global Ai Ethics and Regulatory Leader,
https://www.linkedin.com/in/akoene/

Could you give us an overview of your work?
As Global AI Ethics and Regulatory leader at AI my role is to help the AI Lab, and EY as a whole, to develop ethically responsible AI governance methods, produce thought leadership on AI governance and engage with national/regional Ai regulatory developments (e.g. European Commission; OECD AI policy observatory). As part of this work I also contribute to the development of AI related Standards. I chair the IEEE P7003 Standard for Algorithmic Bias Considerations working group, led the Bias Expert Working Group for the IEEE Ethics Certification Program for Autonomous and Intelligent Systems (ECPAIS), and participate in a number of working groups of the ISO/IEC JTC1 SC42 "Artificial Intelligence" sub-committee. I retain a part-time position at the University of Nottingham's Horizon Digital Economy Research Institute where I contribute to the institute's policy impact and support the stakeholder engagement work of the EPSRC funded ReEnTrust research project. As part of the work at the University of Nottingham, I was the lead author on a Science Technology Options Assessment report for the European Parliament on "A governance framework for algorithmic accountability and transparency" that was published in 2019. I am also a trustee for the 5Rights Foundation that

Are algorithms going to take over the labour market, and which areas if so?
Algorithms are having a profound impact on the shape of the labour market by enabling the automation of increasingly complex tasks that follow predicable and repetitive patterns. The current wave of AI systems, which are essentially complex statistical inference engines, has enabled an acceleration of the labour market automation process into domains that require higher levels of pattern recognition (and pattern reproduction) than was previously possible. In the Audit profession for instance Document Intelligence, which combined image processing and Natural Language Processing, is enabling the automation of much of the audit grunt-work of checking that required documents have been filled in properly and transferring information from those documents into databases for further assessment.

Are algorithmic decisions fair?
Fairness is a moral virtue that relates to the justifiability of decisions. Whether or not an algorithmic decision is fair depends on the context of the decision and the ways in which the system has been optimized by its creators. All algorithmic systems are built and programmed by people. What has changed with various forms of Machine Learning is the ways in which this programming is done. Supervised learning systems are programmed by means of the human labelled training data, reinforcement learning systems are programmed by the human defined goal and cost functions associated with the environment model it is provided with, etc.
As the UN Special Rapporteur for Extreme Poverty concluded in his report on the use of algorithmic decision making in social welfare allocation systems, the underlying problem with the use of these systems is that they have been introduced and optimised to implement a cost-cutting agenda at the expense of vulnerable populations. This agenda is a choice made by humans.
Algorithms do not make moral judgements, they execute instructions and optimise towards defined goals. Humans decide how much to prioritize false-positives over false-negatives. If historical data shows a correlation between criminal behaviour and divorced parents, it is humans who must decide if it is fair to penalize someone's chances of being released on parole based on something their parents did and which they most likely had no control over. It is a human societal choice if we want our criminal justice system to judge individuals based on gross statistical patterns across the populations, or if we want a system based on individual cause-and-effect analysis.

How can we guarantee the fairness of algorithms?
In our work on the IEEE Standard for Algorithmic Bias Consideration we phrase the work as intending to minimize unintended, unjustified and unacceptable difference in algorithmic decisions. In order to address these the development and deployment/use of algorithmic systems must:
• Have a sufficient understanding of the context of use, including who will be impacted by the system (does this include groups with different vulnerabilities?). This includes the need to re-assess system bias if there are significant changes to the context it is used in.
• Have a clear and complete understanding of the decisions that are made during the design/deployment and the implications of the criteria that the algorithm is optimizing for. A typical way in which lack of diversity in the development team may lead to bias is by failing to recognize a design choice is linked to cultural customs. Decisions may also end up being made outside of the system design phase during bug fixes, which might lead to a failure to thoroughly explore the implication of those decisions.
• In order to know if a decision or optimisation criterium is fair it is necessary to explore the justification for that decision. Ensure that this choice be justified to the people impacted by the use of the algorithmic system.
• Check if the justifications for the decisions are acceptable in the context where the system is used. Where possible this should include consultation with representatives of the various groups who have been identified as potentially being impacted by the use of the system.

An important element we added in the ECPAIS Bias certification criteria is the need for ongoing system behaviour monitoring and the ability to perform corrective interventions if unjustified bias is observed.

What's the problem with black box algorithms, and how do they affect the work market?
A key problem with black box algorithms, in regards to fairness, is the difficulty in understanding if a decision was justified or if it should be challenged. A recent article by Panoptykon Foundation nicely explained many of the issues regarding black box algorithms, including the reasons why many of these system need not be a black box as they currently are https://medium.com/@szymielewicz/black-boxed-politics-cebc0d5a54ad.
For the work market specifically, black box algorithms pose a problem when they are used in Human Resources contexts such as hiring, performance evaluation and other HR decisions. Similar to the criminal justice example eluded to earlier, many of the current black box algorithms used in these contexts employ machine learning methods that are trained with historical data to detect patterns in resumes or in observed behaviours that, at the population level, are statistically correlated with good employees. With black box algorithms it can be difficult to subsequently investigate if these correlations make causal sense (or are merely spurious correlations), and are justifiable grounds for the decisions (e.g. don't involve discrimination on proxies of protected characteristics such as sex, race, etc.).

Does AI jeopardize the rights of workers?
Referring back to the report of the UN Special Rapporteur on Extreme Poverty, AI jeopardizes the rights of workers if it is used as a tool to implement policy agendas aimed at reducing the rights of workers. This includes the use of black box algorithms as a means for obscuring the rationale behind decisions and frustrating the ability to challenge the justification of those decisions.

How can we protect the fundamental rights of workers while retaining freedom to innovate new algorithmic methods?
A key part to protecting the fundamental rights of workers in the light of algorithmic systems is to acknowledge that the rights of workers pertain to the justification, outcomes and impacts of decisions, irrespective of the tool that is used in the decision-making process.
Innovation of new algorithmic methods can improve the efficiency of processing, e.g. resume pre-screening, but does not change where the accountability for decisions lies, nor what the obligations are regarding the fundamental rights of workers.

Is state intervention needed to guarantee the transparency and accountability of algorithms as well as the liability of developers, or can the industry regulate itself?
The state has an important role in showing how existing rights and regulations remain valid independent of the use of algorithmic systems. In addition to possible general comments for elaborating how existing legislation should be interpreted in an algorithmic context, a key requirement will be upskilling of the regulatory agencies that are tasked with enforcement of existing legislation. The state remains important to guarantee transparency and accountability of decisions, with or without the use of algorithms. If this requirement is maintained, industry will step in to develop standards and best practices for the transparency of algorithms to enable them to be used while remaining compliant with the general requirements on decision making.
In some select domains or applications the use of algorithms may change the allocation of accountability and/or liability between developers and operators of algorithmic systems. For those cases it will be necessary to provide additional regulatory clarity.

Can workers actively protect themselves from algorithm based decisions?
An important step for workers to protect themselves from algorithm based decisions is to demand that their existing rights regarding explanation, justification and challenge of decisions must be respected irrespective of the use of algorithms in the decision process. Accountability lies with the people using the algorithms. Algorithm based decisions are not perfect and deserve to be challenged.
Year(s) Of Engagement Activity 2020
 
Description Contribution to 100+ Brilliant Women Conference chairing youth panel awards, Oxford 16th of Sept 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Undergraduate students
Results and Impact I was delighted to announce the winners from the schools' competition on AI
Year(s) Of Engagement Activity 2019
 
Description Digital Democracy: critical perspectives in the age of big data. 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Blog summarising the European Communications Research and Education Association (ECREA) conference and UnBias participation in November 2017, Stockholm.
Year(s) Of Engagement Activity 2017
URL https://unbias.wp.horizon.ac.uk/2017/11/20/digital-democracy-critical-perspectives-in-the-age-if-big...
 
Description Digital Technology in Practice, NIHR MindTech symposium 2017. London. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact At this symposium Elvira Perez contributed to a round table to discuss if virtual therapist could bridge the current mental health service provision gap
Year(s) Of Engagement Activity 2017
URL http://www.mindtech.org.uk/mindtech-annual-conference.html
 
Description Disinformation and Discernment: New Skills to Protect Old Values. OEB MidSummit, Iceland. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Elvira to add
Year(s) Of Engagement Activity 2017
 
Description Dr Ansgar Koene Podcast interview with PassWOrd 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact https://www.mixcloud.com/FI_PassW0rd/pity-the-poor-children-for-they-know-not-what-you-do/
Year(s) Of Engagement Activity 2019
 
Description Dr Ansgar Koene contribution to Dutch magazine, Emerce 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Dr Ansgar Kone was identified as an expert and approached by (Saçan,Erdinç E) a journalist working for Emerce, a Dutch magazine, to contribute to an article about AI
Year(s) Of Engagement Activity 2021
URL http://www.emerce.nl
 
Description Dr Ansgar Koene interview by CEO of Zupervise.com 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Dr Koene was interviewed for a blog series by the CEO of Zupervise, (AI risk assessment start up) real-time analytics platform, purpose-built for the three lines of defence to analyse, optimise & govern AI Risks in the regulated enterprise:

What are your thoughts about the regulatory policy landscape on AI governance?
I think there's been a recognition that AI is being used in so many use-cases and therefore regulators that are focusing on multiple sectors need to gain some kind of understanding around how AI will potentially have an impact on the way in which current regulations are trying to provide safety & good operating practice in their sector.

At the moment we've got a bit of an exploration that's going on including on the question of whether there should be a new regulator focusing on AI exclusively, or should it be a case of different regulators needing to just be up-skilled and be empowered to deal with AI questions as is relevant to them. In-fact, the discussion in policy circles started from the point of view of AI at a conceptual level with the debates around AI as some form of automation of both decision making & human intellectual activity. We absolutely want to make sure that humans maintain agency, we want to ensure that there is human oversight.

The big focus of 2018-2019 was principles but now, in 2021 we're at the stage of how do you transform these principles into practical rules and regulations to deal with the challenges that arise from AI. It also becomes more necessary to start to think about the fact that, not all AI is the same thing - machine learning is different from other forms of AI, computer vision is different from natural language processing, which is different from recommendation systems.

Maybe we don't want to have a single regulator to cover all of these, maybe we actually do need to be focusing on each sector with tailored regulation. I think that conversation is ongoing, it hasn't really been resolved yet. In the UK, for instance, the data protection authority, the ICO has been tasked to a large extent to try to deal with questions to do with AI primarily around data privacy but they're being pushed to go beyond the remit of personal data so it's an ongoing discussion.

What are the biggest challenges in regulating AI?
I think one of the biggest challenges that is becoming visible is the question of how do we regulate the addition of an AI component to an existing process that needs governance. There are few exceptions like autonomous vehicles which may be conceived to be adding something completely new to the transportation mix. For example, with AI-enabled recruitment, we're simply automating part of the hiring process by pre-filtering with AI.

Primarily, in such a case of adding AI to an existing process, it's not really clear how to identify if this AI actually introduces new risks that need to be regulated differently. Let's take for instance the example of AI in hiring. At the core you're not allowed to discriminate based on race or gender or other non relevant factors when it comes to the hiring process and really whether or not you're doing this discrimination through an AI or you're doing it through human decision making is pretty much beside the point. What you are regulating is that there shouldn't be any discrimination and so that raises the question on whether we need to change anything in the regulation there or is it just a case of maybe we need a new process for providing the evidence that you are complying with the existing regulation, by applying the appropriate risk assessments.

Within the enterprise, who cares the most about such AI risks?
I guess it's currently still largely approached from the traditional compliance led approach - the development teams are primarily focusing on achieving the functional requirements of the system and then we have compliance teams that are assessing whether you are compliant with various regulatory issues. There isn't really a sort of structural introduction yet, I think, of assessing ethical risks or the societal impact kind of risks . As part of the larger discussion around business ethics and ESG (environmental social governance) these kind of questions need to be answered.

How does one get started with governing AI?
I think you start with a clear AI strategy. When you're creating your requirements set, do you have clear justifications for why you're making certain choices on the selection of the data sets that you're going to use. Further still, have you documented your process for how you've collected the data and how you've chosen which data set to use. To a large extent, this comes down to the ability to document what you've done and to provide justifications for these. I'm focusing on the documentation aspect here because this is the kind of challenge that we see now with the attempts at AI auditing, but then there is insufficient documentary evidence to certify anything.

What is your opinion on AI regulation proposed by academia & it's applicability to enterprise AI use cases?
It's a very broad question as there are quite a number of approaches that are being taken in academia. There are some that are trying to get into technical methods within the computer science community. We've had various academics who have been trying to operationalise a definition of AI fairness.

On the one hand, this is something that you can build into a toolkit to introduce that into development cycle. On the other hand, it is also being criticised within academia as an insufficient understanding of the bigger picture on the need to provide transparency on the decision making process. We have a part of the academic literature that is focusing on ethical concerns to identify application areas where AI is is not the appropriate solution approach. These tend to be something that address government and policymakers.

For instance, the conversation around when is probabilistic machine-led decision making fundamentally not appropriate to a certain type of domain. In society, we can think of the criminal justice system where I would say, people should be judged based on what they did, not based on whether they seem to fall in a population groups that statistically has been shown to end up in prison more often. These kinds of discussions are not really something that addresses enterprise so much as they are things that address government regulation.

Where is the AI regulation discourse leading to?
I think we are we are heading into a direction where these technologies are having a significant impact on people and on society.

Therefore, they need to be regulated in a similar way to other kinds of domains like vaccines in healthcare or the safety measures that we need with self-driving cars in transportation. As a result, it is going to transform this space into something that will be more strongly regulated with certification regimes. It may still take a little bit of time if we consider the countries that are leading on the the process of developing AI regulation - Singapore, Europe to a large extent also the UAE, which is exploring in this space - they are currently performing regulatory gap analysis and drafting new regulation proposals.

This is still going to take probably at least a year to crystallise and then another year or so to formulate into clear regulations. Within that time period we will be seeing the publication of more technical standards around the space. The ISO/IEC's joint technical committee on AI as well as the AI work of the IEEE-SA has really picked up steam and is likely to start publishing standards in the run up to this year. We will soon be getting a bigger body of guidance of what this best practice looks like. Also, in the US, we are seeing greater attention to the question about how an assessment of these systems should work and what role benchmarks can play.

I expect that in about 3 years time we will probably be looking at a completely different landscape regarding any regulation on AI.
Year(s) Of Engagement Activity 2021
URL https://zupervise.com/the-future-of-ai-regulation
 
Description Dr Ansgar Koene interview with Digital Future Society blog psot 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Dr Koene was invited for an interview on the risks and use of black box algorithms in HR by Digital Future Society in Feb 2020. The bog post became live in Aug 2020 and is available to interested parties.
Year(s) Of Engagement Activity 2020
URL https://digitalfuturesociety.com/qanda/ansgar-koene-and-the-risks-of-the-use-of-black-box-algorithms...
 
Description Dr Ansgar Koene interview with Digital Future Society blog psot 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Dr Koene was invited for an interview on the risks and use of black box algorithms in HR by Digital Future Society in Feb 2020. The bog post became live in Aug 2020 and is available to interested parties.
Year(s) Of Engagement Activity 2020
URL https://digitalfuturesociety.com/qanda/ansgar-koene-and-the-risks-of-the-use-of-black-box-algorithms...
 
Description Dr Ansgar Koene invited as a panellist in the European of Platform of regulatory authorities EPRS Plenary 2 Podcast 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Dr Ansgar Koene was invited to participate as a panellist in the EPRA Plenary 2 Podcast - Media Plurality in the age of algorithms: Transparency and Trust - the user's perspective in online content navigation. The podcast will be available on the EPRA website from Thursday 12th November.

During the Podcast, EPRA Vice Chairperson Mari Velsand invited panellists to discuss:

the concepts of transparency, trust and critical engagement in the context of journalistic news content increasingly curated and delivered to audience by means of algorithms and
the role of regulators, journalists and the tech industry in ensuring that online news consumption better supports pluralistic and engaged democratic discourse.
Year(s) Of Engagement Activity 2020
URL https://shows.acast.com/epra/episodes/transparency-and-trust-the-users-perspective-in-online-conte
 
Description Dr Ansgar Koene invited as a webinar Panel Member by Logically (a fact checking company) 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Dr Ansgar Koene invited as a panel member by 'The Content Leads' Logically to participate in a webinar broadcast on YouTube:

Fake news has entered the global lexicon in the last four years. Online platforms unite communities from across continents; however, greater interconnectivity has also broadened the scope of mis/disinformation. Following sessions on misinformation in India and the US, this session by Logically looks at global trends and seek to identify how the confluence of misinformation, journalism and social media may converge or diverge in the coming years. It will seek to cover a broad range of issues, including:

How does fake news influence international relations between countries?

Does fake news predominantly come from a small group of malicious global actors? To what extent is the public responsible for innocently disseminating and pollinating such information?

In using AI to tackle fake news, how problematic are the challenges of inscrutability (the models defy human understanding) and non-intuitiveness (why do the statistical relationships exist as they do?) in applying transparency effectively?

Are certain countries more/less immune to fake news? Will the Finnish model of teaching school pupils to spot slippery information the way to go?
Or is the recent Brazilian Bill passed by the Senate the way to go? What trade-offs does it present for privacy and freedom of expression?

How might fake news evolve and evade identification in the coming years? What is being done to combat this?

Speakers:

- Damian Collins MP, Chair, DCMS Subcommittee on Disinformation
- Marianna Spring, Specialist Disinformation and Social Media Reporter, BBC News
- Viji Alles, Presenter, BBC Radio 4 (Moderator)
- Lyric Jain, Founder and CEO, Logically
- Ansgar Koene, Global AI Ethics and Regulatory Leader, EY
- Hazel Baker, Global Head of UGC Newsgathering, Reuters
Year(s) Of Engagement Activity 2020
URL https://youtu.be/woD2ruvbsPo
 
Description Dr Ansgar Koene invited as a webinar Panel Member by Logically (a fact checking company) 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Dr Ansgar Koene invited as a panel member by 'The Content Leads' Logically to participate in a webinar broadcast on YouTube: Fake news has entered the global lexicon in the last four years. Online platforms unite communities from across continents; however, greater interconnectivity has also broadened the scope of mis/disinformation. Following sessions on misinformation in India and the US, this session by Logically looks at global trends and seek to identify how the confluence of misinformation, journalism and social media may converge or diverge in the coming years. It will seek to cover a broad range of issues, including: How does fake news influence international relations between countries? Does fake news predominantly come from a small group of malicious global actors? To what extent is the public responsible for innocently disseminating and pollinating such information? In using AI to tackle fake news, how problematic are the challenges of inscrutability (the models defy human understanding) and non-intuitiveness (why do the statistical relationships exist as they do?) in applying transparency effectively? Are certain countries more/less immune to fake news? Will the Finnish model of teaching school pupils to spot slippery information the way to go? Or is the recent Brazilian Bill passed by the Senate the way to go? What trade-offs does it present for privacy and freedom of expression? How might fake news evolve and evade identification in the coming years? What is being done to combat this? Speakers: - Damian Collins MP, Chair, DCMS Subcommittee on Disinformation - Marianna Spring, Specialist Disinformation and Social Media Reporter, BBC News - Viji Alles, Presenter, BBC Radio 4 (Moderator) - Lyric Jain, Founder and CEO, Logically - Ansgar Koene, Global AI Ethics and Regulatory Leader, EY - Hazel Baker, Global Head of UGC Newsgathering, Reuters
Year(s) Of Engagement Activity 2020
URL https://youtu.be/woD2ruvbsPo
 
Description Dr Ansgar Koene invited as an expert panel member by Capgemini ELITE management team on a webinar entitled "Ethics in New Normal" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dr Koene participated in an online webinar organised by Capgemini (India) - a global leader in consulting, digital transformation, technology and engineering services: https://www.capgemini.com/ as an expert panel member on a webinar on Ethics in New Normal. Email feedback from Capgemini:

"I am sending this mail on behalf of the ELITE Engagement team (Hemanth, Shivani and myself) that conducted the 2nd edition of the Management Symposium "La Table Ronde". We are delighted to inform you that this webinar on Ethics in New Normal was a resounding success. We had an audience of more than 3.2k, which is the highest recorded for La Table Ronde yet.

The team would like to thank you for your contribution in the insightful discussion. Your inputs around ethics on the technology front and sentiments of the normal population and organizations in understanding/assessing the ethical scope of AI, were a great value addition and very much liked by the audience.

In these times, when the ecosystem is changing and all the contributors are becoming interconnected, we believe it is crucial to learn from other's experiences. Hence, we started this initiative and wish to expand our reach even further. It will only be possible with the support of generous industry veterans like yourself who are willing to contribute their precious time.

A big thank you. We strongly believe that everyone who attended the event had some valuable insights that they could takeaway with them and ponder about. We hope to have many more events like this on many more contemporary and important topics and that our association continues in making such events happen."
Year(s) Of Engagement Activity 2020
 
Description Dr Ansgar Koene invited by AI for Business 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Dr Ansgar Koene was invited by ~AI for Business to do a long-form interview for their website: https://www.aiforbusiness.net/ai-interviews
Year(s) Of Engagement Activity 2020
URL https://youtu.be/b28Vn-QskHc
 
Description Dr Ansgar Koene invited by AI for Business 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Dr Ansgar Koene was invited by ~AI for Business to do a long-form interview for their website: https://www.aiforbusiness.net/ai-interviews
Year(s) Of Engagement Activity 2020
URL https://youtu.be/b28Vn-QskHc
 
Description Dr Ansgar Koene invited by AIBE Speaker series as a panel member on webinar entitled AI Ethics Beyond Theory and into practical application 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Artificial Intelligence in Business and Ethics (AIBE) is the world´s largest non-profit AI summit. AIBE is the annual rendezvous of AI experts, students and professionals. Our mission is to enhance public understanding of artificial intelligence and data-driven technologies.

Dr Ansgar Koene was invited as an expert AI ethics panel member to explore the following topic:

Building trust in AI technologies by taking an ethical approach to AI adoption is a key enabler for businesses to succeed. Scaling the use of AI, however, has led to diverse ethical challenges. For example, autonomous vehicle incident liability, biases in automated recruitment software and substandard AI-based grading algorithms in education.

AI makes existing ethical challenges more urgent and creates new challenges. This panel will discuss how we move beyond the discussions, to begin applying and scaling ethics in AI projects across business, government, society and education.

Discussions will identify the current gap between theory and practice of governance in AI and draw on this panel's experiences of building and implementing ethical & responsible AI approach into teams, governance, and processes.
Year(s) Of Engagement Activity 2020
URL https://www.eventbrite.co.uk/e/ai-ethics-beyond-theory-and-into-practical-application-tickets-122814...
 
Description Dr Ansgar Koene invited by Humans of AI YouTube channel for contribution 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Dr Koene contributed towards the Humans of AI series of interesting talks about AI technology with people innovating in that area: Ansgars presentation - @IEEE @EY - Harnessing AI as a tool, just like we did with electricity
Year(s) Of Engagement Activity 2020
URL https://www.youtube.com/watch?v=t5Y2dwsODUI
 
Description Dr Ansgar Koene invited by Humans of AI YouTube channel for contribution 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Dr Koene contributed towards the Humans of AI series of interesting talks about AI technology with people innovating in that area: Ansgars presentation - @IEEE @EY - Harnessing AI as a tool, just like we did with electricity
Year(s) Of Engagement Activity 2020
URL https://www.youtube.com/watch?v=t5Y2dwsODUI
 
Description Dr Ansgar Koene invited by UNESCO to participate in the Virtual Regional Consultation on the first draft of the Recommendation on the Ethics of Artificial Intelligence (AI) on 27 - 28 July 2020 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Dr Ansgar Koene was invited as below: On behalf of UNESCO, the Kingdom of the Netherlands and the Rathenau Instituut we cordially invite you to participate in the Virtual Regional Consultation on the first draft of the Recommendation on the Ethics of Artificial Intelligence (AI) on 27 - 28 July 2020. UNESCO has embarked on a two-year process to elaborate the first global standard-setting instrument on ethics of AI. The first draft of the Recommendation has been produced by an Ad hoc Expert Group (AHEG) appointed by the Director-General of UNESCO. The Virtual Regional Consultation for the Europe Region (group 1) is part of a series of multi-stakeholder consultations worldwide between 7 July and 8 August 2020. Based on the feedback received during multi-stakeholder consultation process, the Ad hoc Expert Group will revise the Recommendation until September 2020. The main objective of the consultations is to discuss the first draft text of the Recommendation on the Ethics of AI and to include different cultural values and address various regional concerns to develop an inclusive and pluralistic global instrument. In this regional consultation, we highly welcome representatives from governments, academia, the private sector, civil society and youth in Western European and North American Member States to participate in this Virtual Regional Consultation. Your participation will help to enrich the draft Recommendation with your views on how to ensure an ethical use of AI around the world. We would like to ask you to carefully read the guidelines for participants attached to find more information about how to prepare for the consultation as well as guiding principles for our discussions.
Year(s) Of Engagement Activity 2020
URL https://en.unesco.org/artificial-intelligence/ethics
 
Description Dr Ansgar Koene invited by UNESCO to participate in the Virtual Regional Consultation on the first draft of the Recommendation on the Ethics of Artificial Intelligence (AI) on 27 - 28 July 2020 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Dr Ansgar Koene was invited as below:

On behalf of UNESCO, the Kingdom of the Netherlands and the Rathenau Instituut we cordially invite you to participate in the Virtual Regional Consultation on the first draft of the Recommendation on the Ethics of Artificial Intelligence (AI) on 27 - 28 July 2020.

UNESCO has embarked on a two-year process to elaborate the first global standard-setting instrument on ethics of AI. The first draft of the Recommendation has been produced by an Ad hoc Expert Group (AHEG) appointed by the Director-General of UNESCO. The Virtual Regional Consultation for the Europe Region (group 1) is part of a series of multi-stakeholder consultations worldwide between 7 July and 8 August 2020. Based on the feedback received during multi-stakeholder consultation process, the Ad hoc Expert Group will revise the Recommendation until September 2020.

The main objective of the consultations is to discuss the first draft text of the Recommendation on the Ethics of AI and to include different cultural values and address various regional concerns to develop an inclusive and pluralistic global instrument. In this regional consultation, we highly welcome representatives from governments, academia, the private sector, civil society and youth in Western European and North American Member States to participate in this Virtual Regional Consultation.
Your participation will help to enrich the draft Recommendation with your views on how to ensure an ethical use of AI around the world. We would like to ask you to carefully read the guidelines for participants attached to find more information about how to prepare for the consultation as well as guiding principles for our discussions.

When: 27 - 28 July 2020
Time: 9.30 AM - 16.30 PM
Platform: Interprefy (invitation for participants, invitation for observers)
Language: English and French
Year(s) Of Engagement Activity 2020
URL https://en.unesco.org/artificial-intelligence/ethics
 
Description Dr Ansgar Koene invited by the WEF Global Shapers Brussels Hub to participate in an online event AI Ethics and the future of work 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Professor Ansgar participated in an online webinar organised by the Global Sharpers Brussels Hub entitled AI Ethics and the Future of Work (http://globalshapersbrussels.com/) - https://fb.watch/29LNWSlSqk/
Year(s) Of Engagement Activity 2020
 
Description Dr Ansgar Koene invited to participate in Applied AI - Women@EIT (European Insitute of Innovation and Technology: Women at EIT: https://women.eitalumni.eu/ ) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Women at EIT: https://women.eitalumni.eu/ ) are hosting a workshop at the EIT Alumni Connect event (https://eit.europa.eu/our-communities/eit-alumni/eit-alumni-connect-2020 ) on the 28th of November at 4.30-5.30 pm CET. The workshop main aim is to raise awareness of data bias regarding gender equality in AI with relation to responsible production and consumption. The focus is on how we can use the opportunity of AI and data to become more gender equal (implementing of the SDG 5) and at the same time reach our sustainability goals. Discussions about what companies can do and think about and what the consequences are if we don't take this into consideration entering into a new area of AI.

Dr Ansgar Koene participated as a panel member - the panel provided different perspectives to discuss the topic during one hour.

Link to Women@EIT: https://women.eitalumni.eu/
Year(s) Of Engagement Activity 2020
URL https://eit.europa.eu/our-communities/eit-alumni/eit-alumni-connect-2020
 
Description Dr Ansgar Koene invited to participate in Dialogues with Asian Pathfinders 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dr Ansgar Koene was invited by Asian Pathfinders: LinkedIn: https://www.linkedin.com/company/53193591 as an expert in Ethics and AI to participate in a webinar on 26 Nov 2020 entitled Adaptability of AI - security and challenges:

1) Ethics in AI - Dr. Ansgar Koene
Global AI Ethics & Regulatory Leader at EY, London

2) Military Dimensions of AI - Husanjyot Chahal
Research Analyst at Georgetown's Center for Security and Emerging Technology (CSET), Washington DC

3) Governance & AI - Walter M. Pasquarelli
Consultant, Oxford Insights, London

Charumati Haran, Student of Master of Public Policy at Hertie School in Berlin, Germany, will be moderating the session.

Date and Time:
Thursday, November 26, 2020
05:30PM - 06:45PM(IST) / 12:00PM - 01:15PM(GMT)
Year(s) Of Engagement Activity 2020
URL https://us02web.zoom.us/meeting/register/tZAkcOmuqToiEtLyufetciLDk-t1lLePGKuv
 
Description Dr Ansgar Koene lecture at University of Leubeck 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Dr Koene was invited by the University of Luebeck to present an online 'Virtual Ethical Innovation' lecture on "Principles, Standards and Regulation for Trustworthy AI"
Year(s) Of Engagement Activity 2021
URL https://www.eih.uni-luebeck.de/veil/recordings-and-highlights/2021-01-19-ansgar-koene-principles-sta...
 
Description Dr Ansgar Koene participation in IEEE webinar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dr Ansgar Koene was invited to participate in an On-Demand Webinar entitled: how to design a digital world where children can thrive:

"Our expert panel discusses how the new IEEE Standard for an Age-Appropriate Digital Services Framework Based on the 5Rights Principles for Children (IEEE 2089™-2021) will equip organizations with the tools to put young people's best interests at the heart of the design of digital products and services. By following the practical steps outlined in this new framework, designers can ensure that children are catered for in the digital world"
Year(s) Of Engagement Activity 2022
URL https://engagestandards.ieee.org/Childrens-2089-webinar.html?utm_source=email&utm_medium=eblast&utm_...
 
Description Dr Ansgar invite to moderate a breakout session at the UNICEF Global Forum on AI for Children 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Dr Ansgar Koene participated as a panellist in a breakout session entitled "Ensure inclusion of and for children" during the UNICEF Global Forum on AI for Children on the 1st December
Year(s) Of Engagement Activity 2021
URL https://www.unicef.org/globalinsight/featured-projects/ai-children
 
Description Dr Koene approached by RAND Europe, a not-for-profit research organisation that helps to improve policy and decision making through research and analysis 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Third sector organisations
Results and Impact RAND Europe has been commissioned by Microsoft Belgium to review the landscape, and bring together evidence, on the use of labelling initiatives and codes of conduct for AI applications. The work will focus on identifying relevant examples of AI labelling schemes and initiatives, and codes of conduct, for the ethical and safe development of AI; and carry out a comparative analysis of the different examples to articulate common themes as well as notable divergences.

As part of this work, we are conducting scoping consultations with experts involved in or with knowledge of developments within the wider AI accountability (including labelling, codes of conduct) ecosystem. Given your experience in the field, RAND would greatly appreciate the opportunity to conduct a brief interview with you on this subject. The specific aim of the consultation would be to:

• Gather your general views on the use of AI labelling schemes and initiatives, and codes of conduct, for the ethical and safe development of AI
• Identify potentially relevant literature on this topic that we should consider (this can include academic literature or relevant reports from organisations)
• Identify organisations and individuals who might be able to provide us with more information about AI labelling initiatives and codes of conduct
Year(s) Of Engagement Activity 2021
 
Description Dr Koene chair - panel session: IEEE SCCI 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Dr Koene was invited to participate in the IIEEE SSCI 2021 symposium - an event to : To provide academia and industry with an overview of the current ethical AI landscape to discuss and debate how to establish sustainably trustworthy and responsible research and innovation.

Discussion themes include (but are not limited to):

The impact of emerging global legislation on data and AI ethics on conducting AI research and innovation.
Responsibility and Accountability in the AI decision making
Algorithm bias in machine learning
Can Industry Self-regulation Deliver 'Ethical AI'?
Sustainable and Responsible AI
Citizen involvement in conceptualisation of AI products and services to build trust.
Year(s) Of Engagement Activity 2021
URL https://attend.ieee.org/ssci-2021/panel-session-using-ai-to-establish-sustainably-trustworthy-and-re...
 
Description Dr Koene invited to speak at Data Governance Day hosted by Ernst Young 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dr Ansgar Koene presented at a moderated session on Online safety and it's impact on Children hosted by Ernst Young (late January). The panel consisted of a representation from Childnet International who used the occasion to advertise Safer Internet Day to the DPO's and compliance officers from various industry attendees
Year(s) Of Engagement Activity 2021
 
Description Dr Koene presented at a Facial Recognition Technology: Challenges for International Collaboration & Governance' workshop run by the Utrecht University 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Dr Ansgar Koene presented 'Developments for AI Governance through Regulation and Standards: Globally Coordinated Deliberative Approaches vs. Reactive Policy Making in the session Global policies on FR technology and related international collaboration session on 17th November. The event was online live stream: https://livestream.acsaudiovisual.com/uu20211117
Year(s) Of Engagement Activity 2021
URL https://livestream.acsaudiovisual.com/uu20211117
 
Description Dr Koene provided comments to support an article in Protocol.com 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Dr Ansgar Koene was interviewed by Protocol - a media company- for a news article related to Google's firing of leading AI ethics researcher Timnit Gebru.
https://www-protocol-com.cdn.ampproject.org/c/s/www.protocol.com/amp/timnit-gebru-fired-ethics-google-2649129371
Year(s) Of Engagement Activity 2020
URL https://www-protocol-com.cdn.ampproject.org/c/s/www.protocol.com/amp/timnit-gebru-fired-ethics-googl...
 
Description EGA School Fairness Tool Kit Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact This workshop was organised in order to allow young people who had co-created the unfairness toolkit- to have a chance to interact with the awareness cards (one of the three key components of the tool kit). We initially had a case study discussion to allow the young people to re-engage with the key issues and concerns regarding algorithmic fairness. We then conducted an interactive session where they undertook group work using the cards.
Year(s) Of Engagement Activity 2018
 
Description ESRC Annual Festival of Social Sciences, Nottingham 2018 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact "Do you trust internet platforms?" Participatory and hands-on activity to help internet users to explore their online experiences when interacting with algorithm driven platforms. We run two workshops for people aged 16-25 and 65 years old and over, where a total of 17 participants took part.
This activity constituted a pilot study, part of an interdisciplinary research project between the Universities of Nottingham, Oxford and Edinburgh called ReEnTrust, which explores new technological opportunities to enhance/re-build user's trust, and identify how this may be achieved in ways that are user-driven and responsible. Preliminary data collected from this study has been extremely valuable and has contributed to refine particular research topics within our current project (ReEnTrust). Also, participants from this activity expressed their will to contribute to follow-up interviews and also to be part of the ReEnTrust advisory group.
Year(s) Of Engagement Activity 2018
URL https://www.horizon.ac.uk/reentrust-call-for-participants/
 
Description Editorial Responsibility arising from personalisation algorithms. ETHICOMP conference, Turin. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This talk was part of the Ethicomp conference in Turin, Italy and was attended by an audience consisting of Ethicomp participants. Ethicomp is the main conference of the RRI in ICT community, with publications in the ORBIT Journal, launched by the Orbit project (https://www.orbit-rri.org). Following the presentation a number of audience members requested further information and some of them joined the UnBias stakeholders group.
Year(s) Of Engagement Activity 2017
URL https://unbias.wp.horizon.ac.uk/dissemination/presentations/
 
Description Editorial responsibility arising from personalization algorithms. The ethics of machine learning in professional practice. Cambridge, 28th Nov. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact During this workshop I presented the UnBias project which sparked questions and interests among an heterogeneous audience. It was and important engagement event afterwards two of the attendees become stakeholders and supporter of the UnBias project.
Year(s) Of Engagement Activity 2016
 
Description Ethical Hackathon 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact An 'ethical hackathon' is a novel event developed by the Human Centred Computing group at the University of Oxford. It takes a twist on the traditional hackathon format by challenging groups of participants to identify ways to embed ethical considerations into the processes of technical design and development. The event in October 2019 involved postgraduate students at Horizon CDT University of Nottingham, in a day long session. At the start of the session leaders Liz Dowthwaite and Menisha Patel introduced key project themes and invited students to discuss issues connected to algorithmic controversies and governance. After initial themes had been introduced and activities completed, students were put into groups and set a design challenge to work on. At the end of the session the teams presented their designs in front a panel of judges. The judges gave feedback on the presentations and prizes were awarded for the best designs.
Year(s) Of Engagement Activity 2019
 
Description Ethical Hackathon (two day- long events- Oxford and Nottingham) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact An 'ethical hackathon' is a novel event developed by the Human Centred Computing group at the University of Oxford. It takes a twist on the traditional hackathon format by challenging groups of participants to identify ways to embed ethical considerations into the processes of technical design and development. The UnBias team ethical hackathons in November 2018 and January 2019 involving postgraduate students at Horizon CDT University of Nottingham & the Cyber Security CDT University of Oxford. These took the form of a day long session. At the start of the session leaders Helena Webb and Menisha Patel introduced key project themes and invited students to discuss issues connected to algorithmic controversies and governance. During the Nottingham Ethical Hackathon discussion was also facilitated by Liz Dowthwaite, After initial themes had been introduced and activities completed, students were put into groups and set a design challenge to work on. At the end of the session the teams presented their designs in front a panel of judges from the UnBias team. The judges gave feedback on the presentations and prizes were awarded for the best designs.
Year(s) Of Engagement Activity 2018,2019
 
Description Ethicon at University of Nottingham 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact An 'ethicon' is a novel event developed by the Human Centred Computing group at the University of Oxford. It takes a twist on the traditional hackathon format by challenging groups of participants to identify ways to embed ethical considerations into the processes of technical design and development. The UnBias team ran an ethicon in January/February 2018 involving postgraduate students at Horizon CDT University of Nottingham. This took the form of a two part session. In the first, session leaders Helena Webb and Menisha Patel introduced key project themes and invited students to discuss issues connected to algorithmic controversies and governance. Discussion was also facilitated by Liz Dowthwaite and Helen Creswick, At the end of the session students were put into groups and set a design challenge to work on. A week later - after being given further time in the module sessions to work on the challenge - the second session took place. The teams presented their designs in front a panel of judges from the UnBias team. The judges gave feedback on the presentations and prizes were awarded for the best designs.
Year(s) Of Engagement Activity 2018
 
Description Ethics assessment of research and innovation. SATORI. London 24th Nov. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This is a working group organized with European funding. The connections that emerged from that meeting are extremely relevant specially after Brexit resolution because it allows us to keep connected with our European colleagues working on data ethics.
Year(s) Of Engagement Activity 2016
URL http://trilateralresearch.com/satori-mutual-learning-workshop-ethics-assessment-research-innovation/
 
Description European Researchers' Night: Curiosity Carnival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Members of the Digital Wildfire project teamed up the EPSRC project UnBias to run an engagement stall at European Researchers' Night 2017 in Oxford. The Curiosity Carnival formed part of European Researchers' Night, celebrated in cities across Europe. Oxford ran a city wide programme of activities across its universities, libraries, gardens and woods to give members of the public a chance to find out about real research projects and meet the people who conduct them. The combined team set up information displays about the two projects and ran an activity that invited visitors to learn about common scenarios in which safety and fairness on the Internet are undermined and then vote to give their responses to them. Team members then engaged visitors in discussion about our research work and findings. Hundreds of participants - mostly families with children but also students - attended the event and took part in our engagement activities over the course of the evening.
Year(s) Of Engagement Activity 2017
URL https://unbias.wp.horizon.ac.uk/2017/10/02/unbias-takes-part-in-european-researchers-night/
 
Description Evaluation of the participation model 'Sharing the Future' : research project within Primary Parliaments, Nottingham City Council. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact We conducted a feasibility study designed to evaluate the efficacy of 'Sharing the Future' (StF), a new model for citizen participation routinely applied at Primary Parliament, an educational activity offered to primary schools in Nottingham City. Our study showed that the implementation of the model StF boosts children's confidence on their relation with other children and young people in regards to: 1) sharing their views and life experiences, 2) decision-making, as well as making decisions on their own and 3) being listened and respected by their peers. This feasibility study provided the elements and conditions required to conduct a larger future pilot study. From 232 children invited to participate in this study, a total of 102 pupils (aged 9-11) from 19 institutions took part. Data from this work lead to the preparation of a research manuscript to be soon submitted to an international citizen engagement peer-reviewed journal.
Year(s) Of Engagement Activity 2017
 
Description Experiential workshop at the National Trust building The Workhouse in Southwell as part of the book launch 'Voices from the Workhouse'. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Supporters
Results and Impact This invitation came from the National Trust and it is linked to the research project 'What is home?'. The aim of this experiential workshop is to reflect on the notions of home, the history of the The Workhouse and how to improve the visiting experience of future visitors. The workshop was also aimed to engage with an audience that rarely engages in embodied practices such as yoga or flamenco during a weekend retreat. The workshop was a success and many participants reported being interested in contributing or participating in future related events. This activity also links with previous work done by Horizon in this building aiming to improve the visitors experience by introducing interactive elements.
Year(s) Of Engagement Activity 2017
URL http://www.nationaltrust.org.uk/theworkhouse
 
Description Experiential workshop by the National Trust at The Workhouse 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Experiential workshop by the National Trust at The Workhouse to launch the book 'Voices from the Workhouse'. Southwell, 18th March, 2017. I have been invited again to provide a similar experiential workshop
Year(s) Of Engagement Activity 2017
 
Description Facilitator Booklet development workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact The workshop was run with people aged 13-17 years, as part of an EPSRC Impact Exploration Grant that was awarded to the UnBias team in Nottingham. The workshops enabled the team to run sessions using the cards and to receive feedback on them towards development of the Facilitator Toolkit. Each group was also asked to co-design games that could be used to help people to interact with the Awareness Cards and to learn about issues of online fairness
Year(s) Of Engagement Activity 2019
URL https://unbias.wp.horizon.ac.uk/2019/07/21/unbias-awareness-cards-facilitator-booklet/
 
Description Fairness toolkit sessions at schools and youth groups 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact The UnBias project involves the preparation of a 'fairness toolkit' that will help stakeholders of different kinds to understand controversies associated with online algorithms and reflect on the steps they could take to promote fairness online. To assist with the development of the toolkit we ran a series of workshops with young people to seek their experiences and views on these issues. These sessions were led by project collaborator Giles Lane with the assistance of Helena Webb, Menisha Patel and Liz Dowthwaite. They visited two schools and one youth group, running two 2-hour sessions at each location. Secondary school aged young people were invited to take part in a series of fun activities and debates.
Year(s) Of Engagement Activity 2017
 
Description Festival of Ideas at the Computational Foundry, University of Swansea 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Participation in Festival of Ideas at the Computational Foundry, University of Swansea. Open to the public, academia, and industry, the involved taking part in discussions, and displaying the work of the Horizon Digital Economy Research in terms of the use of ideation cards.
Year(s) Of Engagement Activity 2019
 
Description Festival of Science and Curiosity (FOSAC), Nottingham City Library 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Drop-in sessions to introduce people to the UnBias Awareness Cards and to discuss issues of online fairness. This activity was part of a wider outreach event, and run as part of the Impact Exploration Grant Award. We engaged with approximately 50 people, mostly families. Three decks of cards were requested from people working who worked in the Education Sector.
Year(s) Of Engagement Activity 2019
URL https://www.horizon.ac.uk/13229-2/
 
Description First wave of UnBias Youth Juries 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact 144 student from secondary schools and/or 6th form colleges attended the workshops. 14 sessions were run at those institutions or at the UoN.
The UnBias Youth Juries are youth-lead focus groups with different scenarios to prompt discussions about some particular aspects on how the internet works (with focus on algorithm fairness when interacting with automated systems).
Sessions were very participative. Debate and audience change of opinion during the session was highly noticed.
Students and accompanying school teachers showed interest in taking part in related research activities.
Year(s) Of Engagement Activity 2017
 
Description Flamenco-Yoga as creative practice for mutual recovery. Experiential workshop presented at the 5th International Health Humanities Conference, Seville, 15-17 September [page 76] 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The workshop was run twice during the conference. No formal feedback was collected after or during the workshop but the first author captured anecdotal feedback and received numerous invitations to deliver the workshop again in different settings. To date, the first author has run the workshop several times among children and young people and is planing a systematic evaluation to understand the effects that creative practices as mutual recovery can have among s clinical and non-clinical populations.
Year(s) Of Engagement Activity 2016
URL http://www.ihhcsevilla2016.es/
 
Description Futuremakers podcast 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Helena Webb was part of a podacst discussing the topic 'Are all algorithms biased?' The podcast was created and broadcast by the University of Oxford in their Futuremakers series on AI.
Year(s) Of Engagement Activity 2018
URL https://podcasts.ox.ac.uk/are-all-algorithms-biased
 
Description Gem FM radio interview 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact I was interviewed by Gem FM about the work I did bringing yoga into children homes
Year(s) Of Engagement Activity 2016
 
Description Hello from Mozfest! 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Blog sumarising UnBias team engagement activities at Mozfest Festival October 2017, London.
Year(s) Of Engagement Activity 2017
URL https://unbias.wp.horizon.ac.uk/2017/10/30/hello-from-mozfest/
 
Description How do you take care of the internet? Mozfest festival, London 2017. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Drop-in session with interactive activities that sparked debate and reflection on issues such as: privacy, security and how to take care on the internet. Activities including scenarios related to fake news, hate speech, search results and personalised content were displayed and participants were encouraged to express their opinions by voting best possible outcome for each scenario. About 80 people from a diverse audience (from professional practitioners to the general public) engaged in this session. Relevant discussions and views provided our research team of new insights on people's online experiences and allowed to establish new contacts, for ongoing and future collaborations.
Year(s) Of Engagement Activity 2017
URL https://unbias.wp.horizon.ac.uk/2017/10/30/hello-from-mozfest/
 
Description Human Agency on Algorithmic Systems, AoIR conference, Estonia. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This was a presentation at the annual Association of Internet Researchers conference. The session was attended by about 60 audience consisting of other conference attendees, mostly from academia but also including some NGOs and industry interested in understanding the latest academic work on the impact of internet on society. The talk covered results from the UnBias Youth Juries, and stakeholder engagement workshops as well as some information regarding the IEEE P7003 Standard for Algorithmic Bias Considerations working group. At the end of the talk there was a good discussion and some of the participants approached to ask about joining the P7003 working group.
Year(s) Of Engagement Activity 2017
URL https://www.slideshare.net/AnsgarKoene/human-agency-on-algorithmic-systems
 
Description IEEE-SA webinar: The Human Standard - Why Ethical Considerations Should Drive Technological Design 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Online webinar organized by the IEEE Global Initiative for Ethics of Autonomous and Intelligent Systems with contributions from the chairs of the IEEE P70xx ethics related standards, including the P7003 Standard for Algorithmic Bias Considerations that is chaired by Ansgar Koene. The webinar was attended by IEEE members globally.
Year(s) Of Engagement Activity 2017
URL http://unbias.wp.horizon.ac.uk/event/the-human-standard-why-ethical-considerations-should-drive-tech...
 
Description IGF meeting, Geneva (UNOG). 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Presentation about (Self) regulation of algorithmic systems that control online information access.
Year(s) Of Engagement Activity 2017
URL https://igf2017.sched.com/event/DCQW/self-regulation-of-algorithmic-systems-that-control-online-info...
 
Description If Oxford Science and Ideas festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Helena Webb participated in the IF Oxford Science and Ideas festival. She joined other members of the Human Centred Computing group to run a stand in a central Oxford shopping centre. She spent the afternoon telling interested members of the public about the UnBias and RoboTIPS projects and listening to their views on the responsible development of technologies.
Year(s) Of Engagement Activity 2019
 
Description Impact Workshops working with the Fairness Toolkit to explore issues of Online Fairness 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact As part of the Impact Exploration Grant award, a total of eight workshops were run with members of the general public to gain feedback on the Awareness Cards, which are a part of the Fairness Toolkit and enable people to have discussions about issues of online fairness.

Two versions of workshops (eight workshops in total lasting two hours each) were carried out with people aged 13-17 years, 18-29 years, 30-50 years and 65+ years). A total of 48 people attended the workshops.

Some workshop participants reported taking the Awareness Cards away to try out with others, including in some University Seminars.
Year(s) Of Engagement Activity 2018,2019
 
Description Inerviewed fro article in WIRED 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Following revelations of abusive content on YouTube children channels Nicole Kobie interviewed Ansgar Koene for an article in WIRED. The article was published on 24 November 2017.
Year(s) Of Engagement Activity 2017
URL https://www.wired.co.uk/article/youtube-kids-moderation-google
 
Description Institute for Policy and Engagement launch- University of Nottingham. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact The aim of the event was to raise the profile of the Institute with colleagues, particularly academics. In particular, it was an opportunity for academics colleagues already engaged in policy and public engagement work to share their work with their peers.

Members of the UnBias/ReEnTrust team were invited to be one of the policy impact/public engagement stars of the event, and a slide summarising with our main policy impact/public engagement outputs was on display during the event. Our work was also highlighted by Professor Dame Jessica Corner during her presentation, being one of the 4 policy impact/public engagement stars out of 29 exhibited on the event.
Year(s) Of Engagement Activity 2019
URL http://blogs.nottingham.ac.uk/researchexchange/2019/02/12/institute-for-policy-and-engagement-launch...
 
Description Internet Reseaerch Ethics - interviewed for article in Nature 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact In September 2017 Ansgar was contacted by Elizabeth Gibney, a journalist working for Nature, for an interview to contribute to an article she was writing on "Ethics of Internet research trigger scrutiny: Concern over the use of public data spurs guideline update.". The article was published on October 3rd 2017
Year(s) Of Engagement Activity 2017
URL https://www.nature.com/news/ethics-of-internet-research-trigger-scrutiny-1.22746
 
Description Interview at request of Journalist at Digital Future Society - a programme supported bythe Spanish Ministry of Economy/Mobile World Capital 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Dear Mr. Koene,

How are you? My name is Eva Dallo and I'm a Barcelona based journalist working for Digital Future Society, a programme supported by the Spanish Ministry of Economy and Business in collaboration with Mobile World Capital Barcelona that seeks to build an inclusive, equitable and sustainable future in the digital era.

Digital Future Society engages experts, policymakers, civic organisations, and entrepreneurs to explore, experiment, and exchange knowledge in four key areas: public innovation, digital trust and security, inclusion and citizen empowerment and equitable growth.

Kind regards,
Eva Dallo


I herewith get in touch regarding a series of interviews we are conducting with experts in the field of the future of work. We would like to know if you would be interested in participating- the interview consists of a Q&A with nine questions and a short introductory video where the interviewee introduces him/herself (specifications would follow).
-----
RESPONSE:
Interview questions for Digital Future Society - Spanish Ministry of Economy/Mobile World Capital
By Ansgar Koene, Global Ai Ethics and Regulatory Leader,
https://www.linkedin.com/in/akoene/

Could you give us an overview of your work?
As Global AI Ethics and Regulatory leader at AI my role is to help the AI Lab, and EY as a whole, to develop ethically responsible AI governance methods, produce thought leadership on AI governance and engage with national/regional Ai regulatory developments (e.g. European Commission; OECD AI policy observatory). As part of this work I also contribute to the development of AI related Standards. I chair the IEEE P7003 Standard for Algorithmic Bias Considerations working group, led the Bias Expert Working Group for the IEEE Ethics Certification Program for Autonomous and Intelligent Systems (ECPAIS), and participate in a number of working groups of the ISO/IEC JTC1 SC42 "Artificial Intelligence" sub-committee. I retain a part-time position at the University of Nottingham's Horizon Digital Economy Research Institute where I contribute to the institute's policy impact and support the stakeholder engagement work of the EPSRC funded ReEnTrust research project. As part of the work at the University of Nottingham, I was the lead author on a Science Technology Options Assessment report for the European Parliament on "A governance framework for algorithmic accountability and transparency" that was published in 2019. I am also a trustee for the 5Rights Foundation that

Are algorithms going to take over the labour market, and which areas if so?
Algorithms are having a profound impact on the shape of the labour market by enabling the automation of increasingly complex tasks that follow predicable and repetitive patterns. The current wave of AI systems, which are essentially complex statistical inference engines, has enabled an acceleration of the labour market automation process into domains that require higher levels of pattern recognition (and pattern reproduction) than was previously possible. In the Audit profession for instance Document Intelligence, which combined image processing and Natural Language Processing, is enabling the automation of much of the audit grunt-work of checking that required documents have been filled in properly and transferring information from those documents into databases for further assessment.

Are algorithmic decisions fair?
Fairness is a moral virtue that relates to the justifiability of decisions. Whether or not an algorithmic decision is fair depends on the context of the decision and the ways in which the system has been optimized by its creators. All algorithmic systems are built and programmed by people. What has changed with various forms of Machine Learning is the ways in which this programming is done. Supervised learning systems are programmed by means of the human labelled training data, reinforcement learning systems are programmed by the human defined goal and cost functions associated with the environment model it is provided with, etc.
As the UN Special Rapporteur for Extreme Poverty concluded in his report on the use of algorithmic decision making in social welfare allocation systems, the underlying problem with the use of these systems is that they have been introduced and optimised to implement a cost-cutting agenda at the expense of vulnerable populations. This agenda is a choice made by humans.
Algorithms do not make moral judgements, they execute instructions and optimise towards defined goals. Humans decide how much to prioritize false-positives over false-negatives. If historical data shows a correlation between criminal behaviour and divorced parents, it is humans who must decide if it is fair to penalize someone's chances of being released on parole based on something their parents did and which they most likely had no control over. It is a human societal choice if we want our criminal justice system to judge individuals based on gross statistical patterns across the populations, or if we want a system based on individual cause-and-effect analysis.

How can we guarantee the fairness of algorithms?
In our work on the IEEE Standard for Algorithmic Bias Consideration we phrase the work as intending to minimize unintended, unjustified and unacceptable difference in algorithmic decisions. In order to address these the development and deployment/use of algorithmic systems must:
• Have a sufficient understanding of the context of use, including who will be impacted by the system (does this include groups with different vulnerabilities?). This includes the need to re-assess system bias if there are significant changes to the context it is used in.
• Have a clear and complete understanding of the decisions that are made during the design/deployment and the implications of the criteria that the algorithm is optimizing for. A typical way in which lack of diversity in the development team may lead to bias is by failing to recognize a design choice is linked to cultural customs. Decisions may also end up being made outside of the system design phase during bug fixes, which might lead to a failure to thoroughly explore the implication of those decisions.
• In order to know if a decision or optimisation criterium is fair it is necessary to explore the justification for that decision. Ensure that this choice be justified to the people impacted by the use of the algorithmic system.
• Check if the justifications for the decisions are acceptable in the context where the system is used. Where possible this should include consultation with representatives of the various groups who have been identified as potentially being impacted by the use of the system.

An important element we added in the ECPAIS Bias certification criteria is the need for ongoing system behaviour monitoring and the ability to perform corrective interventions if unjustified bias is observed.

What's the problem with black box algorithms, and how do they affect the work market?
A key problem with black box algorithms, in regards to fairness, is the difficulty in understanding if a decision was justified or if it should be challenged. A recent article by Panoptykon Foundation nicely explained many of the issues regarding black box algorithms, including the reasons why many of these system need not be a black box as they currently are https://medium.com/@szymielewicz/black-boxed-politics-cebc0d5a54ad.
For the work market specifically, black box algorithms pose a problem when they are used in Human Resources contexts such as hiring, performance evaluation and other HR decisions. Similar to the criminal justice example eluded to earlier, many of the current black box algorithms used in these contexts employ machine learning methods that are trained with historical data to detect patterns in resumes or in observed behaviours that, at the population level, are statistically correlated with good employees. With black box algorithms it can be difficult to subsequently investigate if these correlations make causal sense (or are merely spurious correlations), and are justifiable grounds for the decisions (e.g. don't involve discrimination on proxies of protected characteristics such as sex, race, etc.).

Does AI jeopardize the rights of workers?
Referring back to the report of the UN Special Rapporteur on Extreme Poverty, AI jeopardizes the rights of workers if it is used as a tool to implement policy agendas aimed at reducing the rights of workers. This includes the use of black box algorithms as a means for obscuring the rationale behind decisions and frustrating the ability to challenge the justification of those decisions.

How can we protect the fundamental rights of workers while retaining freedom to innovate new algorithmic methods?
A key part to protecting the fundamental rights of workers in the light of algorithmic systems is to acknowledge that the rights of workers pertain to the justification, outcomes and impacts of decisions, irrespective of the tool that is used in the decision-making process.
Innovation of new algorithmic methods can improve the efficiency of processing, e.g. resume pre-screening, but does not change where the accountability for decisions lies, nor what the obligations are regarding the fundamental rights of workers.

Is state intervention needed to guarantee the transparency and accountability of algorithms as well as the liability of developers, or can the industry regulate itself?
The state has an important role in showing how existing rights and regulations remain valid independent of the use of algorithmic systems. In addition to possible general comments for elaborating how existing legislation should be interpreted in an algorithmic context, a key requirement will be upskilling of the regulatory agencies that are tasked with enforcement of existing legislation. The state remains important to guarantee transparency and accountability of decisions, with or without the use of algorithms. If this requirement is maintained, industry will step in to develop standards and best practices for the transparency of algorithms to enable them to be used while remaining compliant with the general requirements on decision making.
In some select domains or applications the use of algorithms may change the allocation of accountability and/or liability between developers and operators of algorithmic systems. For those cases it will be necessary to provide additional regulatory clarity.

Can workers actively protect themselves from algorithm based decisions?
An important step for workers to protect themselves from algorithm based decisions is to demand that their existing rights regarding explanation, justification and challenge of decisions must be respected irrespective of the use of algorithms in the decision process. Accountability lies with the people using the algorithms. Algorithm based decisions are not perfect and deserve to be challen
Year(s) Of Engagement Activity 2020
 
Description Interview conducted with Dr Ansgar Koene leading to a feature in an article in L'intelligence artificielle à l'heure de la transparence algorithmique | Les Echos | 09/03/2020 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Dr Koene was approached by a journalist from L'intelligence artificielle à l'heure de la transparence algorithmique | Les Echos and provided an interview which featured in a published article 9/3/2020:
Year(s) Of Engagement Activity 2020
URL https://www.lesechos.fr/idees-debats/sciences-prospective/lintelligence-artificielle-a-lheure-de-la-...
 
Description Interview conducted with Dr Ansgar Koene leading to a feature in an article in L'intelligence artificielle à l'heure de la transparence algorithmique | Les Echos | 09/03/2020 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Dr Koene was approached by a journalist from L'intelligence artificielle à l'heure de la transparence algorithmique | Les Echos and provided an interview which featured in a published article 9/3/2020
Year(s) Of Engagement Activity 2020
URL https://www.lesechos.fr/idees-debats/sciences-prospective/lintelligence-artificielle-a-lheure-de-la-...
 
Description Interview for BBC Scotland 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact I was interviewed by BBC Scotland for a discussion on the Digital Childhood Report
Year(s) Of Engagement Activity 2017
URL http://www.horizon.ac.uk/bbc-radio-scotland-interview-with-dr-elvira-perez-about-children-and-the-in...
 
Description Interview with Legal Publishing company online newsletter/blog 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Following mentoring a group of undergraduate students for a Legal Design Sprint (which they won) I was interviewed about terms and conditions online and whether they were fit for purpose.
Year(s) Of Engagement Activity 2018
URL https://www.justis.com/terms-and-conditions-fit-for-purpose/
 
Description Invited as a panel Member by CDEI (July 2019) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Dr Anasgar Koene was invited by CDEI to attend a panel in July 2019. The output was the briefing paper (see below)

From: @rusi.org>
Sent: Monday, September 16, 2019 12:57 PM
Subject: Published today: Data analytics and algorithmic bias in policing

Dear all,

I am pleased to let you know that our interim Briefing Paper, 'Data Analytics and Algorithmic Bias in Policing' (attached), has been published today and can be downloaded here: RUSI.org/PoliceAlgorith2019

To date, we have engaged with more than 50 subject matter experts, including police officers and staff, academics, technologists, legal experts, civil society organisations and policy-makers from across government. The research has revealed widespread concern regarding the lack of national guidance for police use of data analytics, with participants suggesting that this gap should be addressed as a matter of urgency. The paper identifies specific ways in which bias can be introduced at different stages in the project lifecycle, arguing that the issue cannot be understood solely as a matter of 'data bias', but requires careful consideration of the wider context in which a particular technology is implemented.

This paper is a summary of our interim findings, and as such we do not attempt to provide recommendations as to how the identified risks should be addressed. Our final report for this project will be published in early 2020, and will include a more detailed analysis of the findings, along with specific recommendations for future policy, guidelines and oversight mechanisms for police use of data analytics.

I would like to thank you all once again for your valuable contributions to this project. This paper contains only a small fraction of the content you have all provided to us over the past few months. We are in the process of analysing all the interview data in detail, to ensure our final report provides a comprehensive and accurate summary of the research findings. Please do let me know if you have any additional comments or feedback based on this interim briefing paper, which we will be sure to take into account when writing the final report.

All the best and many thanks again,

------ NB the report cites the Landscape Summary: Bias in Algorithmic Decision-Making' report CDEI, 2019.
Year(s) Of Engagement Activity 2019
URL https://rusi.org/publication/briefing-papers/data-analytics-and-algorithmic-bias-policing
 
Description Invited talk on Algorithmic Fairness, BHCC Symposium, Sheffield, UK, October 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Gave invited talk in Symposium on Biases in Human Computation & Crowdsourcing to an interdisciplinary audience of around 40 academics and postgraduate students.
Year(s) Of Engagement Activity 2019
URL https://sites.google.com/sheffield.ac.uk/bhcc2019/program
 
Description Keynote Speaker to Royal College Psychiatrists Annual Conference. Talk title 'Internet Adiction or Persuasive Design' 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact After the presentation I was invited to contribute to the RCP Report titled 'Technology use and the mental health of children and young people'
Year(s) Of Engagement Activity 2019
 
Description Keynote at Coalesce 21 conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Dr Ansgar Koene was invited to give a keynote at the Coalesce 21 Conference of the Goa Intsitute of Management
Year(s) Of Engagement Activity 2021
URL https://dare2compete.com/o/algorithmic-bias-diversity-and-inclusion-webinar-by-mr-ansgar-koene-ernst...
 
Description Kooth online mental health representations and ethics 2/2 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact This presentation presented a case study to illustrate the ethical challenges of online research. Kooth.com, and online platform that offer counseling services to children and young people was used as an example.
Year(s) Of Engagement Activity 2017
 
Description Legal Design Sprint mentor 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Undergraduate students
Results and Impact Mentor for team of undergraduates for 2 day legal design competition, in which they tackled the issue of terms and conditions being unfit for younger readers (and many adults)
Year(s) Of Engagement Activity 2018
URL https://www.justis.com/winning-idea-legal-design-sprint-2018/
 
Description Lets Talk About Tech Campaign video 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Aligning with Children's Mental Health Week (this week) and running up to Internet Safety Day on the 11th February, we are running a campaign to showcase our visionary research around internet safety, algorithms, and the impact on children's mental health and wellbeing. Digital mental health is a growing area of research at Nottingham, and in addition to this campaign supporting REF, it's an area we are looking to secure more funding in. Our research has already influenced policy with the introduction of the Age Appropriate Design Code, and we aim to continue to be influential in this space.
Year(s) Of Engagement Activity 2020
URL https://twitter.com/UoNresearch/status/1224991085656793095?s=20
 
Description Limited Resource Allocation Task Workshops- London, Edinburgh, Nottingham, Graz 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The limited resource allocation tasks were intended to bring together stakeholders in order to understand how they would respond to fairness in relation to the use of algorithms in a practical situation. Stakeholders were tasked with choosing an algorithm which they felt would produce the most appropriate outcome in particular contexts- and also, this allowed an exploration of how decisions varied with the "stakes" within a context were more or less consequential or serious.

There was so muc
Year(s) Of Engagement Activity 2017,2018
 
Description Massive Open Online Course on Data Ethics, AI, and Responsible Innovation 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Co-led and contributed to content of a new online course that introduces a broad (non-technical and technical) audience to data ethics, AI, and responsible innovation. The course addresses key application domains where algorithmic systems are making important decisions in healthcare, policing, finance, and smart living scenarios, and also provides a practical introduction to basic ethical concepts and frameworks.
Year(s) Of Engagement Activity 2020
URL https://www.edx.org/course/Data-Ethics-AI-and-Responsible-Innovation?utm_source=Data-Ethics-Twitter-...
 
Description Media interview with Defenddigitalme for Byline Festival - Targeting children: the dangers of letting tech into the playground 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Jen Persson Director from defenddigitalme (a call to action to protect children's rights to privacy. We are teachers and parents who campaign for safe, fair and transparent data processing in education, in England, and beyond. We advocate for children's data and digital rights, in response to concerns about increasingly invasive uses of children's personal information. The campaign is funded by an annual grant from the Joseph Rowntree Reform Trust Ltd) invited Ansgar Koene via email to do an interview for a radio program. Ansgars sections starts roughly around the 12minute mark - https://www.mixcloud.com/FI_PassW0rd/pity-the-poor-children-for-they-know-not-what-you-do/
Year(s) Of Engagement Activity 2019
URL https://www.mixcloud.com/FI_PassW0rd/pity-the-poor-children-for-they-know-not-what-you-do/
 
Description Memory Machine (Horizon Servcie Campaing) 4 workshops with people with dementia, carers, researchers (22 Nov, 21 Sept, 15 June, 27 April) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Patients, carers and/or patient groups
Results and Impact Following principles of user-centred and interaction designed, MeMa was tested among the general population but also among dementia carers. To do that, we ran a series of focus groups to understand the context of use (e.g., when, why, how); barriers/enablers to use; relevant types of 'core' content and so on. Through a series of iterative exercises, the project developed from low-fidelity prototypes to explore how users would engage with the product and alternative physical / screen-based interfaces. We developed a tablet-based working prototype, with accessible core content, basic interface and functionality to upload and record additional content (i.e., audio, images, sound bites, etc.).
Year(s) Of Engagement Activity 2018
URL https://www.horizon.ac.uk/project/the-memory-machine/
 
Description Memory Machine Poster at UK Dementia Congress 6-8 Nov 2018 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Patients, carers and/or patient groups
Results and Impact Poster at UK Dementia Congress 6-8 Nov 2018
Year(s) Of Engagement Activity 2018
 
Description Mental Health series at BBC Radio Nottingham 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact During this radio program was interviewed about the research I do to promote creative practices as mutual recovery.
Year(s) Of Engagement Activity 2017
 
Description Mindtech 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Presentation on how Databox can alleviate ethical and privacy challenges in mental health care.
Year(s) Of Engagement Activity 2017
URL http://www.mindtech.org.uk/mindtech-annual-conference.html
 
Description MozFest event drop in session 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact MozFest is an annual event run by the Mozilla Foundation to support and promote the open internet movement. At the 2017 event in London UnBias team members Helena Webb, Menisha Patel, Helen Creswick, Virginia Portillo, Elvira Perez and UnBias collaborator Giles Lane ran a drop in session for attendees called 'How do you take care on the internet'. The session combined elements of UnBias the ESRC project Digital Wildfire to prompt attendees to reflect on their responses to issues connected to safety and fairness on the internet - for instance in relation to excessive use of personalisation algorithms and the spread of fake news across social media. Over one hundred attendees joined the session over the course of half a day and they had the opportunity to learn more about the project by taking information leaflets and talking to team members.
Year(s) Of Engagement Activity 2017
 
Description Navigating through Uncertainty and Unawarness - Blog for eNurture Network+ 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Blog to signal Internet Safety Day
Year(s) Of Engagement Activity 2020
URL https://www.enurture.org.uk/blog/2020/2/5/navigating-through-uncertainty-and-unawareness
 
Description Older people discuss how Smart City Nottingham could bring benefits to health and wellbeing 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Study participants or study members
Results and Impact During this event organised by Age-Friendly Nottingham and that coincided with 'Older peoples day' was the perfect venue to launch our report
Year(s) Of Engagement Activity 2017
 
Description Online mental health representations and ethics 1/2 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact This was a presentation on online ethics part of the network 'Hungry for words'. The presentation focused on the challenges of securing online consent from participants
Year(s) Of Engagement Activity 2017
 
Description Open-space and Citizens Juries to explore older peoples views on optimising health and wellbeing benefits for a Smart City Nottingham 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This poster was presented at the MindTech Symposium
Year(s) Of Engagement Activity 2017
 
Description Organising/Chairing Panel Discussion at HealTAC conference, Cardiff April 24-25 2019 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact I was asked to organize a Panel Discussion at HealTAC conference titled: Natural language processing in mental health: progress, challenges and opportunities.
Year(s) Of Engagement Activity 2019
 
Description Participation in an Ada Lovelace Panel and Q&A session 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Took place in a UoN Computer Science Ada Lovelace Day (27.11.2019) - disseminated UnBias Awareness cards among audience and established new contacts for new (ReEnTrust) project
Year(s) Of Engagement Activity 2019
 
Description Policy Connect Stakeholder engagement workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Dr Ansgar Koene was invited to attend a closed stakeholder engagement workshop in October chaired by Eve Lugg, Policy Manager for Data at Policy Connect: https://www.policyconnect.org.uk/

The roundtable will discuss the recently launched consultation on post-brexit reforms to the UK's data protection regime, with a specific focus on responsible innovation.

"Following the recent publication of the consultation on reforms to create an ambitious, pro-growth and innovation-friendly data protection regime that underpins the trustworthy use of data, we are hosting a roundtable to discuss this in relation to responsible innovation. Considering the UK's need to introduce agile and adaptable data protection laws that maintain high data protection standards without creating unnecessary barriers to responsible data use, we think that responsible innovation needs to be central to the conversation. Given the above, we would like to invite you to the roundtable to partake in what will be a fascinating discussion. "
Year(s) Of Engagement Activity 2021
URL https://www.policyconnect.org.uk/
 
Description Poster presentation: "A multi-stakeholder perspective on the regulation and design of algorithm fairness" , TRILcon17 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Sparked questions and discussions about the novel results presented in the poster, part of the UnBias' stakeholder first workshop. Moreover, this poster received the 'Best Poster Award'.
Year(s) Of Engagement Activity 2017
URL http://unbias.wp.horizon.ac.uk/wp-content/uploads/2016/08/Unbias_TRILCon17_Poster.pdf
 
Description Poster presentation: "Children's rights in the digital world - What do children and young people want to enjoy a safe Internet?". Children, Rights and Childhood network, Birmingham. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact We provided research evidence which explored young people's views and concerns when interacting with internet services such as social media.
The event focused on children's rights and run within the "Child Rights Week". Our work generated lots of interest and discussion from researchers and participants working with young children. In particular, it was very inspiring to meet and discuss Children's Digital Rights with the key note speaker for UNICEF (Martin Russell), educators, and members of the public sector.
Year(s) Of Engagement Activity 2018
URL https://www.birmingham.ac.uk/schools/government-society/events/2018/children-rights-and-childhood.as...
 
Description Poster: Algorithm Fairness in Online Information Systems, WebSci conference, NY. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This poster sparked relevant discussions. It was also presented at "The growing ubiquity of algorithms in society: implications, impacts and innovations" - scientific meeting at the Royal Society, October 30-31, London, UK.
Year(s) Of Engagement Activity 2017
URL http://websci17.org/
 
Description Presentation at ECREA Symposium on Digital Democracy: critical perspectives in the age of new data, held at Södertön University, Stockholm, Nov. 2017 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Academic presentation of 'UnBias' research data which lead to relevant questions and discussion afterwards. It also allowed networking with colleagues in the filed for future research collaborations.
Year(s) Of Engagement Activity 2017
URL https://communicationanddemocracy.wordpress.com/2017/01/09/call-for-papers-ecrea-symposium-digital-d...
 
Description Presentation at Hungry for Words final workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This was the final activity of the workshop which concluded on a round table that engaged a representative and inclusive group interested in male eating disorders
Year(s) Of Engagement Activity 2018
 
Description Presentation at Internet Society - European Chapters meeting 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact In February 2017 the European chapters of the Internet Society (ISOC) held their annual conference to discuss join activities and coordinate projects for the year. In 2017 Ansgar Koene represented the ISOC UK-England chapter with a presentation on "Fake News". Ansgar was invited to lead Uk based activities on User Trust, the ISCO thematic agenda of the year.
Year(s) Of Engagement Activity 2017
URL http://unbias.wp.horizon.ac.uk/event/internet-society-european-chapters-meeting/
 
Description Presentation at MyData conference - Advanced Literacy - Basic education for the digital age 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Dr Ansgar Koene delivered a workshop using the UnBias Awarness Cards. Post conference, he received correspondence below from the conference organiser in response to a request about the cards:

Cards were forwarded to the contact.

Response from Conference Organiser
"The cards are great aren't they! They were produced from work by Ansgar Koene and colleagues - I cc him directly and he can let you know if it is possible to access any or whether it was a small production, as I don't know.

Email correspondence to Conference organiser:
"I am now writing you on behalf of a colleague Bettie Koebler who took notice of the workshop/discussion cards that are in the video (please see the screenshot below). We were wondering if that is a resource that is open or you would be willing to share with our organisation to be used in future workshops as well. (We would of course always respectfully reference the source). And of course we would also completely understand - if it is an internal tool and can't be shared. Please let us know"
Year(s) Of Engagement Activity 2019
URL https://mydata2019.org/presentations/
 
Description Presentation to Colombian Association of Radiology, 4th Congress on Artificial Intelligence 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The Colombian Association of Radiology held a Congress on Artificial Intelligence to discuss state of the art within the field. This was attended by clinicians, academics, students and others. Helena Webb was invited to give a presentation on the importance of ethics in AI. She talked about her interests in this area, the necessity to consider ethical and responsibility issues in relation to AI and gave an overview of projects that she has been involved in that seek to support ethical conduct and outcomes in AI. In particular, she described the co development of the UnBias fairness cards and their use in fostering reflection amongst practitioners,
Year(s) Of Engagement Activity 2022
URL https://www.acrvirtual.org/ia2022
 
Description RT UK - TV News and Social media - interview regarding NHS Wannacry ransomware 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Following the nationwide disruption of NHS services, and subsequent problems in other countries, due to a 'WannaCry' ransomware attack RT UK contacted Ansgar Koene to provice 'expert commentary' on the possible origins, impacts and technical safeguards that should be put in place. The interview was broadcast on 12 May 2017 as a segment of the regular news show. Ansgar has subsequently be asked to provide comment as "Cyber expert" on numerous occasions.
Year(s) Of Engagement Activity 2017
URL https://www.rt.com/uk/
 
Description RT UK - TV News and Social media - interview regarding accusation by US Senate of spying by Kasperski software 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Following the breaking news that the cybersecurity software firm Kasperski was being accused by the US of using its anti-malware software to spy on the US Senate, and thus forbidding further Federal Government use of Kasperski products, RT UK contact Ansgar to provide commentary on the technical feasibility of using anti-malware software for spying. The brief TV interview was broadcast on 19 September 2017.
Year(s) Of Engagement Activity 2017
URL https://www.rt.com/uk/
 
Description RT UK - TV News and Social media content - Debate on UK parliament proposal to regulate Social Media like new media. Koene A. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact RT UK channel contacted Ansgar to invite him to participate in a short TV debate regarding the UK parliament proposal that government should regulate social media companies as traditional media companies. The TV debate was broadcast on TV and the RT UK social media channel on 13 December 2017.
Year(s) Of Engagement Activity 2017
URL https://youtu.be/3Yj6ImETQPQ
 
Description Report for Young People- "Youth Juries: What we learned from you" 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Report that highlights the main findings from a total of 27 Youth Juries (focus groups) run by our research team between 2017-2018. Over 250 children and young people (mostly aged between 13-17 years of age) took part in the Youth Juries sessions. This report is written for the general public, in particular for young people, and includes recommendations for what actions should be taken to make the internet more child-friendly. It was launched in October 2018 and it's been available online since. It has been very welcomed within participants that took part in the Youth Juries, as well as teachers and Third Sector organisations such as the 5Rights Foundation.
Year(s) Of Engagement Activity 2017,2018
URL https://unbias.wp.horizon.ac.uk/wp-content/uploads/2018/10/Youth-Juries-what-we-learned-from-you.-Re...
 
Description Research presentation at Rhodes House, Oxford 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Helena Webb and Marina Jirotka were invited to give a talk at Rhodes House, University of Oxford. They spoke about controversies surrounding algorithmic bias and described the key findings of the UnBias study. They then joined a discussion about algorithmic governance.
Year(s) Of Engagement Activity 2018
 
Description Rinzing art exhibition 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Patients, carers and/or patient groups
Results and Impact This is an photography exhibition launched at the Institute of Mental that explores mental health and wellbeing through street photography.The exhibition was launch at Eating Disorders week to bring awareness on male eating disorders.
Year(s) Of Engagement Activity 2018
 
Description STEM ambassador 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact I am a STEM ambassador as part of the scheme run by STEM Learning. I take part in STEM engagement events under this scheme.
Year(s) Of Engagement Activity 2018,2019,2020
 
Description STEM festival, Bluecoat Beechdale Academy, Nottingham 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Activity designed for young children to help them to learn how the internet works and to reflect on issues of personal information and online filter bubbles, by creating their own "data gardens". While children enjoyed this activity, parents/guardians and teachers engaged with the UnBias research team on interesting discussions including how the nature of the content that they view online often determines what is shown to them. Adults expressed concerns about how to better manage their own and their children online experiences, and requested more information about online privacy and security issues. Our team provided information and a take away leaflet that gave tips with suggestions of tools and solutions to stop companies from tracking users.
Year(s) Of Engagement Activity 2018
URL https://unbias.wp.horizon.ac.uk/2018/06/18/how-to-create-your-own-data-garden/
 
Description School Visit (Bilborough college, Nottingham) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Drop-in session at Bilborough college to promote and recruit participants for the next UnBias Youth Juries (focus groups to reflect, debate and share ideas), taking place at the UoN in March 2017. 35 student expressed their interest to participate.

Staff expressed interest for the students to participate in future events within our project and in related research areas.
Year(s) Of Engagement Activity 2017
 
Description School outreach activities at University of Oxford 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact UnBias team member Helena Webb regularly gives talks about the project as part of the University of Oxford's outreach programme. School students from across the country travel to the University to take part in taster sessions. During these sessions students learn about ongoing projects and have the opportunity to ask questions about them. In the case of presentations given by Dr Webb about UnBias, they are encouraged to debate issues around fairness on the internet and put forward their own views.
Year(s) Of Engagement Activity 2016,2017,2018
 
Description Science in the Park, British Science Association (Wollaton Park, Nottingham) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Family event for all ages. One-to-one contact with parents/guardians, children of all ages, grandparents, teenagers and young adults. Participants expressed their views engaging in interesting discussions about online fairness/bias. In particular parents/guardians were very keen finding out more information about how the internet shapes their online experiences and what users can do about it.
Year(s) Of Engagement Activity 2018
URL https://unbias.wp.horizon.ac.uk/page/3/
 
Description Sky News TV interview for bulletin on NHS Wannacy Ransomware attack 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Following the nationwide disruption of NHS services due to a 'WannaCry' ransomware attack Sky News contacted Ansgar Koene to provice 'expert commentary' on the possible origins, impacts and technical safeguards that should be put in place. The interview was broadcast on 12 May 2017 as segment of a rolling news coverage of the event. Ansgar was contacted again by Sky News on 9 September for comment regarding the Equifax hack.
Year(s) Of Engagement Activity 2017
 
Description Smart Because of You. FoSAC and NUCLEUS (RRI, Horizon 2020) conference, Nottingham. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Presentation about the UnBias Youth Juries (YJ): methodology and data. Also, a general overview of the UnBias project highlighted the wide range of participants that so far have engaged/participated in our project including: young people (mainly 13-17 yrs old), a multi-stakeholder panel including citizens from the public sector, academics, industry, media, policy makers, NGOs, etc. This activity generated interesting questions and discussion and helped to establish new contacts for further collaborations. General interest expressed by the audience and on online social network (twitter).
Year(s) Of Engagement Activity 2017
 
Description Stakeholders group organised by NCC 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Participation in the Nottingham Tech city steering group: a multi disciplinary team of about 15 people from the public and private sectors, academics, NGOs, etc., aiming to explore civic technology in Nottingham. Established new contacts for possible future collaborations
Year(s) Of Engagement Activity 2017
 
Description Studium Generale presentation at TU Eindhoven. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Invited presentation to undergraduate and general audience, open to the public, at the University of Eindhoven. This talk was part of Holland Design Week at Eindhoven and also qualified for credit for undergraduate students at TU Eindhoven as part of their Technology Impact on Society foundations course. The event was attended by 350 people, student and staff a of TU Eindhoven and eternal people. The talks spakred a lively discussion. After the talk Ansgar was invited to participate in a one-week "Lorentz workshop" at the University of Leiden on "Intersectionality and Algorithmic Discrimination" that took place in December 2017
Year(s) Of Engagement Activity 2017
URL http://www.studiumgenerale-eindhoven.nl/nl/agenda/archief/the-age-of-the-algorithm/0/1109/
 
Description TRT Wrold News TV interview related to NHS and globa Wannacry randsomware attacks 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Following the nationwide disruption of NHS services, and subsequent problems in other countries, due to a 'WannaCry' ransomware attack TRT World contacted Ansgar Koene to provice 'expert commentary' on the possible origins, impacts and technical safeguards that should be put in place. The interview was broadcast on 12 May 2017 as a segment of the regular news show.
Year(s) Of Engagement Activity 2017
URL https://www.trtworld.com/
 
Description Talk at HPC Professionals lunch and learn event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dr Ansgar Koene was invited by the Society of High Performance Computing (HPC) Professionals to speak at a lunch and learn event 27 May 2021 'Complex society reduced to simplistic binary stereotypes': https://hpcsociety.org/may-2021-event/
Year(s) Of Engagement Activity 2021
URL https://hpcsociety.org/may-2021-event/
 
Description Tectopia Festival - Digiplay workshop, London 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Five hour workshop and for 19 children aged 7-12 years. Interactive and fun workshop where children engaged learning about online algorithms.
Year(s) Of Engagement Activity 2018
 
Description Telling Tales of Engagement 2017 award- update 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact The purpose of this project is to establish whether watching Human Rights related videos is effective in raising awareness, increasing people knowledge and engagement with Human Rights related issues. The fieldwork for this project is now complete.
A total of 60 participants took part in two surveys, which were sent approximately one week apart. Brief interviews were conducted with a total of eleven participants immediately after they completed the first survey. A mini report of the preliminary results (qualitative and qualitative) has been produced and a research paper with the collected data will be written up and submitted to an appropriate journal.
Year(s) Of Engagement Activity 2019
URL https://unbias.wp.horizon.ac.uk/2019/12/22/telling-tales-of-engagement-raising-awareness-of-human-ri...
 
Description The Conversation article We asked young people what they want from the internet of the future - here's what they said 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact This article at The Conversation highlights that the Internet is not fit for children and online services have to be more child friendly. There is an amendment for the Data Protection bill that has been influenced by articles like this one.
Year(s) Of Engagement Activity 2017
URL https://theconversation.com/we-asked-young-people-what-they-want-from-the-internet-of-the-future-her...
 
Description The Human Bias in AI. Change Forum: Product and Data, London 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Interactive workshop mainly aimed to media industry stakeholders. The aim of the session was to build understanding and promote discussion around the use of algorithms in online platforms. Members of our team run activities using the UnBias Awareness Cards, designed to encourage critical and civic thinking for exploring how decisions are made by algorithms, and the impact that these decisions may have on our lives and the lives of others. The audience engaged in interesting discussions and the facilitators from our research team were invited to take part in future related activities (RCUK workshops).
Year(s) Of Engagement Activity 2019
URL https://www.changeforum.co/
 
Description The Internet and You - widening participation 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact This was an opportunity to showcase our work 'the internet and you' at an event organised by the University of Nottingham - our audience was mainly primary school aged children
Year(s) Of Engagement Activity 2019
 
Description The Internet on our own terms. Press release 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact This was the media press release for the launch of the report 'The Internet On Our Own Terms'.
Year(s) Of Engagement Activity 2017
URL http://www.nottingham.ac.uk/news/pressreleases/2017/february/young-people-call-on-government-for-gre...
 
Description Thresholds broadcast 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact BBC Click Fifty shades of Spray video coverage and interview with Mat Collishaw stimulated increased interest and raised awareness
Year(s) Of Engagement Activity 2017
URL https://www.bbc.co.uk/iplayer/episode/b08s72bm/click-fifty-shades-of-spray
 
Description Two-Day Ethical Hackathon Event Edinburgh 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact The two-day event in Edinburgh was held in order to bring together interdisciplinary teams to tackle design challenges. Importantly, as with the one-day events, this event was a twist on the traditional hackathon model (necessitating the integration of ethics). The event was structured through presentations in the morning, and then the challenge was set, students had the change to work through the night if they so wished, and on the following day they were tasked with presenting their work to a panel of judges. There were also mentors involved in the event who had different expertise, in order to help students understand the remit of the challenge, and in particularly guide them as to how to integrate ethics into their design process and outcome.

The purpose of the event was educational, in the sense of emphasising the importance of considering ethics in relation to algorithmic design. The attendees were very engaged. We were also able to conduct some interviews at the end of the event, and elicit feedback through questionnaires. The response on the whole was positive, with attendees feeling that they had acquired new knowledge through the day.
Year(s) Of Engagement Activity 2018
 
Description USACM panel on Algorithmic Transparency and Accountability. Washington DC. 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The USACM panel on Algorithmic Transparency and Accountability was organized by the US policy SIG of the ACM to communicate the activities that the ACM is doing on Ethics of Algorithmic Decision making to congressional staff members. The even was held at the National Press Club in Washington DC. As part of the panel they invited Ansgar Koene to represent, and talk about, the activities of the IEEE Global Initiative for Ethics of Autonomous and Intelligence systems. The event was attended by about 60 people in person and also live streamed to a global audience. Member of the in-room audience included journalists from the Financial Times and Wired as well as junior staff members working in the offices of members of congress. Following the event Ansgar was interviewed, by e-mail, by a writer for the ACM's AI Matters blog (https://sigai.acm.org/aimatters/blog/2017/10/01/ieee-and-acm/). Following from this ACM/IEEE collaboration the two professional organizations set of an informal grou pto develop future joint activities to promote the work that both organizations are doing on Ethics is AI which Ansgar is a part of.
Year(s) Of Engagement Activity 2017
URL https://www.acm.org/public-policy/algorithmic-panel
 
Description UnBias Awareness Cards workshop at SOAS, University of London. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact Workshop run by two members of our team and requested by a lecturer from SOAS. The aim was to use the UnBias Awareness Cards to bring awareness of online bias, in particular to reflect about freedom of speech online and how algorithms bias what we see. This activity stimulated lots of discussion and engagement within the audience. It also inspired a participant to write a reflective blog for the SOAS coding group website: "What's Fair in the Age of Big Data?" (https://www.soascodingclub.com/soas-coding-club-blog/2018/10/22/whats-fair-in-the-age-of-big-data-fm5w5).
Year(s) Of Engagement Activity 2018
URL https://www.soascodingclub.com/soas-coding-club-blog/2018/10/22/whats-fair-in-the-age-of-big-data-fm...
 
Description UnBias Final Project Event 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact This event was the official final event of the UnBias project. It brought together a range of stakeholders (from a range of different areas, and some quite high profile) in order to showcase project outcomes through presentations and panel discussion. It was a very engaging event held at the Digital Catapult in London, that stimulated discussion about the project outcomes, and then algorithms more generally, amongst those who attended. We were also able to record key interviews and presentations- they are on the project website. Participants were very keen to hear more about ReEnTrust, and many have requested continued participation as stakeholders on this project. Our tangible record of the project of the website will we hope to reach those who were not able to attend the event.
Year(s) Of Engagement Activity 2018
 
Description UnBias Youth Advisory Group 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Note: This is a follow up of the UnBias Youth Advisory Group submitted in 2018.

In total, seven Advisory Group meetings took place between from November 2017-February 2019 with up to 4 participants. Meetings were held periodically throughout this time period to seek feedback and advice from young people aged 16-18 years of age. The young people have given feedback on project recruitment literature, materials and resources used in the wave 2 Youth Juries and feedback on materials included in the UnBias Fairness Toolkit. The Group also reviewed a report for Young People and gave feedback to make sure that the report was as accessible as possible to this age group.Their participation has been extremely valuable for our team as co-creation and delivering accessible material for young people are pillars in our research work.
Year(s) Of Engagement Activity 2017,2018,2019
 
Description UnBias Youth Juries (11 sessions- second wave) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact A total of 116 pupils aged between 13-17 years attended 11 interactive youth-led workshops. Sessions focused on issues of online fairness and algorithm bias in automated decision systems, in particular the impact of these on participants' online experiences.
These workshops sparked questions, debate, reflection and recommendations from participants on how issues of algorithm bias should be tackled. Young people's feedback was very positive as most of them expressed sessions were very interesting and educational and participants' learnt new things applicable in their online behaviours. Participants also felt inspired to learn more about the topics discussed in these sessions. One of the main outcomes was that it lead to the write up and launch of a report for young people "Youth Juries: What we learned from you".
Year(s) Of Engagement Activity 2018
URL https://unbias.wp.horizon.ac.uk/wp-content/uploads/2018/10/Youth-Juries-what-we-learned-from-you.-Re...
 
Description UnBias Youth Jury at the Explorers Fair (Festival of Science and Curiosity, Nottingham) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Undergraduate students
Results and Impact 30 young people attended the youth jury at the Broadway cinema, Nottingham. The juries are youth-lead focus groups designed to stimulate debate giving participants the chance to share their views and express their concerns about a particular topic. In this case the theme was algorithm fairness.

Some of the participants from this jury expressed their interest to take part in future Youth Juries and/or other related events.
Year(s) Of Engagement Activity 2017
URL http://unbias.wp.horizon.ac.uk/event/explorers-fair-expo/
 
Description UnBias Youth Jury at the National Video Game Arcade 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Undergraduate students
Results and Impact 30 young people attended the youth jury at GameCity. The juries are designed to stimulate discussion and debate among jurors. Some of the participants from this jury were recontacted to attend the launch at the House of Lords 'The Internet on our own terms' and answer questions from the audience.
Year(s) Of Engagement Activity 2016
 
Description UnBias mentioned in the media -The Institute 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact UnBias project mentioned in The Institute, the IEEE news source blog which informs members about the latest news and trends in technology and engineering and shares opinions about what's going on in these areas.
Year(s) Of Engagement Activity 2017
URL http://theinstitute.ieee.org/ieee-roundup/blogs/blog/keeping-bias-from-creeping-into-code
 
Description UnBias youth advisory group 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Advisory group opened to young people to contribute to the co-creation of new scenarios for the UnBias Youth Juries. Three meetings took place between November 2017 - February 2018 and four students (16-18 years old) attended. They provided valuable feedback on their experiences when taking part of previous Youth Juries' session and we have used this feedback to make some modifications to the design of the second wave of Juries run in Feb-March 2018. Members of the advisory group have also given feedback on project recruitment literature, materials and resources in general.
Year(s) Of Engagement Activity 2017,2018
 
Description University of Nottingham Vision magazine - Safe Space 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Safe space
Researchers are placing the views and experiences of young people at the heart of policy, making the internet a safer place

This publication is for the purpose of disseminating research conducted at the University of Nottingham to a wider - both academic and non academic audience
Year(s) Of Engagement Activity 2019
URL https://www.nottingham.ac.uk/vision/safe-space
 
Description University of Oxford - Women in Computer Science day 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact The University of Oxford runs an annual Women in Computer Science Day. Young women from around the country aged 14+ attend with their schools or parents to learn about studying computer science and careers in the field. In the 2017 event Helena Webb and Menisha Patel gave a talk about the UnBias project and ran an stand in an exhibition where attendees could find our more information about UnBias and the ESRC Digital Wildfire project. The event was attended by around 200 young women. They had the opportunity to take information leaflets about the project and ask questions to Drs Webb and Patel.
Year(s) Of Engagement Activity 2017
 
Description V&A Digital Design Weekend, London 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Drop-in activities for all, including a range of case studies that related to fake news, hate speech, search results and personalised content. This sparked lively debate and discussion and participants were invited to vote on how some of these issues should be tackled.
A hands-on activity for children (3-13 year olds) was aimed to raise awareness and help children and parents/guardians to reflect on how personal information is used by websites and online systems and how this shapes their online experiences.
Year(s) Of Engagement Activity 2018
URL https://www.vam.ac.uk/event/6YVLW34q/digital-design-weekend-2018-ldf
 
Description Vision - Safe Space 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Interview (by Dr Elvira Perez) and subsequent article for University of Nottingham Vision Magazine to highlight the present and historical research activity addressing protecting young people and children online (placing the views and experiences of young people at the heart of policy, making the internet a safer place).
Year(s) Of Engagement Activity 2019
URL https://www.nottingham.ac.uk/vision/safe-space
 
Description Visit to DCMS 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Researches at the University of Nottingham were invited, together with CDT students, to DCMS to provide presentations and expert advice on:

Monetising Services - Joseph Hubbard-Bailey, Kate Green
• How do companies providing a 'free' service make money?
• How is data monetised?
• How do cookies and advertising work?

Moderating online services - Ansgar Koene, Elvira Perez Vallejos, Liz Dowthwaite
• How does China prevent access to sites?
• How does AI (AI and other things like machine learning) work? How is it used to moderate content?
• How does the process of trusted flaggers work?

Encryption - Derek McAuley
• How does encryption, including end-to-end encryption affect the ability to identify harmful content?
• Where are online services going in terms of encryption and what risks does this bring?
• What is DOH and how might it impact the ability of companies/regulators to moderate content?
Year(s) Of Engagement Activity 2019
 
Description WP3 Observational Studies 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact The observational activities formed a part of WP3- and investigating how users experience algorithms in their online browsing experience. We conducted a series of quasi-naturalistic experiments, mostly with post-graduate students within Oxford. We were also able to attend a youth jury in Nottingham, where we were able to conduct such an experiment with a group of young people.

In each case, pairs of participants were asked to undertake a task using a laptop- they were observed doing so, and audio-visual data was collected as well as screen capture recordings. We were then able to undertake some supplementary informal interviews to engage with their experiences. In each case users were very interested to discuss what they had experienced during the study- the young people in the youth jury were able to understand more about what algorithms are and use their participation in the task as a basis for discussion in the later parts of the jury.

We have been able to understand more about how users interact with algorithms through our analysis of the data collected; and we feel that young people and the students alike were able to understand and engage more with how they themselves experience algorithms.
Year(s) Of Engagement Activity 2017,2018
 
Description When AI goes to war 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact We presented a poster outlining the work done to interrogate young people about AI applied to military artifacts.
Year(s) Of Engagement Activity 2016
 
Description When AI goes to war 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact This was a youth jury to discuss the benefits and problems of having killer robots fully autonomous among young people
Year(s) Of Engagement Activity 2017
 
Description Widening Participation Family Discovery Day. University of Nottingham 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Hands-on activity designed for children aged 3-13 years to help them to learn how the internet works and to reflect on issues of personal information and online filter bubbles, by creating their own "data gardens".
While children enjoyed this activity, parents/guardians and teachers engaged with the UnBias research team on interesting discussions including how the nature of the content that they view online often determines what is shown to them. Adults expressed concerns about how to better manage their own and their children online experiences, and requested more information about online privacy and security issues. Our team provided information and a take away leaflet that gave tips with suggestions of tools and solutions to stop companies from tracking users.
Year(s) Of Engagement Activity 2018
URL https://unbias.wp.horizon.ac.uk/2018/06/18/how-to-create-your-own-data-garden/
 
Description Workshop at TRILcon17- Trust, information and the law 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Stakeholders workshop that aid reflexion and discussion about algorithm fairness throughout three case studies. This sparked pertinent discussion between academics, lawyers, y members from the public sector. Two groups of about 15 participants each attended this activity. Ansgar Koene's contribution in this conference was mentioned by Michael Cross in 'The Law Society Gazette': https://www.lawgazette.co.uk/news-focus/news-focus-algorithms-and-schrödingers-justice/5060964.article
Year(s) Of Engagement Activity 2017
URL http://unbias.wp.horizon.ac.uk/event/trilcon-2017/
 
Description Workshop to 3rd Age University 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Third sector organisations
Results and Impact We provided an overview of Horizon research which included hands on activities to approximately 30 members of the 3rd Age University
Year(s) Of Engagement Activity 2019
 
Description Workshop: Platform-to-business trading practices workshop (EC Digital Single Market) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Workshop organized by the European Commissions Directorate General for the Digital Single Market (DG connect) to explore the possible need for action by the European Commission against anticompetitive behaviour by online platform providers. The workshop invited companies that list their products or services on online platforms (e.g. hotels) and academics and NGOs who have done work related to online platforms, to discuss possible anticompetitive practices. Ansgar Koene was invited by members of the DG connect team to participate in this workshop following his EuroDIG flash session of Algorithm Regulation.
Year(s) Of Engagement Activity 2017
 
Description Workshops: UnBias Youth Juries (2 sessions), held in Nottingham as part of the Festival of Social Science (ESRC), Nov. 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Interactive youth-led workshops which sparked questions, debate, reflexion and recommendations from participants for an internet more user-centred. Participants' feedback was very positive as most of them expressed sessions were very interesting and educational and learnt new things applicable in their work/studies. Participants also felt inspired to learn more about the topics discussed in this activity. One of the main outcomes was that it lead to the creation of a youth advisory group with some of the participants that took part in these workshops.
Year(s) Of Engagement Activity 2017
URL http://www.esrc.ac.uk/public-engagement/festival-of-social-science/
 
Description Wp4 Stakeholder Engagement Workshops 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact As a core part of WP4 we undertook a series of stakeholder workshops in London. The purpose of these workshops was to provide a space in which stakeholders (from our multi-stakeholder panel) could discuss issues surrounding algorithmic fairness. Importantly, we also allowed stakeholders (drawn from areas such as policy, education, NGOs, etc) to also deliberate on what ideas they had for some of the solutions to mitigating problematic issues arising from the use of algorithms. We were able to reach a very large number of stakeholders, many of whom continue to be part of our stakeholder panel for the follow on ReEnTrust Project.
Year(s) Of Engagement Activity 2017,2018
 
Description Yoga in children homes 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Many yoga related sites cite this study.
Year(s) Of Engagement Activity 2016
URL http://www.nottingham.ac.uk/news/pressreleases/2016/december/yoga-can-have-social-benefits-for-child...
 
Description Young People's Policy Recommendations on Algorithm Fairness. WebSci conference, NY. 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The presentation was past of the WebScience confernece in Troy NY, USA which was attended by mostly academics but with some industry and NGO participants interested in the latest state of the art in academic thinking on information flows and user engagement with information on the world wide web. The Unbias talk on "Young People's Policy Recommendations on Algorithm Fairness" provided a strong contribution to the conference theme of Ethics in WebScience, with various members of the previous the Ethics Panel from the previous evening approaching after the talk for further information about our work.
Year(s) Of Engagement Activity 2017
URL https://unbias.wp.horizon.ac.uk/dissemination/presentations/
 
Description Youth Juries Open Educational Resource 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We have created an Open Educational Resource for educators to deliver the youth juries in their own institutions
Year(s) Of Engagement Activity 2016
URL https://uyj.wp.horizon.ac.uk/
 
Description Youth Jury, part of Festival of Science and Curiosity, Nottingham. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Eight participants (aged 13-17) attended this session held at Nottingham Central Library. It was a very interactive and fun youth led-discussion to reflect about algorithmic bias present on daily used online systems and platforms. Participants were very keen and engaged in very relevant discussions, shared their views and experiences, and provided recommendations for an ideal internet.
Year(s) Of Engagement Activity 2017
URL https://www.horizon.ac.uk/unbias-workshop-at-the-festival-of-science-and-curiosity-17-feb-2018/
 
Description Youth Panel competition: "Our Future Internet" 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact Competition opened to school pupils from years 7 to 13 to respond to our 'UnBias' animation Our Future Internet: http://bit.ly/OurFutureInternet. It offered participants a chance to win up to a £100 prize voucher.

The aim was to raise awareness within young people of some of the very serious issues that are prevalent in their use of the Internet. Participants (working in groups or individually) could choose the medium through which they respond: digital media including a short video, visual media or written media, to cover any relevant topic related to the animation.

We received ten submissions from 19 pupils. The quality of work was very high and entries were reviewed by a panel of judges that included Mark Highman - Director of Film Literacy Europe and adviser to the 5Rights Foundation, and two researchers from our team.

Feedback from teachers that supported this activity was very positive. We presented the prizes to the winners during a special event co-organised for this purpose at the relevant school.
Year(s) Of Engagement Activity 2018
URL https://www.horizon.ac.uk/unbias-youth-panel-video-competition-how-do-algorithms-affect-your-online-...