Turing AI Fellowship: Trustworthy Machine Learning
Lead Research Organisation:
UNIVERSITY OF CAMBRIDGE
Department Name: Engineering
Abstract
Machine learning (ML) systems are increasingly being deployed across society, in ways that affect many lives. We must ensure that there are good reasons for us to trust their use. That is, as Baroness Onora O'Neill has said, we should aim for reliable measures of trustworthiness. Three key measures are:
Fairness - measuring and mitigating undesirable bias against individuals or subgroups;
Transparency/interpretability/explainability - improving our understanding of how ML systems work in real-world applications; and
Robustness - aiming for reliably good performance even when a system encounters different settings from those in which it was trained.
This fellowship will advance work on key technical underpinnings of fairness, transparency and robustness of ML systems, and develop timely key applications which work at scale in real world health and criminal justice settings, focusing on interpretability and robustness of medical imaging diagnosis systems, and criminal recidivism prediction. The project will connect with industry, social scientists, ethicists, lawyers, policy makers, stakeholders and the broader public, aiming for two-way engagement - to listen carefully to needs and concerns in order to build the right tools, and in turn to inform policy, users and the public in order to maximise beneficial impacts for society.
This work is of key national importance for the core UK strategy of being a world leader in safe and ethical AI. As the Prime Minister said in his first speech to the UN, "Can these algorithms be trusted with our lives and our hopes?" If we get this right, we will help ensure fair, transparent benefits across society while protecting citizens from harm, and avoid the potential for a public backlash against AI developments. Without trustworthiness, people will have reason to be afraid of new ML technologies, presenting a barrier to responsible innovation. Trustworthiness removes frictions preventing people from embracing new systems, with great potential to spur economic growth and prosperity in the UK, while delivering equitable benefits for society. Trustworthy ML is a key component of Responsible AI - just announced as one of four key themes of the new Global Partnership on AI.
Further, this work is needed urgently - ML systems are already being deployed in ways which impact many lives. In particular, healthcare and criminal justice are crucial areas with timely potential to benefit from new technology to improve outcomes, consistency and efficiency, yet there are important ethical concerns which this work will address. The current Covid-19 pandemic, and the Black Lives Matter movement, indicate the urgency of these pressing issues.
Fairness - measuring and mitigating undesirable bias against individuals or subgroups;
Transparency/interpretability/explainability - improving our understanding of how ML systems work in real-world applications; and
Robustness - aiming for reliably good performance even when a system encounters different settings from those in which it was trained.
This fellowship will advance work on key technical underpinnings of fairness, transparency and robustness of ML systems, and develop timely key applications which work at scale in real world health and criminal justice settings, focusing on interpretability and robustness of medical imaging diagnosis systems, and criminal recidivism prediction. The project will connect with industry, social scientists, ethicists, lawyers, policy makers, stakeholders and the broader public, aiming for two-way engagement - to listen carefully to needs and concerns in order to build the right tools, and in turn to inform policy, users and the public in order to maximise beneficial impacts for society.
This work is of key national importance for the core UK strategy of being a world leader in safe and ethical AI. As the Prime Minister said in his first speech to the UN, "Can these algorithms be trusted with our lives and our hopes?" If we get this right, we will help ensure fair, transparent benefits across society while protecting citizens from harm, and avoid the potential for a public backlash against AI developments. Without trustworthiness, people will have reason to be afraid of new ML technologies, presenting a barrier to responsible innovation. Trustworthiness removes frictions preventing people from embracing new systems, with great potential to spur economic growth and prosperity in the UK, while delivering equitable benefits for society. Trustworthy ML is a key component of Responsible AI - just announced as one of four key themes of the new Global Partnership on AI.
Further, this work is needed urgently - ML systems are already being deployed in ways which impact many lives. In particular, healthcare and criminal justice are crucial areas with timely potential to benefit from new technology to improve outcomes, consistency and efficiency, yet there are important ethical concerns which this work will address. The current Covid-19 pandemic, and the Black Lives Matter movement, indicate the urgency of these pressing issues.
Organisations
- UNIVERSITY OF CAMBRIDGE (Lead Research Organisation)
- Simmons Wavelength Limited (Project Partner)
- GNS Healthcare (Project Partner)
- Clifford Chance LLP (UK) (Project Partner)
- DeepMind (Project Partner)
- Max Planck Institutes (Project Partner)
- University of Alicante (Project Partner)
- Massachusetts Institute of Technology (Project Partner)
- Chelsea & Westminster Hospital NHS Trust (Project Partner)
People |
ORCID iD |
Adrian Weller (Principal Investigator / Fellow) |
Publications

Ali Shahin Shamsabadi
(2023)
Confidential-PROFITT: Confidential PROof of FaIr Training of Trees

Ali Shahin Shamsabadi
(2023)
Mnemonist: Locating Model Parameters that Memorize Training Examples

Antoran J
(2021)
Getting a CLUE: A Method for Explaining Uncertainty Estimates

Ashurst C
(2023)
Fairness Without Demographic Data: A Survey of Approaches

Avin S
(2021)
Filling gaps in trustworthy development of AI.
in Science (New York, N.Y.)

Babbar V
(2022)
On the Utility of Prediction Sets in Human-AI Teams


Bai X
(2021)
Advancing COVID-19 diagnosis with privacy-preserving collaboration in artificial intelligence
in Nature Machine Intelligence

Bakker M
(2021)
Beyond Reasonable Doubt

Barker M
(2023)
Selective Concept Models: Permitting Stakeholder Customisation at Test-Time
in Proceedings of the AAAI Conference on Human Computation and Crowdsourcing
Description | Many publications covering aspects of trustworthy machine learning including fairness, explainability, robustness, privacy and scalability to real-world problems. Contributions to many workshops and advisory boards. Contributions to regulation and policy. |
Exploitation Route | I hope others build on all my work and outcomes. |
Sectors | Communities and Social Services/Policy Digital/Communication/Information Technologies (including Software) Financial Services and Management Consultancy Healthcare Government Democracy and Justice |
URL | https://mlg.eng.cam.ac.uk/adrian/ |
Description | Contributed to: regulators, governance and policy; public outreach, advocacy and education. |
First Year Of Impact | 2021 |
Sector | Digital/Communication/Information Technologies (including Software),Financial Services, and Management Consultancy,Healthcare,Government, Democracy and Justice |
Impact Types | Cultural Societal Economic Policy & public services |
Description | Advisory board member of the Centre for Data Ethics and Innovation |
Geographic Reach | National |
Policy Influence Type | Participation in a guidance/advisory committee |
Impact | I have provided advice since 2018, continuing now. Reports include CDEI bias review, Nov 2020 https://www.gov.uk/government/publications/cdei-publishes-review-into-bias-in-algorithmic-decision-making CDDO algorithmic transparency standard, produced with help of CDEI, Nov 2021 https://www.gov.uk/government/collections/algorithmic-transparency-standard |
URL | https://www.gov.uk/government/collections/algorithmic-transparency-standard |
Description | Co-organized and co-hosted workshops with data protection and healthcare regulators to discuss good governance |
Geographic Reach | National |
Policy Influence Type | Contribution to new or Improved professional practice |
Impact | Contributing to updates on governance standards being developed. More information should be available later. |
Description | Contributed to Science and Technology Select Committee on reproducibility of research |
Geographic Reach | National |
Policy Influence Type | Contribution to a national consultation/review |
Description | Contributed to the Justice and Home Affairs Committee's inquiry on new technologies and the application of the law |
Geographic Reach | National |
Policy Influence Type | Contribution to a national consultation/review |
URL | https://committees.parliament.uk/writtenevidence/39076/pdf/ |
Description | Joined the Global Partnership on AI (GPAI) as an expert, contributing to Responsible AI |
Geographic Reach | Multiple continents/international |
Policy Influence Type | Participation in a guidance/advisory committee |
URL | https://gpai.ai/ |
Description | Joined the Steering Committee for the All-Party Parliamentary Group on Data Analytics' (APGDA) current inquiry on AI Ethics and Growth |
Geographic Reach | National |
Policy Influence Type | Participation in a guidance/advisory committee |
Description | Panelist at LawtechUK roundtable panel on the Adoption of AI in Legal Services |
Geographic Reach | National |
Policy Influence Type | Contribution to new or improved professional practice |
Description | Responsible AI (RAI) UK strategy group |
Geographic Reach | National |
Policy Influence Type | Participation in a guidance/advisory committee |
Description | Specialist adviser to Lords select committee on AI in weapon systems |
Geographic Reach | National |
Policy Influence Type | Contribution to a national consultation/review |
URL | https://committees.parliament.uk/committee/646/ai-in-weapon-systems-committee/ |
Description | WEF Global Future Council on the Future of AI |
Geographic Reach | Multiple continents/international |
Policy Influence Type | Participation in a guidance/advisory committee |
URL | https://www.weforum.org/communities/global-future-council-on-artificial-intelligence/ |
Description | Accenture AI Leaders Podcast on Responsible AI |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Accenture AI Leaders Podcast on Responsible AI |
Year(s) Of Engagement Activity | 2022 |
URL | https://aileaders.libsyn.com/ai-leaders-podcast-14-responsible-ai |
Description | Co-organised Alan Turing Institute 2021 Workshop on Interpretability, Safety, and Security in AI |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Co-organised a two day event that brought together leading researchers along with other stakeholders in industry and society to discuss issues surrounding trustworthy artificial intelligence. The conference presented an overview the state-of-the-art within the wide area of trustworthy artificial intelligence including machine learning accountability, fairness, privacy, and safety; including an overview of emerging directions in trustworthy artificial intelligence, and engage with academia, industry, policy makers, and the wider public. The conference had two parts, running on the two consecutive days. The first was an academic-oriented event followed by a public-oriented event. |
Year(s) Of Engagement Activity | 2021 |
URL | https://www.turing.ac.uk/events/interpretability-safety-and-security-ai |
Description | Co-organised ELLIS Human-Centric Machine Learning Workshop |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Co-organised a day-long workshop of talks and panels on human-centric machine learning. The workshop brought together leading experts, from diverse backgrounds, at the forefront of two themes: - The differential treatment by algorithms of historically under-served and disadvantaged communities - The development of machine learning systems to assist humans for better performance, rather than replace them. The day included talks followed by Q&A and panel discussions with audience participation. |
Year(s) Of Engagement Activity | 2021 |
URL | https://sites.google.com/view/hcml2021 |
Description | Co-organised NeurIPS 2021 Workshop on AI for Science: Mind the Gaps |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Co-organised a one day workshop on gaps that stifle AI advancement: - Unrealistic methodological assumptions or directions. - Overlooked scientific questions. - Limited exploration on the intersections of multiple disciplines. - Science of science. How AI can facilitate the practice of scientific discovery itself is often undiscussed. - Responsible use and development of AI for science. The goal of this workshop aimed to bridge each of the above gaps via: - Discussing directions in ML that are likely/unlikely to have an impact across scientific disciplines and identify reasons behind them. - Bringing to the front key scientific questions with untapped potential for use of ML methodological advances. - Pinpointing grand challenges at the intersection of multiple scientific disciplines (biology, chemistry, physics, neuroscience, etc). - Highlighting how ML can change or complement classic scientific methods and transform the science of scientific discovery itself. The workshop included invited and contributed talks, poster sessions and a panel discussion. A mentorship program was also facilitated. 52 workshop papers were accepted for this workshop. |
Year(s) Of Engagement Activity | 2021 |
URL | https://ai4sciencecommunity.github.io/ |
Description | Co-organised NeurIPS 2021 Workshop on Human-Centered AI |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Co-organised a one day workshop aimed at bringing together researchers and practitioners from the NeurIPS and Human-Centered AI (HCAI) communities and others with convergent interests in HCAI. The workshop had an emphasis on diversity and discussion, and explored research questions that stem from the increasingly wide-spread usage of machine learning algorithms across all areas of society, with a specific focus on understanding both technical and design requirements for HCAI systems, as well as how to evaluate the efficacy and effects of HCAI systems. The workshop had 20 accepted papers under four themes: Ethics, Human(s) and AI(s), Methods, and XAI Explainable AI. |
Year(s) Of Engagement Activity | 2021 |
URL | https://sites.google.com/view/hcai-human-centered-ai-neurips/home |
Description | Co-organised NeurIPS 2021 Workshop on Privacy in Machine Learning |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Co-organised a one-day workshop the focused on privacy-preserving machine learning techniques for large-scale data analysis, both in the distributed and centralized settings, and on scenarios that highlight the importance and need for these techniques (e.g., via privacy attacks). The workshop included talks from invited speakers, panel discussions and presentations of accepted workshop papers. 36 papers were accepted for the workshop. |
Year(s) Of Engagement Activity | 2021 |
URL | https://priml2021.github.io/ |
Description | Co-organized Alan Turing Institute 2023 Workshop on Privacy and Fairness in AI for Health |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Over the past few years, the use of AI has continued to increase in healthcare services. Many healthcare providers are deploying AI-driven applications across various business lines, services, and products. Although AI can bring a range of benefits to healthcare providers and patients, AI can also raise novel risks for patients and even healthcare providers. The UK government is working hard to regulate AI in health to support the use of responsible and trustworthy AI. By doing so, we can unleash the full potential of AI while safeguarding our fundamental values and keeping us safe and secure. The underlying goal of our deep dive workshop is to bring together privacy and fairness experts working in academia, industry, and universities to Mitigate the risks of unfair decisions and information leakage when machine learning is deployed by healthcare providers; and Design auditing frameworks. In service of these goals, our workshop will: Surface cutting-edge research and challenges regarding fairness and privacy to regulators Articulate the challenges faced by regulators to the research community in order to inspire new research directions and information sharing Promote networking between researchers, and between researchers and policymakers, in the hope of new mutually beneficial collaborations. (1) and (2) will be achieved through short presentations, Q&A, and an interdisciplinary panel discussion. (3) will be achieved through networking opportunities throughout and after the event. |
Year(s) Of Engagement Activity | 2023 |
URL | https://private-fair-ai.github.io/ |
Description | Co-organized ICAIF 2022 Workshop on Explainable AI in Finance |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Explainable AI (XAI) forms an increasingly critical component of operations undertaken within the financial industry, brought about by the growing sophistication of state-of-the-art AI models and the demand that these models be deployed in a safe and understandable manner. The financial setting brings unique challenges to XAI due to the consequential nature of decisions taken on a daily basis. There are two encompassing dimensions to this: macro-financial stability and consumer protection. Financial markets transfer enormous amounts of assets on a daily basis. AI-powered automation of a substantial fraction of these transactions, especially by big players in key markets, poses a risk to financial stability if the underlying mechanisms driving market-moving decisions are not well understood. This may trigger a crisis-risking meltdown in the worst-case scenario. At the same time, and just as important as macro-stability, is consumer protection. Automation within the financial sector is tightly regulated: in the US consumer credit space, the Equal Credit Opportunity Act (ECOA), as implemented by Regulation B, demands that explanations be provided to consumers for any adverse action by a creditor; in the EU, consumers have the right to demand meaningful information for automated decisions under the General Data Protection Regulation (GDPR). Safe and effective usage of AI within finance is thus contingent on a strong understanding of theoretical and applied XAI. Currently, there is no industry standard consensus on which XAI techniques are appropriate to use within the different parts of the financial industry - or if indeed the current state-of-the-art is sufficient to satisfy the needs of all stakeholders. This workshop aims to bring together academic researchers, industry practitioners and financial experts to discuss the key opportunities and focus areas within XAI - both in general and to face the unique challenges in the financial sector. |
Year(s) Of Engagement Activity | 2022 |
URL | https://sites.google.com/view/2022-workshop-explainable-ai/home |
Description | Co-organized ICAIF 2023 Workshop on Explainable AI in Finance |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Explainable AI (XAI) forms an increasingly critical component of operations undertaken within the financial industry, brought about by the growing sophistication of state-of-the-art AI models and the demand that these models be deployed in a safe and understandable manner. The financial setting brings unique challenges to XAI due to the consequential nature of decisions taken on a daily basis. There are two encompassing dimensions to this: macro-financial stability and consumer protection. Financial markets transfer enormous amounts of assets on a daily basis. AI-powered automation of a substantial fraction of these transactions, especially by big players in key markets, poses a risk to financial stability if the underlying mechanisms driving market-moving decisions are not well understood. This may trigger a crisis-risking meltdown in the worst-case scenario. At the same time, and just as important as macro-stability, is consumer protection. Automation within the financial sector is tightly regulated: in the US consumer credit space, the Equal Credit Opportunity Act (ECOA), as implemented by Regulation B, demands that explanations be provided to consumers for any adverse action by a creditor; in the EU, consumers have the right to demand meaningful information for automated decisions under the General Data Protection Regulation (GDPR). Safe and effective usage of AI within finance is thus contingent on a strong understanding of theoretical and applied XAI. Currently, there is no industry standard consensus on which XAI techniques are appropriate to use within the different parts of the financial industry - or if indeed the current state-of-the-art is sufficient to satisfy the needs of all stakeholders. This workshop aims to bring together academic researchers, industry practitioners and financial experts to discuss the key opportunities and focus areas within XAI - both in general and to face the unique challenges in the financial sector. |
Year(s) Of Engagement Activity | 2023 |
URL | https://sites.google.com/view/2023-workshop-explainable-ai/home |
Description | Co-organized ICML 2022 Workshop on Human-Machine Collaboration and Teaming |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | Assemble inter-disciplinary experts and stimulate research into this exciting field |
Year(s) Of Engagement Activity | 2022 |
URL | https://sites.google.com/view/icml-2022-hmcat/home |
Description | Co-organized Turing+ICO Workshops on Fairness and Data Protection |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Policymakers/politicians |
Results and Impact | From November 2021 to January 2022, The Alan Turing Institute and the Information Commissioner's Office (ICO) hosted a series of workshops about fairness in AI. The aim of the series was to assist the ICO in the development of an update to the fairness component of its existing non-statutory guidance on AI and data protection, by convening key stakeholders from industry, policy, and academia to identify the key issues and challenges around fairness in AI, and to discuss possible solutions. The guidance is part of ICO25's Action Plan for 2022-2023. The update to the guidance has now been published, and the ICO will ensure any further updates will reflect upcoming changes in relation to AI regulation and data protection. The series consisted of three workshops divided by three broad themes, though there was significant overlap between each discussion. The first discussion sought to establish the parameters of what we mean by 'fairness' and 'unfairness' in the context of AI and UK data protection, the second asked how unfairness might be mitigated, and the final workshop focused on the impact of human decision-making on fairness in AI. |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.turing.ac.uk/data-protection-ai-and-fairness |
Description | Radio 4 Rutherford and Fry discussant on AI alignment following Stuart Russell's final Reith lecture |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | Radio 4 Rutherford and Fry discussant on AI alignment following Stuart Russell's final Reith lecture |
Year(s) Of Engagement Activity | 2021 |
URL | https://www.bbc.co.uk/programmes/m0012q27 |
Description | Turing Podcast on AI in the Financial Sector |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | With Dr Adrian Weller (Programme Director and Turing Fellow) and Kate Platonova (Group Chief Data Analytics Officer at HSBC), Ed Chalstrey discusses how AI is being used in financial services and what data is useful in banking today. Series 3, Episode 9 of the Turing podcast, URL below |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.turing.ac.uk/news/turing-podcast |