UKRI Trustworthy Autonomous Systems Node in Governance and Regulation

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Informatics

Abstract

How can we trust autonomous computer-based systems? Autonomous means "independent and having the power to make your own decisions". This proposal tackles the issue of trusting autonomous systems (AS) by building: experience of regulatory structure and practice, notions of cause, responsibility and liability, and tools to create evidence of trustworthiness into modern development practice. Modern development practice includes continuous integration and continuous delivery. These practices allow continuous gathering of operational experience, its amplification through the use of simulators, and the folding of that experience into development decisions. This, combined with notions of anticipatory regulation and incremental trust building form the basis for new practice in the development of autonomous systems where regulation, systems, and evidence of dependable behaviour co-evolve incrementally to support our trust in systems.

This proposal is in consortium with a multi-disciplinary team from Edinburgh, Heriot-Watt, Glasgow, KCL, Nottingham and Sussex, bringing together computer science and AI specialists, legal scholars, AI ethicists, as well as experts in science and technology studies and design ethnography. Together, we present a novel software engineering and governance methodology that includes:
1) New frameworks that help bridge gaps between legal and ethical principles (including emerging questions around privacy, fairness, accountability and transparency) and an autonomous systems design process that entails rapid iterations driven by emerging technologies (including, e.g. machine learning in-the-loop decision making systems)
2) New tools for an ecosystem of regulators, developers and trusted third parties to address not only functionality or correctness (the focus of many other Nodes) but also questions of how systems fail, and how one can manage evidence associated with this to facilitate better governance.
3) Evidence base from full-cycle case studies of taking AS through regulatory processes, as experienced by our partners, to facilitate policy discussion regarding reflexive regulation practices.
 
Description Adivising on and serving as member of the Legal Technologist Accreditation stream of the Law Society of Scotland
Geographic Reach National 
Policy Influence Type Contribution to new or Improved professional practice
Impact The new accreditation route recognises special expertise in technology and AI for lawyers. Uniquely, not just for the UK but internationally, it is an accreditation route administered by a professional and regulatory body (the Law Society of Scotland) that is open to non-lawyers. It incentivises training and investment in technology by law firms, assists in their digital transformation, while ensuring that professional standards are met. While there is an ongoing discussion whether there computer science should become a regulated profession, law took the lead in setting up a new way to bring software developers into the regulatory remit of a profession
URL https://www.lawscot.org.uk/members/career-growth/specialisms/areas-of-specialism/accredited-legal-te...
 
Description Evidence Submission on UK National Data Strategy
Geographic Reach National 
Policy Influence Type Gave evidence to a government review
Impact Submission feeds into an ongoing process of deliberation about the best route forward for UK post Brexit in relation to data protection policy. Our contribution provides input into this debate.
URL https://nottingham-repository.worktribe.com/output/6785230/response-to-the-governments-data-a-new-di...
 
Description Evidence to Justice and Home Affairs Committee on New technologies and the application of the law
Geographic Reach National 
Policy Influence Type Participation in a national consultation
Impact Contribution to evidence seeking process of UK government, which informs policymaking in this area.
URL https://committees.parliament.uk/writtenevidence/38857/pdf/
 
Description TAS consultation response to IP for AI generated works
Geographic Reach National 
Policy Influence Type Participation in a national consultation
URL https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4003626
 
Description Written Evidence for Government Review
Geographic Reach National 
Policy Influence Type Gave evidence to a government review
URL https://committees.parliament.uk/writtenevidence/43469/pdf/
 
Description Optos 
Organisation Optos plc
Department Optos
Country United Kingdom 
Sector Private 
PI Contribution Our Node Co-I Crabree and PDRA McGarry have been conducting fieldwork with the AI development team at Optos, as part of our cross-cutting theme investigations
Collaborator Contribution Optos have given access to our team, and also engaged with them to explain the development process and permit observational studies as part of the ethnographic work
Impact Fieldwork reports entered with paper outputs
Start Year 2021
 
Description Thales 
Organisation Thales Group
Department Thales UK Limited
Country United Kingdom 
Sector Private 
PI Contribution Fieldwork study by Co-I Crabtree and PDRA McGarry
Collaborator Contribution Time and participation of experts to assist with fieldwork, especially on
Impact Report added to publications - informs the design of our next stage experiments
Start Year 2021
 
Description AI in Health and Care: Safe and Effective Deployment 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Workshop aims to
- identify and discuss some of the critical components of safe and effective deployment of AI in health and social care
- when algorithm is in use; applies to integrating, piloting and routine use
- does not apply to algorithm development

Proposed outcomes
Connections, awareness & knowledge of each other's work
Discussing tricky problems or 'critical components' to:
•Deepen collective understanding
•Share problems we see from respective vantage points
•Map potential solutions
•Sum up with areas for further inquiry / allow conversation to continue
Year(s) Of Engagement Activity 2021
 
Description Automation and Autonomous Systems Roundtable 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact UKRI has commissioned the National Coordinating Centre for Public Engagement (NCCPE) to convene a Roundtable to bring together colleagues from across UKRI and the wider research and innovation community, with interest in the areas of Automation and Autonomous systems (AAS). The work is being undertaken as part of the Sciencewise1 public dialogue programme, which is delivered by UKRI, to serve the insight and evidence needs of research funders, and policymakers.

The purpose of the roundtable is to build a shared understanding of where public perspectives may be most critically needed in AAS and identify areas where further work is necessary with the possible support of the Sciencewise team. It will also explore how UKRI might best approach future cross-cutting areas like automation, which have strong public interest and significant ethical sensitivity.
Year(s) Of Engagement Activity 2022
 
Description Chairing session: What does AI offer people as they get older and what is the role of co-design? 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact AI Scotland Summit session: 'What does AI offer people as they get older and what is the role of co-design?
Aims
The aims of our participation in this conference are to:
• engage with a wider audience of people interested in AI matters, including Scottish Government, third sector, companies and other academia; and
• raise the profile of the research to support greater impact and could lead to other opportunities for speaking, partnerships etc.
Year(s) Of Engagement Activity 2022
URL https://www.scottishaisummit.com/
 
Description Decision Making, Planning and Trustworthy Autonomous Systems 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact UDRC Themed Meeting on Autonomous Systems
As part of the UDRC phase III, a themed meeting on Autonomous Systems will be held on Wednesday 24th November 2021. This will be an in person event with the option to attend online. This event is intended for academic researchers, industrial partners and Dstl staff to learn about and discuss current trends in Autonomous Systems. The program will
consists of a series of talks from academia and defence industry.
Year(s) Of Engagement Activity 2021
URL https://udrc.eng.ed.ac.uk/udrc-themed-meeting-autonomous-systems
 
Description Expecting the Future Presentation at 4S conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presented the paper 'Expecting the Future: Emerging Narratives on Automated Facial and Emotion Recognition Technologies.' with D. Miranda. at the leading Science and Technology Studies event the 4S Conference 2021 Toronto, Canada/Online.
Year(s) Of Engagement Activity 2021
URL https://www.research.ed.ac.uk/en/activities/4s-2021
 
Description Governance and Regulation: Bridging the Gaps between Law and Ethics 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Participated in a panel discussion today on Governance and Regulation: Bridging the Gaps between Law and Ethics, alongside AVM Tamara Jennings, Director Legal Services (RAF), Rhodri Morgan from HSE and chaired by Professor Chris Watkins, Professor in Computer Science, Royal Holloway.
Year(s) Of Engagement Activity 2021
URL https://www.tas.ac.uk/eventslist/trusting-machines/trust-machines-conference-programme/
 
Description Guest Talk: Logic & Learning: From Aristotle to Neural Networks 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk to the doctoral training center UnRAVeL - UNcertainty and Randomness in Algorithms, VErification and Logic in Aachen (Virtual).
Year(s) Of Engagement Activity 2021
URL https://www.unravel.rwth-aachen.de/cms/UnRAVeL/Das-Graduiertenkolleg/Aktuelle-Veranstaltungen/~rjrar...
 
Description Invited lecture on 'Regulating for Human Centered AI' 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited lecture on 'Regulating for Human Centered AI' at the '2021 Bilgi University IT Law International Summer School on Artificial Intelligence'. Istanbul, Turkey / Online.
Year(s) Of Engagement Activity 2021
URL https://aipractice.bilgi.edu.tr/en/
 
Description Invited talk at the First Annual LAST-JD-RIOE Conference of the Marie Sklodowska-Curie Action program "Rights in Internet of Everything" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The event was part of the Marie Sklodowska-Curie Action program "Rights in Internet of Everything", a EU project that sponsors a 3 year PhD program involving 15 European Universities It was attended by past and current students on the program, and also other PhD and Master students from participating institutions. I received several expressions of interest to come to the UK for a PhD in this field
Year(s) Of Engagement Activity 2021
URL https://rioe.oeg.fi.upm.es/conference/#contact
 
Description Optos 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dialogue with Optos about margins of error in machine learning
Year(s) Of Engagement Activity 2021
 
Description Organised and ran an international day long workshop on 'Human Data Interaction Through Design: An Exploratory from Theory to Practice Using Design as a Vehicle' at 'ACM SIGCHI Conference 2021'. Yokohama, Japan/Online. 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Workshop with experts to define agenda for HDI research in HCI - involved industry, academia and beyond. Resulted in a Manifesto for action and community building follow on work.
Year(s) Of Engagement Activity 2021
URL http://designresearch.works/chi2021-hdi-workshop/
 
Description Participating in ReMeP workshop as invited particpant on roundtable on "Computational Law" 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Participated in a panel discussion on computational law as part of ReMeP - research meets practice. The aim was to introduce governments and legislators to the potential of computational law, and industry on the business opportunities to support this new form of lawmaking. The panel was attended by 140 participants A follow up request from the Austrian government to explore the possibility of a computational version of the GDPR
Year(s) Of Engagement Activity 2021
URL https://remep.live/
 
Description Seminar on legal ontologies given at ReMeP: Research Meets Practice 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact ReMeP is an annual event to bring research in law and technology to practice. It is attended mainly by representatives from industry, legal practice and government Chaired panel and gave keynote talk on Legal Ontologies. 470 participants, lead to extended questions and discussions, and 2 follow up invitations Recordings of the talks are made available as training tools
Year(s) Of Engagement Activity 2020
URL https://www.remep.net/materials-2020/
 
Description Seminar: Situated Communication 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited seminar by Alex Lascarides at the Universite de Paris (virtual).
Year(s) Of Engagement Activity 2021
 
Description TAS Conversations - EU AI Act 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact AI is changing the world we live in - how do we prepare for that? What are the risks? And how do we make sure that we create trustworthy AI? Watch this hour of top-level debate and discussion on the EU AI Act - the first-ever legal framework on artificial intelligence.
Year(s) Of Engagement Activity 2021
URL https://www.tas.ac.uk/bigeventscpt/eu_ai_act/
 
Description Talk: Safe and secure software systems - An automated reasoning perspective 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Andrew Ireland gave this talk as part of the Heriot-Watt Summer Talk Series (Dubai) that was also promoted by the British Computer Society.
Year(s) Of Engagement Activity 2021
URL https://www.bcs.org/events/2021/june/webinar-safe-and-secure-software-systems-an-automated-reasoning...
 
Description Thales 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dialogue with Thales about cyber-security in autonomous systems, reported at Node annual meeting inc. partners
Year(s) Of Engagement Activity 2021
 
Description Turing Trustworthy AI forum 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Explore challenges in 'Human-Machine teaming'

- How is an AI/ML prediction different (or not) to a decision?
- What are the different human-machine management models?
- What does it mean for a human to manage an AI-powered decision?
Year(s) Of Engagement Activity 2021
 
Description Workshop: a Recreational Airspace Users Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The main item on the agenda for discussion was regulation for allowing drones to fly in the same airspace as manned aircraft. Alex Lascarides contributed to the discussion by talking about the TAS Governance Node work on simulation and testing, and on how users can interact with the simulation environment to find situations where systems might fail. The user experience that learned about at this meeting will inform our work on supporting user interaction with the simulation environment we're developing
Year(s) Of Engagement Activity 2021
 
Description Workshop: Carnegie Trust Workshops on Artificial Intelligence and International Rule of Law 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The Artificial Intelligence and the International Rule of Law Workshop Series aims to explore fundamental questions about the relationship between international legal obligations and emerging applications of artificial intelligence (AI) that bear upon key domains of international law: the laws of war, human rights and refugee law.
Year(s) Of Engagement Activity 2021
 
Description invited panellist on 'Trustworthy AI by Design' at the 'Centre for Internet Governance Research Annual Conference' 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Invited panellist on 'Trustworthy AI by Design' at the 'Centre for Internet Governance Research Annual Conference'. University of Sussex, England/Online
Year(s) Of Engagement Activity 2021
 
Description workshop: Governance and regulation of software as medcal device/AI as medical device 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Problematising: aiming to identify key issues surrounding software/AI-based medical devices and the difficultiesthat may arise for governance and regulation. The goal of this session to gain a clear view of the main problems; participants have different perspectives on these issues and the session is an opportunity to clarify the key challenges facing us.
Solutioneering: consider potential solutions to the challenges and implications for governance and regulation.
Year(s) Of Engagement Activity 2022