Responsible AI for Long-term Trustworthy Autonomous Systems (RAILS): Integrating Responsible AI and Socio-legal Governance
Lead Research Organisation:
University of Oxford
Department Name: Engineering Science
Abstract
Society is seeing enormous growth in the development and implementation of autonomous systems, which can offer significant benefits to citizens, communities, and businesses. The potential for improvements in societal wellbeing is substantial. However, this positive potential is balanced by a similar potential for societal harm through contingent effects such as the environmental footprint of autonomous systems, systemic disadvantage for some socio-economic groups, and entrenchment of digital divides. The rollout of autonomous systems must therefore be addressed with responsibilities to society in mind. This must include engaging in dialogue with society and with those affected, trying to anticipate challenges before they occur, and responding to them.
One such anticipated challenge is the effect of change on autonomous systems. Autonomous systems are not designed to be deployed in conditions of perfect stasis, as they are unlikely to encounter such conditions in real-world environments. They are frequently designed for changing environments, like public roads, and may also be designed to change themselves over time, for instance by means of learning capabilities. Not only that, but these changes in deployed systems and in their operating conditions are also likely to take place against a shifting contextual background of societal alteration (e.g. other technologies, 'black swan' events, or simply the day-to-day operation of communities). The effects of such change, on the systems themselves, on the environments within which they are operating, and on the humans with which they engage, must be considered as part of a responsible innovation approach.
The RAILS project brings together a team from UCL and the Universities of York, Leeds and Oxford, from multiple disciplines, with the aim of engaging with the challenges associated with the long-term operation of autonomous systems and the effects of change on these systems. In particular, we will explore how the notion of responsibility is affected by
(i) open-ended dynamic environments - situations that change over time, and
(ii) lifelong-learning systems - i.e. systems that are designed to adapt themselves to their circumstances and 'learn' over time.
The RAILS project will focus on such independent long-term autonomous systems in different applications. These will include (i) autonomous vehicles and (ii) autonomous robot systems such as unmanned aerial vehicles (drones). RAILS will look at social and legal contexts, as well as technical requirements, in order to assess whether and how these systems can be designed, developed, and operated in a way that they are responsible, accountable, and trustworthy.
The overall aim of the RAILS project is to bring together responsible development principles with governance mechanisms and technical understanding to create new understandings of how autonomous systems can adapt to change, how they can be deployed in a responsible and trustworthy way, and how such deployment can be framed by governance to ensure accountability and flexibility.
One such anticipated challenge is the effect of change on autonomous systems. Autonomous systems are not designed to be deployed in conditions of perfect stasis, as they are unlikely to encounter such conditions in real-world environments. They are frequently designed for changing environments, like public roads, and may also be designed to change themselves over time, for instance by means of learning capabilities. Not only that, but these changes in deployed systems and in their operating conditions are also likely to take place against a shifting contextual background of societal alteration (e.g. other technologies, 'black swan' events, or simply the day-to-day operation of communities). The effects of such change, on the systems themselves, on the environments within which they are operating, and on the humans with which they engage, must be considered as part of a responsible innovation approach.
The RAILS project brings together a team from UCL and the Universities of York, Leeds and Oxford, from multiple disciplines, with the aim of engaging with the challenges associated with the long-term operation of autonomous systems and the effects of change on these systems. In particular, we will explore how the notion of responsibility is affected by
(i) open-ended dynamic environments - situations that change over time, and
(ii) lifelong-learning systems - i.e. systems that are designed to adapt themselves to their circumstances and 'learn' over time.
The RAILS project will focus on such independent long-term autonomous systems in different applications. These will include (i) autonomous vehicles and (ii) autonomous robot systems such as unmanned aerial vehicles (drones). RAILS will look at social and legal contexts, as well as technical requirements, in order to assess whether and how these systems can be designed, developed, and operated in a way that they are responsible, accountable, and trustworthy.
The overall aim of the RAILS project is to bring together responsible development principles with governance mechanisms and technical understanding to create new understandings of how autonomous systems can adapt to change, how they can be deployed in a responsible and trustworthy way, and how such deployment can be framed by governance to ensure accountability and flexibility.
Organisations
- University of Oxford (Lead Research Organisation)
- Society for Computers and Law (Project Partner)
- Autonomous Drivers Alliance (Project Partner)
- Transport Research Laboratory (United Kingdom) (Project Partner)
- Amazon (United States) (Project Partner)
- Addleshaw Goddard LLP (Project Partner)
- Imperium Drive (Project Partner)
- Oxbotica Ltd (Project Partner)
Publications
Ten Holter C
(2022)
Built in, not bolted on: responsible innovation in UK Centres for Doctoral Training - the new normal?
in The Learning Organization
Drayson G
(2023)
CC-SGG: Corner Case Scenario Generation using Learned Scene Graphs
Winfield AFT
(2022)
Ethical Risk Assessment for Social Robots: Case Studies in Smart Robot Toys
in Part of the Intelligent Systems, Control and Automation: Science and Engineering book series (ISCA,volume 102)
Kochakarn P
(2023)
Explainable Action Prediction through Self-Supervision on Scene Graphs
Kochakarn P
(2023)
Explainable Action Prediction through Self-Supervision on Scene Graphs
Kochakarn P
(2023)
Explainable Action Prediction through Self-Supervision on Scene Graphs
Description | All-Party Parliamentary Group on Data Analytics (APGDA) Steering Committee member - Marina Jirotka |
Geographic Reach | National |
Policy Influence Type | Contribution to a national consultation/review |
Description | CDEI report on Responsible Innovation in Self-Driving Vehicles |
Geographic Reach | National |
Policy Influence Type | Participation in a guidance/advisory committee |
URL | https://www.gov.uk/government/publications/responsible-innovation-in-self-driving-vehicles |
Description | Contributions to the Law Commission review on self-driving vehicles |
Geographic Reach | National |
Policy Influence Type | Contribution to a national consultation/review |
URL | https://www.lawcom.gov.uk/legal-reforms-to-allow-safe-introduction-of-automated-vehicles-announced/ |
Description | Written and oral evidence to the UK Commons Transport Select Committee |
Geographic Reach | National |
Policy Influence Type | Participation in a guidance/advisory committee |
URL | https://driverless-futures.com/2022/12/21/evidence-to-the-transport-select-committee/ |
Description | Application of System Theoretic Process Analysis to understand safety requirements for a domestic robot (EPSCR iCASE) |
Amount | £34,968 (GBP) |
Funding ID | ICASE voucher ref 230058 |
Organisation | Dyson |
Sector | Private |
Country | United Kingdom |
Start | 10/2023 |
End | 09/2027 |
Description | Effective Collaborative Robot Learning for Varying Manipulation Tasks and Environmental Contexts (AIMS CDT) |
Amount | £60,000 (GBP) |
Organisation | Toshiba Research Europe Ltd |
Sector | Private |
Country | United Kingdom |
Start | 10/2023 |
End | 09/2027 |
Title | AI Driving Simulator. |
Description | In the context of RAILS, we developed a physical driving simulator for demonstrating our research on automated commentary driving and to conduct user studies. This driving simulator has been showcased at academic (TAS all-hands-meeting) and outreach events (Goodwood) - see Engagement |
Type Of Technology | Physical Model/Kit |
Year Produced | 2022 |
Impact | This demonstrator attracted interest at academic and outreach events which resulted in new connections and collaboration on a joint research proposal (under review). |
Title | RoAD Recorder |
Description | The software has originally been developed in the TAS RoAD project. IN RAILS the software has been maintained and extended. The RoAD Recorder is about Responsible AV Data - ethical, legal, and societal challenges of using data from autonomous vehicles. In the event of an accident or a 'near miss', a vehicle should log data to its Event Data Recorder (EDR). The purpose of this project is to provide such data to explore its usefulness and inform discussion. This is being done with the use of simulators such as CARLA. |
Type Of Technology | Software |
Year Produced | 2022 |
Open Source License? | Yes |
Impact | The software has been used in two demonstrations using a physical driving simulator at the Goodwood Festival of Speed and the TAS All-hands-meeting. |
URL | https://github.com/cognitive-robots/road-recorder |
Description | AI Festival panel |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Undergraduate students |
Results and Impact | Panel entitled "Closing the gap" as part of AI Festival at Brookes University - topic of panel was women in AI. |
Year(s) Of Engagement Activity | 2023 |
URL | https://www.brookes.ac.uk/about-brookes/events/2023/02/festival-of-ai |
Description | APGDA: AI and ethics roundtable - Prof Marina Jirotka |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Policymakers/politicians |
Results and Impact | Prof. Marina Jirotka was invited as a speaker in a panel debate as part of a virtual evidence sessions which focusses on uncovering key current developments in the UK on creating ethical AI regulation, as well as looking at what is happening in other jurisdictions, and lastly examining any big issues around ethics in the supply chain. |
Year(s) Of Engagement Activity | 2023 |
URL | https://www.policyconnect.org.uk/events/appg-data-analytics-inquiry-evidence-session-2 |
Description | Article on drum machines, automation and AI |
Form Of Engagement Activity | A magazine, newsletter or online publication |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Jack Stilgoe was commissioned for a long read in the online magazine Aeon. Focussing on the history of drum machines, the essay investigated automation in music and what lessons could be learned for artificial intelligence in society. |
Year(s) Of Engagement Activity | 2023 |
URL | https://aeon.co/essays/what-drum-machines-can-teach-us-about-artificial-intelligence |
Description | BBC news article |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Alan Winfield was interviewed for a BBC news article about robots, Elon Musk and driverless cars. |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.bbc.co.uk/news/technology-60154782 |
Description | Blogpost for TAS Hub |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Outcomes of workshop with project industry partners |
Year(s) Of Engagement Activity | 2022 |
URL | https://tas.ac.uk/responsible-ai-lets-make-driving-boring-again/ |
Description | Contributions to the Law Commission review on self-driving vehicles |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Policymakers/politicians |
Results and Impact | The Law Commission's final report on self-driving vehicles referred to the two sets of written evidence submitted by Driverless Futures. |
Year(s) Of Engagement Activity | 2022 |
URL | https://driverless-futures.com/2022/01/27/the-law-commission-and-the-daily-telegraph/ |
Description | Daily Telegraph article |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Article for the Daily Telegraph newspaper in the UK "Moral maze: can driverless cars ever make human choices?: In matters of life and death on the road, Harry de Quetteville asks if machines could be made to think as we do", 27 Jan 2022. Jack Stilgoe spoke to the journalist to help frame and inform the story. Substantial quoting from the interview. |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.telegraph.co.uk/cars/features/can-driverless-cars-ever-truly-make-human-choices/ |
Description | Digital Twins Course in Dept. for Continuing Education, Oxford |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Lars Kunze is the co-director of the digital twin course at the Department for Continuing Education at Oxford University. Concepts and methods for the responsible use of AI and digital twin technology has been presented by Lars Kunze and it was also part of an course exercise. In 2022, the department offered two courses: one in May and another in November. Each course is taught on 6 days with sessions of 3 hours on each day. Next (upcoming) instance of the course: https://www.conted.ox.ac.uk/courses/digital-twins-enhancing-model-based-design-with-ar-vr-and-mr |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.conted.ox.ac.uk/courses/digital-twins-enhancing-model-based-design-with-ar-vr-and-mr |
Description | EC JRC Workshop on Explainable AVs |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Lars Kunze was invited by the European Commission JRC to act as an expert for explainable autonomous vehicles. The involvement included a 2-day workshop (29-30 of March 2022) and a written joint report by all experts. |
Year(s) Of Engagement Activity | 2022 |
URL | https://web.archive.org/web/20220831055511/https://ec.europa.eu/jrc/communities/en/community/humaint... |
Description | Evidence contribution to All-party Parliamentary Group Committee on Data Analytics |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Policymakers/politicians |
Results and Impact | The APPG-DA was sourcing evidence for the review of AI governance and our research group contributed |
Year(s) Of Engagement Activity | 2023 |
Description | Government report on responsible innovation for self-driving vehicles |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Policymakers/politicians |
Results and Impact | Jack Stilgoe was an external expert author (alongside Prof John McDermid from York) of a report with the UK Government Centre for Data Ethics and Innovation on responsible innovation for self-driving vehicles. This report was timed to inform forthcoming Government legislation on safety and testing. JS did a day of media appearances to coincide with the launch of the report. The report and associated media generated substantial debate among stakeholders. |
Year(s) Of Engagement Activity | 2022 |
URL | https://driverless-futures.com/2022/08/30/cdei-report-on-responsible-innovation-in-self-driving-vehi... |
Description | ICRA 2023 Workshop MACTAS |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Lars Kunze is the main organiser of the accepted ICRA 2023 Workshop on Multidisciplinary Approaches to Co-Creating Trustworthy Autonomous Systems in London, May 2023 |
Year(s) Of Engagement Activity | 2023 |
URL | https://mactasworkshop.github.io/2023/ |
Description | Institute for Ethical AI - Oxford AI Meetups |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Invited talk about explainable and trustworthy autonomous systems. The Institute for Ethical AI are leading a series of Oxford Artificial Intelligence Meetups. They have invited us to do a presentation on RI in AI virtually (RI more generally is an option too). |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.meetup.com/Oxford-Artificial-Intelligence-Meetup/ |
Description | Invited talk at Safety Critical Systems Symposium 2023 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | provided an invited talk at Safety Critical Systems Symposium 2023 on frameworks for the safety assurance of autonomous systems. |
Year(s) Of Engagement Activity | 2023 |
URL | https://scsc.uk/sss'23 |
Description | Newsletter article of Marina Jirotka in the University Bulletin and article on University website |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Postgraduate students |
Results and Impact | Marina Jirotka featured in a bio article / interview. |
Year(s) Of Engagement Activity | 2022,2023 |
URL | https://www.ox.ac.uk/news-and-events/oxford-people/Marina-Jirotka |
Description | Opinion piece in the Conversation |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | An opinion piece by Jack Stilgoe in the Conversation argues that current experiments in San Francisco need to be democratised. |
Year(s) Of Engagement Activity | 2022 |
URL | https://driverless-futures.com/2023/03/03/what-weve-learned-from-experiments-in-san-francisco-and-ph... |
Description | Podcast panel discussion: "Am I safe in an autonomous vehicle?" |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Schools |
Results and Impact | Panel discussion on the Living with AI Podcast: Challenges of Living with Artificial Intelligence, discussing the topic of "Am I safe in an autonomous vehicle?" Panel members: Mohammad Naiseh, Associate Professor Helge Wurdemann, Dr Jo-Ann Pattinson, Tom Webster. Podcast is used as a teaching tool for schools and universities. |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.buzzsprout.com/1447474/10861085 |
Description | Presentation of RAILS project at TAS Hub day |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Other audiences |
Results and Impact | Presentation of project goals, team members and research approach to Trustworthy Autonomous Systems day at Carlton House Terrace |
Year(s) Of Engagement Activity | 2022 |
Description | Robot Talk podcast - "Self-Driving Cars: behind the wheel" |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Richard Hawkins was interviewed for an episode of the Robot Talk podcast about Self-Driving Cars. |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.ukras.org.uk/RobotTalk/episode-11-self-driving-cars/ |
Description | Simulator Demonstration at Goodwood Festival of Speed |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | Lars Kunze, together with colleagues form the ORI, demonstrated automated commentary driving (ACD) using a physical driving simulator setup at the Future Lab of the Festival of Speed, Goodwood. Festival visitors were able to experience ACD first hand by using the simulator. |
Year(s) Of Engagement Activity | 2022 |
URL | https://www.goodwood.com/motorsport/festival-of-speed/explore-festival-of-speed/fos-future-lab/ |
Description | TAS AHM |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Presented RAILS project to TAS community. Several new research connections have been established based on the talk and related work. |
Year(s) Of Engagement Activity | 2022 |
URL | https://tas.ac.uk/bigeventscpt/all-hands-meeting-2022/ |
Description | TAS AHM - Simulator Demonstration |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | Lars Kunze, together with colleagues form the ORI, demonstrated automated commentary driving (ACD) using a physical driving simulator setup at TAS all-hands-meeting in London. Attendees (academic/industry) of the TAS AHM were able to experience ACD first hand by using the simulator. |
Year(s) Of Engagement Activity | 2022 |
URL | https://tas.ac.uk/bigeventscpt/all-hands-meeting-2022/ |
Description | TAS Robotics Workshop at King's College London |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | Presented overview of ongoing work in RAILS and the Cognitive Robotics Group. Talks and sandpit sessions to discuss the challenges around human-computer interaction and human-robot interaction. |
Year(s) Of Engagement Activity | 2022 |
URL | https://tas.ac.uk/bigeventscpt/tas-robotics-kcl/ |
Description | Trustworthy Autonomous Systems Industry Day 2023 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Industry/Business |
Results and Impact | Invited talk at the Trustworthy Autonomous Systems Industry Day 2023 (mid March). |
Year(s) Of Engagement Activity | 2023 |
URL | https://www.eventbrite.co.uk/e/trustworthy-autonomous-systems-industry-day-2023-tickets-522567542587 |
Description | Workshop with industry partners |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Daylong workshop to engage with project partners including industry and policymakers to |
Year(s) Of Engagement Activity | 2022 |
Description | Written and oral evidence to the UK Commons Transport Select Committee |
Form Of Engagement Activity | A formal working group, expert panel or dialogue |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Policymakers/politicians |
Results and Impact | The Driverless Futures team gave written evidence to the Transport Select Committee's inquiry on self-driving vehicles. Jack Stilgoe was asked to appear as a witness. |
Year(s) Of Engagement Activity | 2022 |
URL | https://driverless-futures.com/2022/12/21/evidence-to-the-transport-select-committee/ |