UKRI Trustworthy Autonomous Systems Node in Resilience

Lead Research Organisation: University of York
Department Name: Computer Science

Abstract

Imagine a future where autonomous systems are widely available to improve our lives. In this future, autonomous robots unobtrusively maintain the infrastructure of our cities, and support people in living fulfilled independent lives. In this future, autonomous software reliably diagnoses disease at early stages, and dependably manages our road traffic to maximise flow and minimise environmental impact.

Before this vision becomes reality, several major limitations of current autonomous systems need to be addressed. Key among these limitations is their reduced resilience: today's autonomous systems cannot avoid, withstand, recover from, adapt, and evolve to handle the uncertainty, change, faults, failure, adversity, and other disruptions present in such applications.

Recent and forthcoming technological advances will provide autonomous systems with many of the sensors, actuators and other functional building blocks required to achieve the desired resilience levels, but this is not enough. To be resilient and trustworthy in these important applications, future autonomous systems will also need to use these building blocks effectively, so that they achieve complex technical requirements without violating our social, legal, ethical, empathy and cultural (SLEEC) rules and norms. Additionally, they will need to provide us with compelling evidence that the decisions and actions supporting their resilience satisfy both technical and SLEEC-compliance goals.

To address these challenging needs, our project will develop a comprehensive toolbox of mathematically based notations and models, SLEEC-compliant resilience-enhancing methods, and systematic approaches for developing, deploying, optimising, and assuring highly resilient autonomous systems and systems of systems. To this end, we will capture the multidisciplinary nature of the social and technical aspects of the environment in which autonomous systems operate - and of the systems themselves - via mathematical models. For that, we have a team of Computer Scientists, Engineers, Psychologists, Philosophers, Lawyers, and Mathematicians, with an extensive track record of delivering research in all areas of the project. Working with such a mathematical model, autonomous systems will determine which resilience- enhancing actions are feasible, meet technical requirements, and are compliant with the relevant SLEEC rules and norms. Like humans, our autonomous systems will be able to reduce uncertainty, and to predict, detect and respond to change, faults, failures and adversity, proactively and efficiently. Like humans, if needed, our autonomous systems will share knowledge and services with humans and other autonomous agents. Like humans, if needed, our autonomous systems will cooperate with one another and with humans, and will proactively seek assistance from experts.

Our work will deliver a step change in developing resilient autonomous systems and systems of systems. Developers will have notations and guidance to specify the socio-technical norms and rules applicable to the operational context of their autonomous systems, and techniques to design resilient autonomous systems that are trustworthy and compliant with these norms and rules. Additionally, developers will have guidance to build autonomous systems that can tolerate disruption, making the system usable in a larger set of circumstances. Finally, they will have techniques to develop resilient autonomous systems that can share information and services with peer systems and humans, and methods for providing evidence of the resilience of their systems. In such a context, autonomous systems and systems of systems will be highly resilient and trustworthy.

Organisations

 
Description The Trustworthy Autonomous Systems Node in Resilience was set up with the ambitious aim to deliver mathematically based notations, models, methods and systematic approaches for developing autonomous systems capable of handling the disruptions and uncertainty encountered in real-world applications. The Node has focused on sociotechnical resilience, which involves ensuring that autonomous systems achieve their complex technical goals resiliently and without violating our social, legal, ethical, empathy and cultural (SLEEC) rules and norms. The Node has already delivered significant research aligned with its aim, including the key results summarised below.

- We devised a process for deriving SLEEC rules for autonomous systems from a set of high-level social, legal, ethical, empathetic and cultural norms, thereby bridging the gap between normative principles and operational practice.

- We defined a controlled, domain-specific language for expressing SLEEC rules, and we developed methods and tools for translating these rules into a mathematically based notation, for checking their consistency (to identify conflicting and redundant rules), and for verifying whether a resilient autonomous system complies with a given set of rules.

- We devised methods for determining the hazards that may lead to disruptions and uncertainties in the operation of an autonomous system, and the actions that can enable the system to mitigate these hazards resiliently.

- We developed methods for predicting disruptions that may affect an autonomous system before they happen, to enable the system to adjust its behaviour in order to avoid them, and for detecting sudden changes and failures in order to respond to them.

- We delivered techniques enabling autonomous systems to achieve optimal trade-offs between conflicting objectives they need to optimise (e.g., utility, cost, risk) as they adapt in order to achieve their goals in the presence of uncertainty and disruptions.
Exploitation Route We co-developed the project research with partners who represent all types of autonomous system stakeholders, including solution developers, end-user organisations, robot manufacturers, research organisations, operators, policy makers, and standards organisations. These committed partners are spearheading the adoption of the project results.

Additionally, we organised the 2022 UKRI Trustworthy Autonomous Systems Health and Social Care Workshop (https://resilience.tas.ac.uk/events/HSCW), where our research was shared with a wide range of resilient autonomous systems stakeholders, including NHS Trust hospitals, social care organisations, and special educational needs school teachers.

We have co-organised leading conferences (including the 19th International Conference on Software Engineering and Formal Methods, the 40th International Conference on Computer Safety, Reliability, and Security, and the 20th IEEE International Conference on Software Architecture) where our research contributed to defining the conference scope, theme and topics, and therefore influenced international research agendas and roadmaps.

We have contributed project results to international standards under development, including the IEEE Guide on Verification of Autonomous Systems, and ISO/IEC/IEEE 15939 Systems and Software Engineering Dependability.
Sectors Aerospace, Defence and Marine,Digital/Communication/Information Technologies (including Software),Healthcare,Transport,Other

URL https://resilience.tas.ac.uk/
 
Guideline Title CONSORT-AI Guidelines
Description CONSORT-AI Guidelines: took part in a study that generated a new reporting guideline for clinical trials evaluating interventions with an AI component
Geographic Reach Multiple continents/international 
Policy Influence Type Citation in clinical guidelines
Impact improved transparency and completeness of reporting of clinical trials that aim to evaluate interventions involving artificial intelligence
URL https://www.bmj.com/content/370/bmj.m3164
 
Description Contributions to IEEE Guide on Verification of Autonomous Systems
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to new or Improved professional practice
URL https://standards.ieee.org/ieee/2817/7644/
 
Description Executive Summary
Geographic Reach Local/Municipal/Regional 
Policy Influence Type Influenced training of practitioners or researchers
 
Description First Response Practice
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description IEEE Roadmap on Verification of Autonomous Systems
Geographic Reach Multiple continents/international 
Policy Influence Type Membership of a guideline committee
 
Description NATO Meaningful Human Control in AI Enabled Systems
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description UKRI Roundtable on Public Engagement with Automation and Autonomous Systems
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
 
Description Working Group on Health software and health IT systems safety, effectiveness and security, with responsibility for producing guidance for the use of assurance cases for safety and security.
Geographic Reach Multiple continents/international 
Policy Influence Type Membership of a guideline committee
 
Description provision of autonomous system assurance training to members of The Technical Cooperation Program
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Ambient Assisted Living for Long-term Monitoring and Interaction
Amount £249,992 (GBP)
Organisation Lloyd's Register Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 06/2021 
End 11/2022
 
Description Assuring Responsibility for Trustworthy Autonomous Systems
Amount £703,615 (GBP)
Funding ID EP/W011239/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2022 
End 06/2024
 
Description Catalysis for Circular economy and sustainable manufacturing theme
Amount £140,364 (GBP)
Organisation Research Complex at Harwell 
Department UK Catalysis Hub
Sector Public
Country United Kingdom
Start 05/2021 
End 10/2022
 
Description Confident safety integration for Cobots (CSI: Cobot) - Part 2
Amount £169,897 (GBP)
Organisation Lloyd's Register Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2021 
End 06/2022
 
Description Confident safety integration for Cobots (CSI: Cobot) - Part 2
Amount £169,897 (GBP)
Organisation Lloyd's Register Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 07/2021 
End 06/2022
 
Description Digital twinning for safety assurance and knowledge exchange in industrial robotics - Phases 1 & 2
Amount £61,786 (GBP)
Organisation University of Sheffield 
Sector Academic/University
Country United Kingdom
Start 11/2022 
End 06/2023
 
Description Digital twins for high-value engineering applications - DTHIVE
Amount £898,048 (GBP)
Organisation Alan Turing Institute 
Sector Academic/University
Country United Kingdom
Start 10/2021 
End 03/2022
 
Description Evaluating the safety and patient impact of an Artificial Intelligence Command Centre in the UK National Health Service
Amount £456,613 (GBP)
Funding ID NIHR129483 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 03/2021 
End 08/2022
 
Description Global Research and Innovation Programme
Amount £30,500 (GBP)
Funding ID EP/V026747/1 additional GRIP funding 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 10/2021 
End 03/2022
 
Description Imagining Robotic Care
Amount £148,246 (GBP)
Organisation University of Southampton 
Sector Academic/University
Country United Kingdom
Start 04/2021 
End 03/2022
 
Description Protecting environments with unmanned aerial vehicle swarms
Amount £7,800,000 (GBP)
Organisation Windracers 
Sector Charity/Non Profit
Country United Kingdom
Start 07/2022 
End 07/2024
 
Description Reimagining Trustworthy Autonomous Systems with Disabled Young People
Amount £150,000 (GBP)
Organisation University of Southampton 
Sector Academic/University
Country United Kingdom
Start 04/2022 
End 03/2023
 
Description University of York - EPSRC capital award for core equipment 2022/23
Amount £875,000 (GBP)
Funding ID EP/X034798/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 11/2022 
End 11/2022
 
Title Human-Human Dressing Assistive Trials 
Description In December, 2021, we carried out a Human-Human assistive dressing trials to collect data that we need to train our robotic dressing assistive system. The study involved experienced occupational therapists (OTs) from North Yorkshire County Council and 7 healthy volunteers. We gathered data from assistive daily living tasks that reflects the process taken in Care Homes, in particular dressing scenarios. Study Objectives: • To evaluate the interaction, attention, and information gathered from information provided in human interaction to carry out dressing tasks. • To evaluate and report on a safety study with older adult users interacting with a human for an ADL Task. • To identify the hazards and implications that need to be considered for a more holistic approach for designing of safe physical Human-Robot Interaction (HRI) between the user and the robots in different scenarios. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact This study aims at understanding occupational therapists' dressing strategies of people with different types of physical impairments, forces and trajectories involved in the strategies and interaction modalities such as speech that help maintain safe interaction. This will help develop control algorithms and provide methods for establishing safe operations in robot-assisted dressing. 
 
Description AGILE Project: Chatty Car 
Organisation University of Nottingham
Country United Kingdom 
Sector Academic/University 
PI Contribution Support project proposal and funding for postdoc position at 0.2FTE. Collaborate with partners to support the design of a future natural language interface
Collaborator Contribution Use existing datasets supported by relevant theory and apply multi-disciplinary methods to the design of a natural language interface, encompassing both lexical and non-lexical mediators of trust, and curating findings in a preliminary design framework.
Impact Collaborative partnerships with other University partners
Start Year 2021
 
Description Agile Project: Trustworthy Human-Swarm Partnerships in Extreme Environments 
Organisation Defence Science & Technology Laboratory (DSTL)
Country United Kingdom 
Sector Public 
PI Contribution Post Doc 0.2FTE for 8 months on the project Generate operator centric requirements for the design of a trust-worthy human-swarm interaction. This was achieved by conducting interviews with current drone operators. The outputs of this feed into interface design and user testing.
Collaborator Contribution Provide network contacts for participant recruitment for the interview study. Conduct workshops and develop use cases and a scenario for user testing study.
Impact One journal publication has been submitted and is under peer review. Two conference paper have been accepted but DOIs are not yet available. Collaboration partnerships have been formed with other departments at the University of Southampton and with industry.
Start Year 2021
 
Description Agile Project: Trustworthy Human-Swarm Partnerships in Extreme Environments 
Organisation Thales Group
Country France 
Sector Private 
PI Contribution Post Doc 0.2FTE for 8 months on the project Generate operator centric requirements for the design of a trust-worthy human-swarm interaction. This was achieved by conducting interviews with current drone operators. The outputs of this feed into interface design and user testing.
Collaborator Contribution Provide network contacts for participant recruitment for the interview study. Conduct workshops and develop use cases and a scenario for user testing study.
Impact One journal publication has been submitted and is under peer review. Two conference paper have been accepted but DOIs are not yet available. Collaboration partnerships have been formed with other departments at the University of Southampton and with industry.
Start Year 2021
 
Description Agile Project: Trustworthy Human-Swarm Partnerships in Extreme Environments 
Organisation University of Nottingham
Country United Kingdom 
Sector Academic/University 
PI Contribution Post Doc 0.2FTE for 8 months on the project Generate operator centric requirements for the design of a trust-worthy human-swarm interaction. This was achieved by conducting interviews with current drone operators. The outputs of this feed into interface design and user testing.
Collaborator Contribution Provide network contacts for participant recruitment for the interview study. Conduct workshops and develop use cases and a scenario for user testing study.
Impact One journal publication has been submitted and is under peer review. Two conference paper have been accepted but DOIs are not yet available. Collaboration partnerships have been formed with other departments at the University of Southampton and with industry.
Start Year 2021
 
Description Boeing Research & TEchnology 
Organisation Boeing
Country United States 
Sector Private 
PI Contribution Basic research and expertise on the verification of autonomous agents with deep-learning components.
Collaborator Contribution Use case and expertise on autonomous agents from the avionics domain (airport taxiing).
Impact Research paper under review.
Start Year 2022
 
Description Circus foundations and Isabelle/UTP 
Organisation Galois, Inc.
Country United States 
Sector Private 
PI Contribution This is a collaboration that has just started. We are going to provide expertise on the techniques.
Collaborator Contribution They are taking a lead to mechanise the techniques and apply them on Rigorous Digital Engineering.
Impact Development of theories and techniques, and their application to industrial case studies.
Start Year 2022
 
Description Collaboration on RoboWorld language: syntax, semantics, examples, tools 
Organisation UFPE
Country Brazil 
Sector Academic/University 
PI Contribution Expertise on robotics, the RoboChart language, and CyPhyCircus semantics
Collaborator Contribution Expertise on controlled natural languages, partnership in the design and implementation of RoboWorld
Impact Development of the RoboWorld language and a prototype tool for it, journal paper under review
Start Year 2019
 
Description Grant Collarobation 
Organisation University of Sheffield
Country United Kingdom 
Sector Academic/University 
PI Contribution Consulting-Mentoring role in Grant & Research Development
Collaborator Contribution Grant Submission
Impact Grant Submission Multi-disciplinary work: sociology, psychology, software engineering
Start Year 2022
 
Description Hybrid Model Checking - CORA 
Organisation Technical University of Munich
Country Germany 
Sector Academic/University 
PI Contribution We bring expertise in robotics, the design and semantics of the RoboWorld language, and the CyPhyCircus hybrid process algebra.
Collaborator Contribution They bring expertise in hybrid model checking, and development on the CORA hybrid model checker to support our models.
Impact Models in CORA based on the semantics of RoboWorld, testing and enhancement of the CORA hybrid model checker.
Start Year 2022
 
Description NASA Ames 
Organisation National Aeronautics and Space Administration (NASA)
Department NASA Ames Exploration Center
Country United States 
Sector Public 
PI Contribution Basic research on quantifying the uncertainty of deep-learning perception components for autonomous systems.
Collaborator Contribution Use cases, data sets, machine learning models, deep-learning expertise.
Impact Basic research described in two papers currently under review.
Start Year 2022
 
Description Pump Priming - SAVE: Stuational Awareness (SA) and trust during shift between autonomy levels in automated vehicles 
Organisation University College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Support project proposal. Provide Human Factors support within the project to facilitate the user study and the data collection
Collaborator Contribution Develop and implement driving scenarios involving automated vehicles. Identify the role of feedback (type) & guidance of the driving assistive technology in increasing SA & trust.
Impact Collaborative partnerships with other universities.
Start Year 2021
 
Description Rowan University 
Organisation Rowan University
Country United States 
Sector Academic/University 
PI Contribution Expertise in verification of neural networks, basic research.
Collaborator Contribution Expertise in Bayesian techniques for deep learning, basic research.
Impact Ongoing work on joint grant proposals.
Start Year 2022
 
Description Royal Academy of Engineering Industry Fellowship 
Organisation Thales Group
Department Thales UK Limited
Country United Kingdom 
Sector Private 
PI Contribution Co-supervision of student and industry fellows.
Collaborator Contribution We are developing a technique to model and verify mobile and autonomous robots with humans in the loop.
Impact Multi-disciplinary: Engineering and Psychology.
Start Year 2021
 
Description TAS AGILE: Digital Twins for Human-Assistive Robot Teams 
Organisation Heriot-Watt University
Country United Kingdom 
Sector Academic/University 
PI Contribution Contribute to the Validation and Verification of a Digital Twin for the Robot-Assisted Dressing (RAD) application.
Collaborator Contribution Create the Digitial Twin and demonstrate its use.
Impact Collaborative partnerships with other Universities
Start Year 2022
 
Description TAS AGILE: Digital Twins for Human-Assistive Robot Teams 
Organisation University of Nottingham
Country United Kingdom 
Sector Academic/University 
PI Contribution Contribute to the Validation and Verification of a Digital Twin for the Robot-Assisted Dressing (RAD) application.
Collaborator Contribution Create the Digitial Twin and demonstrate its use.
Impact Collaborative partnerships with other Universities
Start Year 2022
 
Description TAS Integrator: Preserving Marine Life in a Shipping World: AI to the Rescue (PREVAIL) 
Organisation Thales Group
Country France 
Sector Private 
PI Contribution Co-design an AI advisory tool to ensure trustworthy decisions.
Collaborator Contribution Collect and process data using Deep Neural Network.
Impact Maximise the use of resources available in TAS community.
Start Year 2022
 
Description TAS Integrator: Preserving Marine Life in a Shipping World: AI to the Rescue (PREVAIL) 
Organisation University of Manchester
Country United Kingdom 
Sector Academic/University 
PI Contribution Co-design an AI advisory tool to ensure trustworthy decisions.
Collaborator Contribution Collect and process data using Deep Neural Network.
Impact Maximise the use of resources available in TAS community.
Start Year 2022
 
Description TAS Integrator: Verifiably Human-Centric Robot Assisted Dressing 
Organisation Durham University
Country United Kingdom 
Sector Academic/University 
PI Contribution Create an abstract model of the application for verification using formal methods.
Collaborator Contribution Design and conduct experiments.
Impact Collaborative partnerships with other Universities
Start Year 2022
 
Description TAS Integrator: Verifiably Human-Centric Robot Assisted Dressing 
Organisation King's College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Create an abstract model of the application for verification using formal methods.
Collaborator Contribution Design and conduct experiments.
Impact Collaborative partnerships with other Universities
Start Year 2022
 
Description TAS Integrator: Verifiably Human-Centric Robot Assisted Dressing 
Organisation University of Lincoln
Country United Kingdom 
Sector Academic/University 
PI Contribution Create an abstract model of the application for verification using formal methods.
Collaborator Contribution Design and conduct experiments.
Impact Collaborative partnerships with other Universities
Start Year 2022
 
Description TAS Integrator: Verifiably Human-Centric Robot Assisted Dressing 
Organisation University of Manchester
Country United Kingdom 
Sector Academic/University 
PI Contribution Create an abstract model of the application for verification using formal methods.
Collaborator Contribution Design and conduct experiments.
Impact Collaborative partnerships with other Universities
Start Year 2022
 
Description TAS Integrator: Verifiably Human-Centric Robot Assisted Dressing 
Organisation University of Sheffield
Country United Kingdom 
Sector Academic/University 
PI Contribution Create an abstract model of the application for verification using formal methods.
Collaborator Contribution Design and conduct experiments.
Impact Collaborative partnerships with other Universities
Start Year 2022
 
Description TAS Integrator: Verifiably Human-Centric Robot Assisted Dressing 
Organisation University of Southampton
Country United Kingdom 
Sector Academic/University 
PI Contribution Create an abstract model of the application for verification using formal methods.
Collaborator Contribution Design and conduct experiments.
Impact Collaborative partnerships with other Universities
Start Year 2022
 
Description York and Scarborough Teaching Hospitals NHS Foundation 
Organisation York Teaching Hospital NHS Foundation Trust
Country United Kingdom 
Sector Public 
PI Contribution Autonomous systems, robotics and AI expertise.
Collaborator Contribution Use case, data, emergency medicine expertise, consultant time.
Impact Research papers under review, partnership on grant proposals under review.
Start Year 2022
 
Title LINKING SOURCE CODE WITH COMPLIANCE REQUIREMENTS 
Description Concepts for linking source code with compliance requirements are presented. One example comprises analyzing a set of compliance requirements to identify one or more compliance topics. The example further comprises determining keywords for the identified one or more compliance topics. An item of source code is then analyzed to identify occurrences of the keywords in the source code. Mapping information representing a relationship between the item of source code and the compliance requirements is then generated based on the identified occurrence of the keywords. 
IP Reference US2020117573 
Protection Patent granted
Year Protection Granted 2020
Licensed No
Impact N/A (yet)
 
Title RELEVANCE OF A SOURCE CODE CHANGE TO COMPLIANCE REQUIREMENTS 
Description Concepts for identifying relevance of a source code change to compliance requirements are presented. One example comprises obtaining mapping information linking an item of source code with a set of compliance requirements, the mapping information representing a relationship between the item of source and the set of compliance requirements. A changed element of an item of source code is identified. The mapping information is analyzed based on the changed element to determine if the changed element relates to a compliance requirement. If it is determined that the changed element relates to a compliance requirement, an indication of th compliance requirement is generated. 
IP Reference US2020117427 
Protection Patent granted
Year Protection Granted 2020
Licensed No
Impact N/A
 
Title SLEECVAL: Specification and Validation of Normative Rules for Autonomous Agents 
Description Growing range of applications use autonomous agents such as AI and robotic systems to perform tasks deemed dangerous, tedious or costly for humans. To truly succeed with these tasks, the autonomous agents must perform them without violating the social, legal, ethical, empathetic, and cultural (SLEEC) norms of their users and operators. We introduce SLEECVAL, a tool for specification and validation of rules that reflect these SLEEC norms. Our tool supports the specification of SLEEC rules in a DSL. we co-defined with the help of ethicists, lawyers and stakeholders from health and social care, and uses the CSP refinement checker FDR4 to identify redundant and conflicting rules in a SLEEC specification. We illustrate the use of SLEECVAL for two case studies: an assistive dressing robot, and a firefighting drone. 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2023 
Impact SLEECVAL enables specification and validation of non-functional rules with the focus on SLEEC (social, legal, ethical, empathetic, and cultural) principles for autonomous agents. To best of our knowledge, our language and tool are novel in their support to formalise and validate normative rules that address SLEEC concerns. Our vision is to provide an automated framework to specify, validate, and verify that agents follow rules, reporting redundancy and conflicts. 
URL https://sleec.github.io
 
Title xinwei2124/fPMC_ICSE: Fast Parametric Model Checking (fPMC) ICSE 2021 
Description No description provided. 
Type Of Technology Software 
Year Produced 2021 
Impact This software eased adoption of our new algorithm that can be used to signficiantly extend applicability of parametric model checking. 
URL https://zenodo.org/record/4522170
 
Description 18th International Summer School on Trustworthy Software 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Ana Cavalcanti and Jim Woodcock led courses on Software Engineering for Robotics and Isabelle/HOL as part of the 18th International Summer School on Trustworthy Software organised by the School of Software Engineering of East China Normal University. RoboStar colleagues James Baxter , Gustavo Carvalho , Alvaro Miyazawa , Pedro Ribeiro and Matt Windsor delivered practicals and seminars.
Year(s) Of Engagement Activity 2022
 
Description A talk on: "UKRI Uncertainty-Aware Machine Learning Methods for Trustworthy Autonomous Systems" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A talk at the "Trusting Machines? Cross-sector Lessons from Healthcare and Security" event The event was created as part of the collaboration between RUSI, the TAS Hub and National Gallery X.

The topic of the toc is summarised briefly below:
Sensors provide enormous amounts of information beyond the capacity that a human could process. Being part of autonomous systems, the data are coming in real time and need to be processed quickly and fed to the control and decision-making levels.
This talk will discuss recently developed machine learning methods able to deal with data challenges such as volume, velocity, veracity and variety. The recent trends in machine learning and autonomy are towards development of trustworthy solutions, able to work under different conditions - due to the external environmental changes and the dynamics of the autonomous systems. Autonomous systems need to be safe and reliable, and include both information from hard and soft sensors - cameras, LiDARs, radars, wireless sensor networks and data from the Internet or other sources.
Generic principles valid for at least four domains will be discussed - for surveillance, health, manufacturing and transport systems.
This talk aims to stimulate discussions from multi-disciplinary areas and consider questions like these:
• How could be define different levels of trustworthiness and resilience, respectively?
• How could we characterise trustworthiness? What criteria could we have? How could we quantify the impact of uncertainties and provide resilience?
• What are going to be the next generations of methods for uncertainty-aware autonomous systems?
• How could we fuse reliably the data from multiple heterogeneous sensors, in order to provide resilience?
• How do we link the technological aspects with ethics, societal and human-centred factors?
Year(s) Of Engagement Activity 2021
URL https://rusi.cplus.live/event/trusting-machines-21
 
Description Are you Talking to your Autonomous Car? (Maybe you should!) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Ana Cavalcanti featured as a speaker in the latest Living with AI Podcast of the TAS Hub.
Year(s) Of Engagement Activity 2022
URL https://www.buzzsprout.com/1447474/11041372
 
Description Autonomy and Safety Seminar Presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact A presentation in which I presented a language for the encoding of rules which capture Social, Legal, Ethical, Empathetic and Cultural Norms for use in robotic autonomous systems (RAS).
Year(s) Of Engagement Activity 2021
 
Description Bristol BAME STEM seminar on Software Engineering for Robotics 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Webinar on my career journey and research interests to a wide range of scientists and Science Administrators for wider dissemination.
Year(s) Of Engagement Activity 2023
 
Description Collaboration and engagement with University of KwaZulu-Natal on the Regulation of AI in Africa 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Study participants or study members
Results and Impact Collaboration and engagement on the regulation, policy, and governance of Artificial intelligence in South Africa, and more widely in Africa. (involves 12 African jurisdictions)
Included topics of
whether South Africa requires an AI Act,
the human rights implications of novel technologies and algorithmic systems (agents),
the mapping of the AI landscape in Africa, and
cross-border data flows in Africa (and data protection laws across Africa)
This is an ongoing involvement.
Year(s) Of Engagement Activity 2021,2022
 
Description Designing Identity-Adaptive Robots: How group dynamics facilitate cooperation between human and non-human agents in emergencie 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Talk on the impact of intelligent agents on social relationships and behaviours. Audience reported increased interest in the potentials of group dynamics in socio-technical systems.
Year(s) Of Engagement Activity 2023
 
Description Discussion on the assurance of machine learning for autonomous driving 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact A video presentation created for JLR on how assurance may be used for machine learnt components in autonomous driving. I also participated in a Q&A in which the participants asked questions arising from the presentation.
Year(s) Of Engagement Activity 2021
 
Description ECNU summer school 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact In August 2022, we delivered a series of lectures via video link to graduate students at East China Normal University (ECNU) in Shanghai, China as part of a summer school on Trustworthy Systems.
Year(s) Of Engagement Activity 2022
URL https://seisummerschool.github.io/2022/
 
Description Executive Summary 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Practical recommendations for emergency response coordination through the use of new technologies
Year(s) Of Engagement Activity 2023
 
Description Expert advisor on the WHO/ITU Regulation of AI in Global Health report 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact I participated as an advisor and co-author on the WHO/ITU 'Regulation of AI in Health' report in the 'Privacy and Data Protection' and the 'Collaboration and Engagement' working groups. The report is to be published in 2022.
This report is aimed at regulators worldwide.
Year(s) Of Engagement Activity 2021,2022
 
Description Introducing the Project 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Presentation on the REASON project (Resilience Node), and specifically the challenges and risks associated with human-drone cooperation in emergency/humanitarian operations, as well as proposed approaches to mitigate these risks. Approximately 20 individuals, including academics, software designers, & policymakers attended this meeting.
Year(s) Of Engagement Activity 2021
 
Description Invited Trustworthy Autonomous Systems Node in Verifiability talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Gave UKRI TAS Node in Verifiability seminar on 22 April 2021, attended by members of the UKRI Trustworthy Autonomous Systems network. The talk led to discussions about novel techniques for ensuring collaboration between autonomous systems as well as between autonomous systems and humans.
Year(s) Of Engagement Activity 2021
URL https://verifiability.org/events/
 
Description Invited Trustworthy Autonomous Systems Node in Verifiability talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Gave UKRI TAS Node in Verifiability seminar on 22 April 2021, attended by members of the UKRI Trustworthy Autonomous Systems network. The talk led to discussions about novel techniques for the modelling and verification of autonomous systems, and to plans for extending these techniques to address further challenges of safety-critical autonomous systems.
Year(s) Of Engagement Activity 2021
URL https://verifiability.org/events/
 
Description Invited course at the School of Software, Northwest Polytechnical University, Xi'an, China 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact This is a school organised by a consortium of Chinese universities that provides younger researchers with the background on the state of the art in the area of software engineering. The course sparked questions and discussion afterwards.
Year(s) Of Engagement Activity 2021
 
Description Invited course at the School of the International Colloquium on Theoretical Aspects of Computing 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact This is a school associated with an international conference that provides younger researchers with the background on the state of the art in the area of theoretical aspects of computing. The course sparked questions and discussion afterwards, and led to a publication.
Year(s) Of Engagement Activity 2021
URL https://ictac2021.github.io/school.html
 
Description Invited lecture at the International School on Formal Methods and Theoretical Informatics (ETMF) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Undergraduate students
Results and Impact This is a school associated with an international conference that provides younger students with the background to follow the paper presentations in the conference. The course sparked questions and discussion afterwards and during the main event.
Year(s) Of Engagement Activity 2021
URL https://sites.google.com/computacao.ufcg.edu.br/sbmf2021/program
 
Description Invited talk - Specifying for Trustworthiness workshop at TAS All Hands meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Gave a talk to TAS Hub Researchers on specifying (and reasoning about) Cooperation between Autonomous Systems and Humans. The talk led to discussion about integrating the difference perspective on cooperation, and the development of a follow up paper.
Year(s) Of Engagement Activity 2021
 
Description Invited talk at BCS on How can we define operational requirements in robotics? 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact On the 9th of May, Ana Cavalcanti lead a seminar on "How can we define operational requirements in robotics?" together with colleagues James Baxter and Gustavo Carvalho . The event was organised by BCS London Central Branch and BCS North London branch, for IT professionals.
Year(s) Of Engagement Activity 2022
 
Description Invited talk at the IROS2021 Workshop on Standardised Software Frameworks for Robotics in Nuclear 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This was a workshop organised by colleagues in industry. It gave us the opportunity to present our results and spark discussion with practitioners who are themselves interested in technological developments in the area.
Year(s) Of Engagement Activity 2021
URL https://ukaeaevents.com/2021-ieee-rsj/
 
Description Invited talk at the York & North Yorkshire LEP Skills Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Ana Cavalcanti delivered an invited talk at the York & North Yorkshire LEP Skills Conference on "Software Engineering for Robotics in Industry 4.0". The talk can be watched on YouTube
Year(s) Of Engagement Activity 2022
URL https://www.ynylep.com/news/festival-of-engagement/event/id/2220
 
Description Keynote at Context-aware, Autonomous and Smart Architecture Workshop co-located with the 16th European Conference on Software Architecture 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Gave a keynote entitled "The Road to Collaboration is Paved with Goals' that explores the role of goals in supporting autonomy and collaboration.
Year(s) Of Engagement Activity 2022
URL https://casaecsa.github.io/casa2022/
 
Description Keynote at International Conference on Formal Aspects of Component Software 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The participants at the 2021 International Conference on Formal Aspects of Component Software attended the keynote presentation, which let to discussions about the challenges of modelling, analysing and verifying the software of resilient trustworthy autonomous systems, and about approaches for addressing these challenges.
Year(s) Of Engagement Activity 2021
URL http://facs2021.inria.fr/keynote-speakers/
 
Description LSP Workshop with Stakeholders 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact 5 government and industry stakeholders participated the Lego Serious Play workshop which encouraged discussions on innovation & resilience. Participants created their own models to define resilience and created a set of SLEEC norms based on their experience. The workshop is expected to be run for a second time later this year with stakeholders from different sectors.
Year(s) Of Engagement Activity 2022
 
Description Living With AI Podcast: Challenges of Living with Artificial Intelligence 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Podcast produced by the UKRI Trustworthy Autonomous Systems Programme on Assistive AI, and the problems of robots in a care role.
Year(s) Of Engagement Activity 2021
URL https://lwaip.buzzsprout.com/1447474/8282015-who-cares-about-robots-in-the-home
 
Description Not like the Others: Designing Identity-Aware Autonomous Systems for Emergencies 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Presentation of research on the integration of group psychological principles in autonomous systems and its consequencies for new challenges and innovation in psychology.
Year(s) Of Engagement Activity 2022
URL https://www.bps.org.uk/member-networks/social-psychology-section
 
Description Organisation of RoboStar and YorRobots Industry Exhibition 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact On 11-12 October YorRobots and RoboStar hosted an Industry Exhibition. There were talks and demonstrations by RoboStar colleagues and from YorRobots, The Institute for Safe Autonomy, Adelard, AgroIntelli, BAE, Bristol Robotics Lab, ClearSy, Chemspeed Technologies, Connected Places, LabMan, LDRA, DRis-Q, Dyson, RobotCenter, ShadowRobot, Thales and VSI.
Year(s) Of Engagement Activity 2022
URL https://robostar.cs.york.ac.uk/events/yrie2022/
 
Description Panel on Considerations for ensuring the safe utilisation of AI in healthcare (with MHRA, NHS X and NHS Digital) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Panel on Considerations for ensuring the safe utilisation of AI in healthcare (with MHRA, NHS X and NHS Digital)
Year(s) Of Engagement Activity 2021
 
Description Poster presented in UKRI TAS Hub ECR Networking Event 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact To meet other researchers and seek collaboration opportunities across the TAS Hub
Year(s) Of Engagement Activity 2022
 
Description Presentation at RoboStar and YorRobots Industry Exhibition 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact On 11th October 2022, I gave a talk and presented a poster on the semantics of the RoboWorld language and the use of hybrid model checking to check the semantics.
Year(s) Of Engagement Activity 2022
URL https://robostar.cs.york.ac.uk/events/yrie2022/
 
Description Presentation at TAS - All Hands meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Presentation of the ongoing work related to assistive dressing robots, progress made so far on building the system in terms of both hardware and software involved. Then a discussion with other members on topics related to TAS subjects.
Year(s) Of Engagement Activity 2021
 
Description Presentation at TAS All Hands 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Presentation of work completed on the project at the TAS All hands project to present approach to other project members
Year(s) Of Engagement Activity 2021
 
Description Presentation at TAS Hub ECR Event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact I gave a presentation about the assistive dressing robots. Discussions made about different robotics applications in real life.
Year(s) Of Engagement Activity 2021
 
Description Presentation at TAS-RUSI workshop : 'Trusting Machines? Cross-sector Lessons from Healthcare and Security' 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact This was a presentation entitled "Assurance of machine Learning for use in Autonomous Systems" and considered the difficulties for assuring autonomous systems in open environments and a framework for the generation of an assurance case building on existing practice and desiderata associated with machine learnt components.
Year(s) Of Engagement Activity 2021
URL https://rusi.org/events/conferences/trusting-machines-cross-sector-lessons-healthcare-and-security
 
Description Presentation on TAS All Hands Meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Presented recent work in TAS All Hands Meeting and planned to seek feedback and potential collaborations. However, due to the Covid restrictions, the event was hold online, and thus the feedbacks are limited.
Year(s) Of Engagement Activity 2021
 
Description Presentation to the TAS all hands meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Presentation of work to date including a process for the elicitation of SLEEC rules derived from high level principles and stakeholder knowledge. Presentation of a language to encode these rules for use in autonomous systems.
Year(s) Of Engagement Activity 2021
 
Description Presentations at RoboStar and YorRobots Industry Exhibition 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact On 11-12 October, I gave three talks on the work developed in the RoboCalc and RoboTest projects at an Industry Exhibition.
Year(s) Of Engagement Activity 2022
URL https://robostar.cs.york.ac.uk/events/yrie2022/
 
Description Research Talk at the TAS all hands meeting 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Presentation of ideas on how to use game theory to develop resilient systems, based on past research on game-theoretic applications to software development,
Year(s) Of Engagement Activity 2021
URL https://vimeo.com/showcase/8775391/video/590905911
 
Description Safe AI in Healthcare 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Talk to a multidisciplinary audience at Imperial College London (Human Factors and Design Safety Group)
Year(s) Of Engagement Activity 2021
 
Description Safe Use of Machine Learning in Autonomous Applications 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Talk to KI ABSICHERUNG (largest automotive consortium in Germany that investigates safe AI for automated driving, led by key industry companies including VW, BMW, Audi and BOSCH )
Year(s) Of Engagement Activity 2021
 
Description The socio-technical dimensions of AI: Towards a moral economy of data 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Talk to a multi-disciplinary audience on the intersection of AI with social, ethical, and cultural norms - and the need to reframe the debate away from ethical decision-making to socio-ethical-cultural responsiveness.
Year(s) Of Engagement Activity 2021
 
Description Towards Trustworthy Machine Learning Methods 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact This is an invited plenary talk as part of the 12th International Conference on Electronics, Communications and Networks (CECNet 2022), November 4th-7th, 2022, Online Conference, Beijing. The talk stimulated interesting discussions and possibly new research collaborations.
Year(s) Of Engagement Activity 2022
URL http://cecnetconf.org/Program
 
Description Towards Trustworthy Machine Learning Methods 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact This is an invited talk as part of the Terrorism Risk Assessment, Modelling and Mitigation Seminar Series (TRAMMSS) at Cranfield University, 11 November 2022
Year(s) Of Engagement Activity 2022
URL https://www.cranfield.ac.uk/events/events-2022/towards-trustworthy-machine-learning-methods
 
Description UK-RAS Network RoboTalk Podcast 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Podcast organised by the UK-RAS network to describe research and projects of my group.
Year(s) Of Engagement Activity 2022
URL https://www.ukras.org.uk/news/robot-talk-episode-thirty-out-now
 
Description Understanding the societal consequences of human-robot collaborations for resilience 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Workshop in which the audience was asked to report what changes they would have made if they were in charge of the local resilience planning and why. They answers informed ongoing research on a Decisions from Evidence Synthesis for emergency resilience.
Year(s) Of Engagement Activity 2023
 
Description Women in Machine Learning and Data Science 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact This was a talk entitled Machine Learning and Software Engineering: Separated by a Common Language, , which led to interesting discussions between researchers, practitioners and academics on the topic of machine learning and responsible software engineering.
Year(s) Of Engagement Activity 2021
 
Description Youtube channel for Resilience Node Seminar 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact A seminar series hosted by the team, which have hundreds of viewers internationally.
Year(s) Of Engagement Activity 2021,2022
URL https://www.youtube.com/channel/UCkZi5sOtN4bbcTWcll_0vhg
 
Description invited talk in the University of Bristol's Trustworthy Systems Lab Seminar series 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Around 20 members of the research lab attended the invited talk, which was followed by discussions of opportunities for future research, including in collaboration.
Year(s) Of Engagement Activity 2021
URL https://www.bristol.ac.uk/engineering/research/trustworthy-systems-laboratory/seminar-series/
 
Description women+@DCS seminar on Software Engineering for Robotics 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact This activity supports the effort at the University of Sheffield to engage women in science.
Year(s) Of Engagement Activity 2021
URL https://sites.google.com/sheffield.ac.uk/womendcs/