Twenty20Insight

Lead Research Organisation: Durham University
Department Name: Computer Science

Abstract

As digital technology permeates every area of modern life, we risk becoming over-dependent on complex systems that operate in an opaque way, creating a risk that they exhibit emergent properties that adversely affect their users or their wider environment. This is particularly true as developers increasingly rely on AI or ML techniques as a means to define system behaviour when the problem space is too complex or poorly understood for human developers to explicitly specify that behaviour. We are tackling incompletely understood problems by developing systems whose behaviour and wider impact are by necessity also incompletely understood. This trend, which is largely enabled by an abundance of data harvested from (e.g.) mobile devices, sensors and social media, is radically changing how systems are developed and how they are used. We need a new approach to software engineering that

(i) places greater emphasis on making explicit the risks of unintended behaviour for innovative new software products either through limitations on our understanding of the envisioned product's behaviour or through misuse, and

(ii) actively supports explainability of the exposed behaviour by the running system.

Twenty20Insight is an interdisciplinary project bringing together academic experts in Software Engineering (SE), RE, Design Thinking and ML to help system stakeholders and developers understand and reason about the impact of intelligent systems on the world in which they operate.

Planned Impact

We expect the project to have a major impact on five classes of beneficiaries beyond the academic research community:

Industrial partners. Twenty20Insight will have an early impact on our industrial partners, with whom we will work to advance the state-of-the-art in stakeholder engagement for the co-design of intelligent systems, and techniques for reasoning about uncertainty and making ML systems' decision-making more transparent.


Early career researchers. The project will provide the attached PDRAs with advanced training in research methods, publication and career development, which will eventually develop the PDRAs into independent researchers by the end of the project.

Research community. Twenty20Insight will develop fundamental new knowledge about engineering intelligent systems both within and at the boundaries between the design, RE/SE, AI and ML research communities.

Industries revolutionised by AI. AI/ML can be applied to nearly every industrial sector. Designing the increasingly complex AI systems requires stakeholders to understand both the Horizon of Possibilities and Envelope of Acceptability when applying ML to their problems. This project has a great potential to tackle such a problem and thus will significantly impact all the industries revolutionised by AI.

Software development companies. The new tools and techniques that will be developed in this project will have the potential to enhance software development companies to meet the increasing demand for intelligent systems operating in an uncertain world.

Wider society. The principal beneficiary of initiatives that aim to generate transformative ideas in order to advance technology, such as the Twenty20Insight project, is ultimately the wider society. This is particularly true of Twenty20Insight since one our explicit aims is to discover techniques to better understand systems' societal impact and enable stakeholders to handle such knowledge in a systematic and transparent way.

Publications

10 25 50
 
Description We have the following achievements and findings:
Further techniques for quantifying the impact of values on requirements and software engineering (O1, WP2, WP3).

For the above, we have added new publications:

2 Conference Publications:
- "History-aware explanations: towards enabling human-in-the-loop in self-adaptive systems", Juan Parra-Ullauri, Antonio García-Domínguez, Nelly Bencomo, Luis Garcia-Paucar/ Book
Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, MODELS 2022

- "The Implications of 'Soft'Requirements", Alistair Sutcliffe, Pete Sawyer, Nelly Bencomo
Conference 2022 IEEE 30th International Requirements Engineering Conference (RE)
178-188, IEEE

1 Journal paper (in SoSym) to support runtime awareness about the levels of satisfaction of quality properties and measure degrees of uncertainty to support autonomous decision-making (O2, O3, WP3, WP4, WP5)
- "The uncertainty interaction problem in self-adaptive systems", Javier Cámara, Javier Troya, Antonio Vallecillo, Nelly Bencomo, Radu Calinescu, Betty HC Cheng, David Garlan, Bradley Schmerl, 2022, Journal Software and Systems Modeling, 21,. 4 1277-1294, Springer Berlin Heidelberg


Further steps towards incorporating speculative techniques in Requirements Engineering to better anticipate unwanted emergent behaviour in uncertain environments. (O1, WP1, WP2). One further workshop was delivered to the industry partner (BT) and students and colleagues at Durham University. Based on the results, we have 1 Conference paper under submission.
Exploitation Route Two research proposals are under submission, and two are being prepared to follow the work of "towards incorporating speculative techniques in Requirements Engineering and Software Engineering" and decision-making under uncertainty for Digital Health.
Sectors Digital/Communication/Information Technologies (including Software),Healthcare

URL https://aihs.webspace.durham.ac.uk/Twenty20insight/
 
Description We continued to take the first steps towards incorporating speculative techniques in Requirements Engineering to better anticipate unwanted emergent behaviour in uncertain environments. The process has been used to deliver two new workshops on using Digital Twins with our Industry partner (BT) and with researchers at Durham University. We have submitted a research paper to a Conference and have applied for further funding taking our work to be applied on the domain of Digital Health.
First Year Of Impact 2023
Sector Digital/Communication/Information Technologies (including Software),Healthcare
Impact Types Societal

 
Description Requirements Models for Artificial Intelligence (ReqModAI): Framework and Case study
Amount £4,991 (GBP)
Funding ID RF090115 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 06/2022 
End 06/2022
 
Title RDM case study Logistic Regression to support Digital Twins Fidelity 
Description RDM case study Logistic Regression to support Digital Twins Fidelity 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact The state of a self-adaptive system (SAS) can be described using the quality properties that it needs to satisfy (i.e. its quality objectives). However, the states of these quality properties are often not directly observable due to uncertain environmental contexts. Instead, a belief over these hidden states can be maintained. Partially Observable Markov Decision Processes (POMDPs) provide a principled approach to decision-making for SASs as they seek to satisfy their, often competing, quality objectives. Here, beliefs in POMDPs can represent the probabilities of the SAS's quality objectives being met. However, how accurately do these beliefs reflect the real but hidden states of the quality objectives? In this paper, we focus on answering this question, as poor accuracy would inevitably lead to poor decision-making. We evaluate the beliefs for both single- objective (with single reward) and multi-objective (with multiple rewards) POMDPs to underpin the decision-making in two case studies, in the networking and Internet of Things (IoT) domains. Logistic Regression is used to estimate the fidelity (i.e. accuracy) of these beliefs when reflecting the hidden state of the SASs in terms of the satisfaction of the quality objectives. Our results provide strong evidence that beliefs in POMDPs offer a good representation of the state of the quality objectives in SASs. 
URL https://gitlab.com/re_research/rdmcasestudylogisticregression/
 
Description 3rd Winter Modeling Meeting San Vigilio di Marebbe, Italy - February 4-11 2023 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The scope of the meeting was to bring together world-class experts in modeling, model-based engineering, model-driven engineering, and language engineering (briefly M-ALL) to examine the impediments that prevent M-ALL from being more prominently considered by students, researchers, and practitioners. M-ALL has been successfully applied in many software and system engineering areas and now addresses even broader and novel scenarios, e.g., digital twins and analytics, in an increasing number of application domains, including natural and multi-physics systems. Such new perspectives challenge both the seminal, yet limited, foundations, e.g., conformity and instantiation, on which most of the tools and methods are based, as well as the way of teaching M-ALL to possibly heterogeneous communities (from software engineers to domain experts and citizen developers). During the meeting, we discussed, defined and formalized the new M-ALL foundations required to address the aforementioned upcoming scenarios.

I presented the work done with my researchers in Twenty20Insight about the Fidelity of DTs and Models@run.time. A paper is under developement and will be submitted to a Journal in Summer.
Year(s) Of Engagement Activity 2023
URL https://wmm23.notion.site/
 
Description Invitation to talk at Dagstuhl Seminar 22362 Model-Driven Engineering of Digital Twins ( Sep 04 - Sep 09, 2022 ), Germany 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The intended primary goal of the seminar was to create a community and establish a research roadmap; we discussed the following topics:
- Challenges faced in the real-world development of Digital Twins (DT).
- Opportunities offered by MDE.
- Active exploration of collaboration opportunities.

A key outcome of the seminar was a research roadmap for the new software engineering discipline for Digital Twins (DT).

I presented the work done with my researcher Huma Samin in Twenty20Insight about the Fidelity of DT.
A paper has been developed and will be submitted to a Journal soon.
Year(s) Of Engagement Activity 2022
URL https://www.dagstuhl.de/en/seminars/seminar-calendar/seminar-details/22362
 
Description MDENet's Annual Symposium 2022. https://mde-network.com/annual-symposium-dec-22-recordings/ 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dr Huma Samin and I presented the work done in Twenty20Insight about ReqModAI Requirements Models for Artificial Intelligence: Framework and Casestudy (WP3).
Year(s) Of Engagement Activity 2022
URL https://mde-network.com/annual-symposium-dec-22-recordings/