Turtles: Protocol-Based Foundations for Distributed Multiagent Systems

Lead Research Organisation: University of Oxford
Department Name: Computer Science

Abstract

There is growing interest in distributed systems and architectures whose components are autonomous social parties such as humans and organisations. The parties in such systems interact with each other via their software agents for the purposes of exchanging information and services. The interactions normally take the form of conversations (as opposed to invocations) realised over asynchronous messaging. Naturally, a crucial area of study for computer science and software engineering is the specification and enactment of interaction protocols, that is, the rules of encounter by which parties in the system would interact. Considered as such, the notion of protocol represents a generalisation of the notion of "contract" advocated in Design by Contract approaches.

A key question concerns the nature of contracts. Work in areas such as concurrency and Web services, has predominantly conceptualised protocols in terms of message ordering and occurrence constraints that must be respected by the parties' agents. We refer to such protocols as messaging protocols. Although messaging protocols serve the important purpose of distributed coordination, considered as contracts, they are too low-level for multiagent settings of autonomous parties. Specifically, they do not capture social constraints such as the commitments that are binding on the parties in the interaction.

This gap represents a substantial opportunity. In real life, commitments in fact represent the atoms of what people normally understand as contracts. Commitments accommodate the balance between, on the one hand, autonomy and flexibility, and, on the other hand, correct behaviour. Commitment specifications capture stakeholder requirements in multiparty domains. Further, the states of commitments underlie most key performance indicators (KPIs) that stakeholders are interested in any multiagent domain. Work in commitment protocols in multiagent systems has made progress in developing computational abstractions for commitments. However, important challenges related to expressiveness and distributed enactment of commitment protocols have not even been adequately formulated, let alone tackled. Ensuring correct distributed enactment for expressive commitment protocols is crucial to realising the full value of commitments as a human-level architectural abstraction.

The broad objective of Turtles is to bring commitment-based contracts to distributed computing. This project develops foundational theory, software, and methodology for building commitment-based distributed systems. To encourage wider adoption, the project will embed the algorithms in prototypes, and develop a tool-supported methodology for specifying and implementing social protocols. Further, Turtles will develop real systems based on use cases and practices from a number of industrial partners and evaluate these systems based on their feedback.

The success of Turtles will enable capturing important subtleties of real-life social and business interactions and transform how we design software systems for crucial multiparty domains such as healthcare, disaster response, smart cities, banking, education, and e-commerce and e-business, where commitments are crucial.

Planned Impact

We list the potential beneficiaries and summarise the potential impact on each of them. (Please also see Pathways to Impact.)

KNOWLEDGE:

Researchers: Turtles will contribute foundational scientific knowledge crucial to building distributed systems of multiple parties. This will enable bringing together the currently distinct communities of programming languages and multiagent systems and lead to the formation of joint new communities and agendas based on computational social abstractions. The impact on service-oriented computing and business process management will also be profound.

Software developers: Turtles will contribute knowledge toward the development of systems that support business interactions within and across organisations. This knowledge will improve communication with stakeholders, and simplify the development and maintenance of such systems.

Stakeholders: Turtles will contribute knowledge that enables stakeholders to understand requirements and key performance indicators in terms of commitments, and to relate these artifacts to software.

PEOPLE:

Researchers and students: Turtles will train PDRAs and PhD students working on related projects. The project's results will also inform undergraduate and graduate students via courses and tutorials.

Software developers and stakeholders: Turtles will train software developers and stakeholders via the substantial engagement with our partners.

Protocol analysts: In the medium-to-long run, we envisage the emergence of a new class of professionals called protocol analysts, who will be experts in specifying, analysing, and implementing social protocols.

SOCIETAL:

Standards bodies: Standardisation is key to better interoperability and therefore better services in important societal domains such as healthcare, smart cities, energy, and so on. Standardisation efforts are enormously complex and standards are often not adopted. Sometimes, the reason for this is due to unclear and complex specifications. Turtles will simplify the creation of protocol-based standards via a formal, tool-supported language that enables expressing both the messaging and high-level concerns.

ECONOMIC:

Business ecosystems: Via easier standardisation, Turtles will support the proliferation of business ecosystems consisting of stakeholders, including government, and developers in many multiparty domains. This will lead to reduced implementation costs and increased business engagements via better interoperability. To give a concrete example, RosettaNet is used by Intel and other companies to conduct billions of dollars worth of business. A study showed that adopting the RosettaNet protocols reduced transaction costs and increased throughput (http://tinyurl.com/p38nc7r). A healthcare study postulated billions of dollars worth of savings for the United States both during protocol implementation and post-implementation periods (http://tinyurl.com/o7sv8y6).
 
Description Some developers are interested in our research as they are potentially useful to software development.

Session types are the analogue of data types for communication. A session type will again act as a fundamental unit of compositionality: the first thing a developer writes or reads about each process will be the session types of its communications, and session type discipline will guarantee that communications from different processes are in accord with the session type.

This project has carried forward the above research programme with deep insights into the logical structure of session types, with a series of implementations in widely-used languages (including C, MPI, Erlang, Go, Haskell, Java, F#, PureScript, Scala, OCaml, Haskell, TypeScripts, Rust and Python) and in specialist languages for session types (such as Session C), and with collaborations with industry. Industrial applications of session types include the Scribble specification language maintained by Red Hat, the design of monitoring software used by the Ocean Observatories Initiative, and analysis of microservices carried out by ThoughtWorks and Estafet. We have disseminated our results through academic publication, by participating in summer schools, and by organising workshops.

Many keynote talks at the international conferences and workshops such as FCT'21 and ST'30 (Oct 23)
Exploitation Route Several prototype tools were published and used by both academia and industry. Including Scribble (FASE 2017), (POPL 2017, ICSE 2018) Session Type Providers (CC 2018). The joint-development of Scribble continues with RedHat, and they will eventually consider to integrate their existing projects.

For more details of other tools please see http://mrg.cs.ox.ac.uk/tools
Sectors Digital/Communication/Information Technologies (including Software)

Education

Financial Services

and Management Consultancy

Security and Diplomacy

URL https://mrg.cs.ox.ac.uk/
 
Description The tools developed by Turtles are currently used for other two DSbD projects, AppControl and Morello-HAT.
First Year Of Impact 2023
Sector Digital/Communication/Information Technologies (including Software),Financial Services, and Management Consultancy,Security and Diplomacy
Impact Types Policy & public services

 
Title Artifact: Deadlock-Free Asynchronous Message Reoerderign in Rust with Multiparty Session Types 
Description Artifact of the paper: "Deadlock-Free Asynchronous Message Reoerderign in Rust with Multiparty Session Types", PPoPP 2022. 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact The Rumpsteak framework is a framework for MPST in Rust. This software is an artifact that shows its capabilities. 
URL https://zenodo.org/record/5786034
 
Title Artifact: Design-By-Contract for Flexible Multiparty Session Protocols 
Description Artifact for the paper : Design-By-Contract for Flexible Multiparty Session Protocols 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact Artifact supporting the paper : Design-By-Contract for Flexible Multiparty Session Protocols 
URL https://drops.dagstuhl.de/entities/document/10.4230/DARTS.8.2.21
 
Title Artifact: Dynamically Updatable Multiparty Session Protocols 
Description Artifact for the paper: Dynamically Updatable Multiparty Session Protocols 
Type Of Technology Software 
Year Produced 2023 
Open Source License? Yes  
Impact Artifact supporting the paper Dynamically Updatable Multiparty Session Protocols 
URL https://drops.dagstuhl.de/entities/document/10.4230/DARTS.9.2.10
 
Title Artifact: Generalised Multiparty Session Types with Crash-Stop Failures 
Description Artifact supporting the paper : Generalised Multiparty Session Types with Crash-Stop Failures 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact Artifact for the paper : https://github.com/alcestes/mpstk-crash-stop 
URL https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.CONCUR.2022.35
 
Title Artifact: Generic Go to Go: Dictionary-Passing, Monomorphisation, and Hybrid 
Description Artifact for paper: Generic Go to Go: Dictionary-Passing, Monomorphisation, and Hybrid 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact Artifact for paper : Generic Go to Go: Dictionary-Passing, Monomorphisation, and Hybrid 
URL https://dl.acm.org/doi/10.1145/3563331
 
Title Artifact: Rollback Recovery in Session-Based Programming 
Description Artifact for paper: Rollback Recovery in Session-Based Programming 
Type Of Technology Software 
Year Produced 2023 
Impact Artifact for paper: Rollback Recovery in Session-Based Programming 
URL https://link.springer.com/chapter/10.1007/978-3-031-35361-1_11#chapter-info
 
Title Artifact: Stay Safe Under Panic: Affine Rust Programming with Multiparty Session Types 
Description Artifact for the paper: Stay Safe Under Panic: Affine Rust Programming with Multiparty Session Types (https://doi.org/10.4230/DARTS.8.2.9) This introduces the `multicrusty` library. 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact Artifact supporting the paper. 
URL https://drops.dagstuhl.de/entities/document/10.4230/DARTS.8.2.9
 
Title Designing Asynchronous Multiparty Protocols with Crash-Stop Failures (Artifact) 
Description We introduce Teatrino, a toolchain that supports handling multiparty protocols with crash-stop fail- ures and crash-handling behaviours. Teatrino accompanies the novel MPST theory in the related article, and enables users to generate fault-tolerant protocol-conforming Scala code from Scribble protocols. Local types are projected from the global protocol, enabling correctness-by-construction, and are expressed directly as Scala types via the Effpi concurrency library. Teatrino extends both Scribble and Effpi with support for crash-stop behaviour. The generated Scala code is execut- able and can be further integrated with existing systems. The accompanying theory in the related article guarantees deadlock-freedom and liveness properties for failure handling protocols and their implementation. This artifact includes examples, extended from both session type and distributed systems literature, featured in the related article. 
Type Of Technology Software 
Year Produced 2023 
Open Source License? Yes  
Impact Artifact supporting the paper Designing Asynchronous Multiparty Protocols with Crash-Stop Failures 
URL https://zenodo.org/record/7974824