POST: Protocols, Observabilities and Session Types

Lead Research Organisation: Imperial College London
Department Name: Computing

Abstract

Communication is not only an essential organisation principle for emerging large-scale distributed applications, such as those for e-Commerce, e-Science, e-Healthcare and financial technology (FinTech): it is also an effective way to use computational resources, such as microservices and manycore chips. In this new paradigm, communication and concurrency are the norm in software development rather than a marginal concern, enabling architects and programmers to harness the power of hundreds or even thousands of concurrent processes interacting through *message passing*.

However, for this paradigm there is no well-established methodology for software development with safety and security gurantee based on clear and mathematically accurate criteria on its behaviour. This leaves uncertainty on the correctness of the construction of distributed infrastructure. The aim of this fellowship is to establish general and practical foundations for safety enforcement of communication-intensive concurrent and distributed applications, building on a general theory of *multiparty session types*.

Communications in a distributed application are commonly organised into multiple structured conversations (*protocols*) where a developer or programmer wishes to enforce *observabilities* of system behaviours to follow a safety and security criteria given by a protocol. Here *observability* of systems behaviours means a visible sequence of message exchanges with more complex information such as dependency of data, secure information, cost and timing of communications. In the multiparty session types, an end-point system properly carries out its responsibility, so that observable systems behaviours as a whole obey an agreed-upon protocol.

Multiparty session types articulate the basic dynamics in a respective computing paradigm, thus serving as a foundation for modelling, specification, verification, systematic testing and certification, enhanced with other methods such as monitoring and logical assertions. This fellowship aims to fulfil this potential of multiparty session types as types for communication by carrying out experiments. To achieve this goal, the following technical objectives have been identified:

1. The establishment of a uniform type theory for multiparty session types capturing a full range of application-level protocols based on behavioural theory and game semantics, as a foundation of the whole methodology.

2. The establishment of a dependent and refinement type theory of specifications and verifications; and of a scalable algorithm to verify safety and security properties based on automata theory.

3. The development and release of an open-source toolchain, based on (1,2), combined with Application Programming Interface (API) and with industry tools.

4. A theoretically well-founded architecture which can efficiently monitor, trace, log and enforce correct observational behaviour against specifications written in (3).

5. Experiments through collaboration with academic and industry partners, realising formal safety and security assurance against advanced protocols for real-world applications, including multi robotics/UAVs, financial and healthcare systems.

Throughout the research programme, an active and extensive dialogue between theories (1,2) and practice (3,4,5) will be the key enabler for reaching the goals of the fellowship, ultimately establishing cross-disciplinary and co-created ICT research. The project also links assurance methodologies based on session types to the standardisation for Cloud Computing (Cloud Native Computing Foundation) and to the public regulatory requirements for the documentation of financial and e-Healthcare protocols, meeting the goals of People at the Heart of ICT.

Planned Impact

This fellowship aims to establish the safety and security assurance principles for communication-intensive software based on session types. The outcomes, based on mathematically accurate criteria for communications, are likely to have deep and broad impacts on the information technology sector, and hence be of considerable benefit to society.

(1) Who might benefit from this research?

The immediate beneficiaries are the partners of this proposal and the application domains they are engaged in, i.e. micro/web-services, distributed containers, legacy code analysis, e-Science (robotics), e-Healthcare and financial technology. More generally, those engaged in the development of large-scale distributed applications will also benefit, ranging from global enterprise to infrastructure for public sectors (e.g. healthcare).

The second class of beneficiaries are those who build tools and environments for the runtime management of distributed tracing (observabilities of systems behaviours), including the middleware targeted to Cloud Computing. The validation algorithms and monitor/tracing architecture offer a comprehensive framework for (hybrid) static and runtime safety enforcement, with automatic or semi-automatic development-time verification.

The third beneficiaries are end-users and society as a whole, especially users of IT infrastructure developed in this fellowship proposal. Once the usability of the methodology is established, the new assurance methodology will lead to a comprehensive certification framework as a public standard. For example, distributed tracing is being standardised through the OpenTracing standard, a language-agnostic standard for distributed tracing, developed by the industry partner (Red Hat). Collaborations with Red Hat will have a large impact on establishing the link with two major open source projects, OpenTracing and Jaeger, part of the Cloud Native Computing Foundation (CNCF).

(2) How might they benefit from this research?

This fellowship project will produce a toolchain whose comprehensive support for software development and safety assurance will be tested through prominent real large-scale use cases with the project partners. The development and management experience with the partners will make our toolchain usable by professional developers (including architects, designers and programmers).

In the medium term, when associated development methods have reached sufficient maturity, it is possible for application providers who are using OpenTracing or GitOps to contribute to the enhancement of the toolchain or provide their own versions. Similarly the ecosystem of programming languages and APIs for protocols (i.e. Scribble framework developed) will be enriched by those working inside and outside academic institutions.

Finally the end-users will benefit from the use of safer services with a clear guarantee of their behaviour. Applications will enjoy enriched functionalities because more time and cost can be spent on new functions rather than maintenance and informal manual assurance. Such a trend will be accelerated when the competition in the software tools market for the development and assurance of communication-intensive software emerges (through DevOps), and will be further assisted by a public standard for software quality based on the proposed methodologies. This will ensure the safety of large-scale infrastructures such as financial networks, healthcare and robotics/UAVs, all of which offer critical services and knowledge to our society, and influencing the policy and regulator, putting People in the Heart of ICT.

Publications

10 25 50
 
Description Communication is also an effective way to use computational resources, such as microservices and manycore chips. In this new paradigm, communication and concurrency are the norm.The first key discovery is by combining logics and session types, we could achieve an advanced protocol specifications for distributed systems and parallel computing. The second key discovery is using the theory of asynchronous type theory, we could optimise message passing communications for high-performance computational environments. These are implemented by F* and Rust.
Exploitation Route The programmers can use our tools (all are listed in Software).
Sectors Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Financial Services, and Management Consultancy,Security and Diplomacy

URL http://mrg.doc.ic.ac.uk/
 
Description The theory developed in this project was applied to OpenTelemetry distributed tracing tool, which generated positive impacts to policy, economic and society. It is too early to say the usability of other tools and theory in this project.
First Year Of Impact 2022
Sector Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Security and Diplomacy,Other
Impact Types Societal,Economic,Policy & public services

 
Description Binghamton University 
Organisation Binghamton University
Country United States 
Sector Academic/University 
PI Contribution Collaborations, software development, technical discussion and meetings
Collaborator Contribution Collaborations, software development, technical discussion and meetings
Impact A dialogue has started towards the goals of the fellowship, from the theory, to applications and ultimately aiming at establishing cross-disciplinary and co-created ICT research. The project links assurance methodologies based on session types to the standardisation for robotics (and other applications) and to the public regulatory requirements for the documentation of financial and e-Healthcare protocols, meeting the goals of People at the Heart of ICT.
Start Year 2020
 
Description Cognizant 
Organisation Cognizant Technology Solutions
Country United States 
Sector Private 
PI Contribution Collaboration with Cognizant and Estafet applied Scribble developed by Imperial in a microservice environment.
Collaborator Contribution Our partners proposes to use Scribble in their product.
Impact There is an ongoing dialogue on using Scribble in the partner's products, including ZDLC https://www.cognizant.com/ZDLC.
Start Year 2013
 
Description Development of the Scribble language and framework 
Organisation Cognizant Technology Solutions
Country United States 
Sector Private 
PI Contribution Developing an open source product.
Collaborator Contribution Providing use cases, technical meetings, and hosting KTS.
Impact TBD
Start Year 2013
 
Description Development of the Scribble language and framework 
Organisation Red Hat UK
Country United Kingdom 
Sector Private 
PI Contribution Developing an open source product.
Collaborator Contribution Providing use cases, technical meetings, and hosting KTS.
Impact TBD
Start Year 2013
 
Description Development of the Scribble language and framework 
Organisation University of California, San Diego (UCSD)
Country United States 
Sector Academic/University 
PI Contribution Developing an open source product.
Collaborator Contribution Providing use cases, technical meetings, and hosting KTS.
Impact TBD
Start Year 2013
 
Description Development of the Scribble language and framework 
Organisation VMware, Inc.
Country United States 
Sector Private 
PI Contribution Developing an open source product.
Collaborator Contribution Providing use cases, technical meetings, and hosting KTS.
Impact TBD
Start Year 2013
 
Description Estafet 
Organisation Estafet Ltd
Country United Kingdom 
Sector Private 
PI Contribution Implementation of microservices monitoring based on Scribble.
Collaborator Contribution Technical discussions and collaboration on partner's product.
Impact A product based on monitor generation for microservices based on the Scribble framework was proposed. See http://estafet.com/scribble-3/.
Start Year 2016
 
Description JPEX Ltd. 
Organisation JPEX Limited
Country United Kingdom 
Sector Private 
PI Contribution Collaborations, software development, technical discussion and meetings
Collaborator Contribution Collaborations, software development, technical discussion and meetings
Impact A dialogue has started towards the goals of the fellowship, from the theory, to applications (robotics) and ultimately aiming at establishing cross-disciplinary and co-created ICT research. The project links assurance methodologies based on session types to the standardisation for Cloud Computing (Cloud Native Computing Foundation) and to the public regulatory requirements for the documentation of financial and e-Healthcare protocols, meeting the goals of People at the Heart of ICT.
Start Year 2020
 
Description Max Planck Institutes (Grouped) 
Organisation Max Planck Society
Country Germany 
Sector Charity/Non Profit 
PI Contribution Collaborations, software development, technical discussion and meetings
Collaborator Contribution Collaborations, software development, technical discussion and meetings
Impact A dialogue has started towards the goals of the fellowship, from the theory, to applications (robotics) and ultimately aiming at establishing cross-disciplinary and co-created ICT research. The project links assurance methodologies based on session types to the standardisation for robotics (and other applications) and to the public regulatory requirements for the documentation of financial and e-Healthcare protocols, meeting the goals of People at the Heart of ICT.
Start Year 2020
 
Description November Group, LLC 
Organisation November Group, LLC
Country United States 
Sector Private 
PI Contribution Ongoing discussions with the partners on applications of Scribble developed at Imperial with logical assertions, and applying the technology to specify and verify smart contract on a block-chain.
Collaborator Contribution Use cases for session types and Scribble.
Impact .
Start Year 2013
 
Description RedHat 
Organisation Red Hat UK
Country United Kingdom 
Sector Private 
PI Contribution Development of open source software Scribble
Collaborator Contribution Collaborations, software development, technical discussion and meetings
Impact The open source tool Scribble is available online at https://github.com/scribble/scribble-java. The software led to a number of publications in international conference such as FASE 2016, FASE 2017, ECOOP 2016 and ECOOP 2017.
Start Year 2013
 
Description Weaveworks 
Organisation Weaveworks Ltd
Country United States 
Sector Private 
PI Contribution There is an ongoing discussions on the topics of distributed containers and Kubernetes development and configurations, as well as recent applications of session types in Go.
Collaborator Contribution Use cases of session types.
Impact .
Start Year 2013
 
Title Artifact: Deadlock-Free Asynchronous Message Reoerderign in Rust with Multiparty Session Types 
Description Artifact of the paper: "Deadlock-Free Asynchronous Message Reoerderign in Rust with Multiparty Session Types", PPoPP 2022. 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact The Rumpsteak framework is a framework for MPST in Rust. This software is an artifact that shows its capabilities. 
URL https://zenodo.org/record/5786034
 
Title CAMP: Cost-Aware Multiparty Session Protocols (artifact) 
Description This is the artifact for the paper *CAMP: Cost-Aware Multiparty Session Protocols*.
The artifact comprises: - A library for specifying cost-aware multiparty protocols.
- The raw data used for comparing the cost models with real execution costs.
- The cost-aware protocol specifications of the benchmarks that we studied. The library for specifying cost-aware protocols also provides functions for
extracting cost equations from them, and for estimating recursive protocol
latencies (i.e. average cost per protocol iteration). We provide a script for
extracting cost equations, and instantiating them using the parameters used in
the paper.
 
Type Of Technology Software 
Year Produced 2020 
Open Source License? Yes  
Impact Led to a publication to OOPSLA 2020. 
URL https://zenodo.org/record/4046893
 
Title EMTST 
Description A Coq library to implement and reason about Session types. Together with a subject reduction for binary session types. 
Type Of Technology Software 
Year Produced 2020 
Impact Led to a publication at TACAS 2020 
URL https://doi.org/10.5281/zenodo.3516299
 
Title Featherweight Go (Artifact) 
Description This paper presents Featherweight Go (FG) and Featherweight Generic Go (FGG), a core calculus of Go and a proposal for extending it with F-bounded polymorphism. The calculi are in the same vein as Featherweight Java (FJ), but where Featherweight Generic Java (FGJ) was translated into FJ via erasure, FGG translates into FG via monomorphization (which is also formalized). The two calculi are proven sound using the normal progress and preservation arguments. Additionally a bisimulation is shown to exist between a FGG program and its monomorphization (if it exists); in other words that monomorphization preserves the semantics of the program. The artifact consists of an implementation of type checkers and interpreters for FG and FGG, as well as a monomorphization procedure (including the check if it is possible). It includes the examples from the paper, and a comparison using the Go compiler as reference. Type preservation and bisimulation for these programs are tested dynamically. Additionally, the same is tested for all well-typed programs up to a certain size (which are generated in a manner similar to property-based testing). 
Type Of Technology Software 
Year Produced 2020 
Open Source License? Yes  
Impact Led to a publication at OOPSLA 2020 
URL https://zenodo.org/record/4048297
 
Title Godel2 
Description Godel2 is a model checker for MiGo types that returns results of liveness and safety checks for channels and mutex locks, as well as the presence of data races on shared pointers. 
Type Of Technology Software 
Year Produced 2020 
Impact Led to a publication at ECOOP 2020 
 
Title KMC 
Description A tool for checking k-multiparty compatibility in communicating session automata 
Type Of Technology Software 
Year Produced 2019 
Impact Led to a publication at CAV 2019 
URL https://bitbucket.org/julien-lange/k-checking
 
Title Multiparty Motion Coordination: From Choreographies to Robotics Programs (Artifact) 
Description Software artifact for the paper "Multiparty Motion Coordination: From Choreographies to Robotics Programs" submitted to OOPSLA 2020 The artifact has been packaged into a virtual machine (Ubuntu 20.04). The username and password for the virtual machine is "pgcd". 
Type Of Technology Software 
Year Produced 2020 
Open Source License? Yes  
Impact Led to a publication at OOPSLA 2020 
URL https://zenodo.org/record/4051783
 
Title NuScr - An implementation of MPST toolchain in OCaml 
Description Multiparty session types (MPST) provide a typing discipline for message-passing concurrency, ensuring deadlock freedom for distributed processes. We present NuScr (Nu-Scribble), an extensible toolchain, implemented in OCaml, for MPST-based protocols. The toolchain can convert multiparty protocols in the Scribble protocol description language into global types in the MPST theory; global types are projected into local types, and local types are converted to their corresponding communicating finite state machine (CFSM). The toolchain also generates OCaml APIs from CFSMs, which implement endpoints in the protocol. Our design allows for language-independent code generation, and opens possibilities to generate APIs for other programming languages. We design our toolchain with modularity and extensibility in mind, so that extensions of core MPST can be easily integrated within our framework. 
Type Of Technology Software 
Year Produced 2020 
Impact The NuScr toolchain has a modular and extensible design, so that extensions of core MPST can be easily integrated within our framework. 
URL https://nuscr.github.io/nuscr/
 
Title Parallel Algebraic Language 
Description A Haskell EDSL for building session-typed parallel code. 
Type Of Technology Software 
Year Produced 2020 
Impact Led to a publication at CC 2020. 
 
Title STScript 
Description STScript is a toolchain that generates TypeScript APIs for communication-safe web development over WebSockets using routed MPST (RouST). STScript uses TypeScript, ReactJS, and NodeJS as an implementation language, client framework and server framework respectively. 
Type Of Technology Software 
Year Produced 2020 
Impact Led to a publication at CC 2021. 
 
Title Session* 
Description Session* is a toolchain for specifying message passing protocols using Refined Multiparty Session Types and safely implementing the distributed endpoint programs in F*. 
Type Of Technology Software 
Year Produced 2020 
Impact Led to a publication at OOPSLA 2020. 
 
Title Zooid: a DSL for Certified Multiparty Computation 
Description Artifact for the Zooid language and Coq mechanisation of the metathory of multiparty session types. This artifact contains the implementation for "Zooid: a DSL for Certified Multiparty Computation" from PLDI'21 This code is maintained at: https://github.com/emtst/zooid-cmpst 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact See: http://mrg.doc.ic.ac.uk/publications/zooid-paper/ 
URL https://zenodo.org/record/4581294
 
Title dot-parser 
Description A parser for DOT files in Rust 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact No such parser previously existed in Rust. This is used for the developement of the Rumpsteak framework. 
URL https://crates.io/crates/dot-parser
 
Title mpstpp 
Description Multiparty Session Types Plus + 
Type Of Technology Software 
Year Produced 2020 
Impact Led to a publication at ESOP 2020 
 
Title ocaml-mpst 
Description ocaml-mpst is a communication library powered by Multiparty Session Types (abbreviated as MPST) in OCaml. Thus it ensures: * Deadlock-freedom, * Protocol fidelity (communication will take place according to a prescribed protocol) and * Communication safety (you do not get any type-mismatch errors) --- under the assumption that all communication channels are used linearly. 
Type Of Technology Software 
Year Produced 2020 
Impact Led to a publication at ECOOP 2020