FlexCap: Exploring Hardware Capabilities in Unikernels and Flexible Isolation OSes

Lead Research Organisation: The University of Manchester
Department Name: Computer Science

Abstract

In this project, entitled FlexCap, we propose to study the use of Morello's hardware capabilities to provide safety and isolation in Operating Systems (OSes). We propose to enable capability support and evaluate its efficiency in two OSes: FlexOS, a library OS offering a highly-configurable isolation profile that can be customised at build time towards specific use cases; as well as Unikraft, a high-performance/low latency unikernel.

FlexOS allows the user to specialise the isolation/safety strategy of the operating system seamlessly at build time. Several fundamental parameters are customisable, including the granularity of kernel components isolation as well as the hardware mechanism used to enforce that isolation. FlexOS currently supports the Intel Memory Protection Keys and Extended Page Table mechanisms. Porting the OS to Morello would enable to benefit from the efficient compartmentalization brought by capabilities. In particular, the fine-grained memory protection and high degree of scalability resulting from the use of hardware capabilities should increase performance as well as security security, and decrease memory footprint in FlexOS, compared to the other mechanisms currently supported by the OS.

FlexOS is itself an extension of the Unikraft unikernel, so porting FlexOS to Morello will first require porting Unikraft to the platform. Unikraft is a high-performance/low-latency unikernel targeting cloud applications. The high degree of performance it provides is achieved by running the application and the kernel code inside a single, completely unprotected address space. This obviously raises security concerns and porting Unikraft to Morello will allow to explore bringing back safety into high-performance unikernels, leveraging the security benefits brought by the capabilities' ability to provide safe versions of legacy programming languages (i.e. pure/hybrid capabilities).

Finally, we also propose to explore advanced use of capabilities in FlexOS and Unikraft by studying 1) the possibility of incremental porting of Unikraft to pure capabilities and 2) horizontal compartmentalization of FlexOS components.

These two OSes, FlexOS and Unikraft, are unique use cases for the application of Morello's hardware capabilities, and have never been explored in this context. These use cases differ significantly from the two OSes already available in the CHERI/Morello's software ecosystem (CheriBSD and CheriOS). CheriBSD is a general purpose monolithic OS and is unlikely to achieve the high level of performance of Unikraft. It also does not lend itself to flexible isolation like FlexOS. Further, CheriOS is unlikely to offer the same performance and compatibility with existing applications as Unikraft/FlexOS.
 
Description The development and evaluation of the ports of Unikraft/FlexOS to the ARM Morello platform has shown that hardware capability-based compartmentalisation can be achieved with a variable engineering cost, which can be quite low if one is willing to trade-off on scalability and security, and that performance overheads are similar to other intra-address space isolation mechanisms (e.g. memory protection keys), and lower than more heavyweight (page table/extended page table) solutions.
Exploitation Route We have open-sourced all the developed software for other researchers/practicioners to reuse, build upon, or compare against.
Sectors Digital/Communication/Information Technologies (including Software)

URL https://olivierpierre.github.io/project-flexcap/
 
Title Loupe: Driving the Development of OS Compatibility Layers 
Description This artifact contains the source code of Loupe, presented at ASPLOS'24 ("Loupe: Driving the Development of OS Compatibility Layers"), the proof-of concept of our OS feature analysis method, along with the OS feature usage data generated for the paper. The goal of this artifact is to allow readers to understand and reuse Loupe in their experimental OS development workflows Abstract of the paper: Supporting mainstream applications is fundamental for a new OS to have impact. It is generally achieved by developing a layer of compatibility allowing applications developed for a mainstream OS like Linux to run unmodified on the new OS. Building such a layer, as we show, results in large engineering inefficiencies due to the lack of efficient methods to precisely measure the OS features required by a set of applications. We propose Loupe, a novel method based on dynamic analysis that determines the OS features that need to be implemented in a prototype OS to bring support for a target set of applications and workloads. Loupe guides and boosts OS developers as they build compatibility layers, prioritizing which features to implement in order to quickly support many applications as early as possible. We apply our methodology to 100+ applications and several OSes currently under development, demonstrating high engineering effort savings vs. existing approaches: for example, for the 62 applications supported by the OSv kernel, we show that using Loupe, would have required implementing only 37 system calls vs. 92 for the non-systematic process followed by OSv developers. We study our measurements and extract novel key insights. Overall, we show that the burden of building compatibility layers is significantly less than what previous works suggest: in some cases, only as few as 20% of system calls reported by static analysis, and 50% of those reported by naive dynamic analysis need an implementation for an application to successfully run standard benchmarks. 
Type Of Material Database/Collection of data 
Year Produced 2023 
Provided To Others? Yes  
Impact The tool is actively used by a company (Unikraft GmbH). Several researchers from academia and industry declared their interest for the tool. 
URL https://zenodo.org/record/8386115
 
Description Collaboration with Microsoft Research and Imperial College London 
Organisation Imperial College London
Country United Kingdom 
Sector Academic/University 
PI Contribution Following the internship of a PhD student involved in the project, we published a paper on that topic, in collaboration with Imperial College London. Most of the technical work was doen by the student while at MSR. The paper writing was done jointly between UoM, MSR, and Imperial.
Collaborator Contribution Following the internship of a PhD student involved in the project, we published a paper on that topic, in collaboration with Imperial College London. Most of the technical work was doen by the student while at MSR. The paper writing was done jointly between UoM, MSR, and Imperial.
Impact Paper publication: Hugo Lefeuvre, David Chisnall, Marios Kogias, and Pierre Olivier, "Towards (Really) Safe and Fast Confidential I/O", 19th Workshop on Hot Topics in Operating Systems (HotOS), 2023
Start Year 2022
 
Description Collaboration with Microsoft Research and Imperial College London 
Organisation Microsoft Research
Department Microsoft Research Cambridge
Country United Kingdom 
Sector Private 
PI Contribution Following the internship of a PhD student involved in the project, we published a paper on that topic, in collaboration with Imperial College London. Most of the technical work was doen by the student while at MSR. The paper writing was done jointly between UoM, MSR, and Imperial.
Collaborator Contribution Following the internship of a PhD student involved in the project, we published a paper on that topic, in collaboration with Imperial College London. Most of the technical work was doen by the student while at MSR. The paper writing was done jointly between UoM, MSR, and Imperial.
Impact Paper publication: Hugo Lefeuvre, David Chisnall, Marios Kogias, and Pierre Olivier, "Towards (Really) Safe and Fast Confidential I/O", 19th Workshop on Hot Topics in Operating Systems (HotOS), 2023
Start Year 2022
 
Description Collaboration with Rice University (USA) 
Organisation Rice University
Country United States 
Sector Academic/University 
PI Contribution We published 2 papers on the topic of interface security in compartmentalised software. UoM is the main contributor behind the first paper in terms of software development, evaluation, and paper redaction.
Collaborator Contribution We published 2 papers on the topic of interface security in compartmentalised software. Rice is the main contributor behind the second paper in terms of software development, evaluation, and paper redaction.
Impact 2 papers published: - Hugo Lefeuvre, Vlad-Andrei Badoiu, Yi Chien, Felipe Huici, Nathan Dautenhahn, and Pierre Olivier, "Assessing the Impact of Interface Vulnerabilities in Compartmentalized Software", Network and Distributed System Security (NDSS), 2023 - Yi Chien, Vlad Badoiu, Yudi Yang, Yuqian Huo, Kelly Kaoudis, Hugo Lefeuvre, Pierre Olivier, and Nathan Dautenhahn, "CIVSCOPE: Analyzing Potential Memory Corruption Bugs in Compartment Interfaces by Establishing Lower Bound and Upper Bound", Workshop on Kernel Isolation, Safety and Verification (KISV), 2023
Start Year 2022
 
Description Collaboration with the Unikraft community 
Organisation Karlsruhe Institute of Technology
Country Germany 
Sector Academic/University 
PI Contribution We are collaborating with the Unikraft (https://unikraft.org/) community on several research avenues. 3 papers, in which Manchester is a major player (the PI's PhD student is 1st author) are accepted in top-tier systems conferences, and more papers are under submission/in the process of being written, with high degrees of involvment from Manchester.
Collaborator Contribution It is variable depending on the partners: - NEC/Unikraft.io: we have regular (weekly) meetings with the CEO and engineers from Unikraft.io, helping to drive the research and participating in paper writing. We also have an engineer from NEC working on a paper effort. - Polytechnic Bucharest: regular meetings with a professor and a PhD student, collaborating on several paper efforts - Lancaster University: regular meetings with a professor and a PhD student, collaborating on several paper efforts - KIT: a MS student from KIT is working on one of our paper efforts as part of his thesis
Impact - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, Yi Chen, Felipe Huici, Nathan Dautenhahn, Pierre Olivier, "Assessing the Impact of Interface Vulnerabilities in Compartmentalized Software", Network and Distributed System Security (NDSS) Symposium, 2023. - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, Alexander Jung, ?tefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier, "FlexOS: Towards Flexible OS Isolation", 22nd Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2022. - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, ?tefan Teodorescu, Pierre Olivier, Tiberiu Mosnoi, Razvan Deaconescu, Felipe Huici, and Costin Raiciu, "FlexOS: Making OS Isolation Flexible", 18th Workshop on Hot Topics in Operating Systems (HotOS), 2021. - Publication: Alexander Jung, Hugo Lefeuvre, Charalampos Rotsos, Pierre Olivier, Daniel Onoro-Rubio, Mathias Niepert, and Felipe Huici, "Wayfinder: Towards Automatically Deriving Optimal OS Configurations", 12th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys), 2021 - Publication: Hugo Lefeuvre, Gaulthier Gain, Daniel Dinca, Alexander Jung, Simon Kuenzer, Vlad-Andrei Badoiu, Razvan Deaconescu, Laurent Mathy, Costin Raiciu, Pierre Olivier, and Felipe Huici, "Unikraft and the Coming of Age of Unikernels", USENIX ;login:, 2021 - Publication: Hugo Lefeuvre, "FlexOS: easy specialization of OS safety properties", ACM Middleware Doctoral Workshop, 2021. - Talk given at FOSSDEM'22 https://fosdem.org/2022/schedule/event/tee_flexos/ - Tutorial on Unikraft given at ASPLOS'22 https://asplos-conference.org/tutorials/#unikraft
Start Year 2020
 
Description Collaboration with the Unikraft community 
Organisation Lancaster University
Country United Kingdom 
Sector Academic/University 
PI Contribution We are collaborating with the Unikraft (https://unikraft.org/) community on several research avenues. 3 papers, in which Manchester is a major player (the PI's PhD student is 1st author) are accepted in top-tier systems conferences, and more papers are under submission/in the process of being written, with high degrees of involvment from Manchester.
Collaborator Contribution It is variable depending on the partners: - NEC/Unikraft.io: we have regular (weekly) meetings with the CEO and engineers from Unikraft.io, helping to drive the research and participating in paper writing. We also have an engineer from NEC working on a paper effort. - Polytechnic Bucharest: regular meetings with a professor and a PhD student, collaborating on several paper efforts - Lancaster University: regular meetings with a professor and a PhD student, collaborating on several paper efforts - KIT: a MS student from KIT is working on one of our paper efforts as part of his thesis
Impact - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, Yi Chen, Felipe Huici, Nathan Dautenhahn, Pierre Olivier, "Assessing the Impact of Interface Vulnerabilities in Compartmentalized Software", Network and Distributed System Security (NDSS) Symposium, 2023. - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, Alexander Jung, ?tefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier, "FlexOS: Towards Flexible OS Isolation", 22nd Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2022. - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, ?tefan Teodorescu, Pierre Olivier, Tiberiu Mosnoi, Razvan Deaconescu, Felipe Huici, and Costin Raiciu, "FlexOS: Making OS Isolation Flexible", 18th Workshop on Hot Topics in Operating Systems (HotOS), 2021. - Publication: Alexander Jung, Hugo Lefeuvre, Charalampos Rotsos, Pierre Olivier, Daniel Onoro-Rubio, Mathias Niepert, and Felipe Huici, "Wayfinder: Towards Automatically Deriving Optimal OS Configurations", 12th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys), 2021 - Publication: Hugo Lefeuvre, Gaulthier Gain, Daniel Dinca, Alexander Jung, Simon Kuenzer, Vlad-Andrei Badoiu, Razvan Deaconescu, Laurent Mathy, Costin Raiciu, Pierre Olivier, and Felipe Huici, "Unikraft and the Coming of Age of Unikernels", USENIX ;login:, 2021 - Publication: Hugo Lefeuvre, "FlexOS: easy specialization of OS safety properties", ACM Middleware Doctoral Workshop, 2021. - Talk given at FOSSDEM'22 https://fosdem.org/2022/schedule/event/tee_flexos/ - Tutorial on Unikraft given at ASPLOS'22 https://asplos-conference.org/tutorials/#unikraft
Start Year 2020
 
Description Collaboration with the Unikraft community 
Organisation NEC Corporation
Department NEC Laboratories Europe GmbH
Country Germany 
Sector Private 
PI Contribution We are collaborating with the Unikraft (https://unikraft.org/) community on several research avenues. 3 papers, in which Manchester is a major player (the PI's PhD student is 1st author) are accepted in top-tier systems conferences, and more papers are under submission/in the process of being written, with high degrees of involvment from Manchester.
Collaborator Contribution It is variable depending on the partners: - NEC/Unikraft.io: we have regular (weekly) meetings with the CEO and engineers from Unikraft.io, helping to drive the research and participating in paper writing. We also have an engineer from NEC working on a paper effort. - Polytechnic Bucharest: regular meetings with a professor and a PhD student, collaborating on several paper efforts - Lancaster University: regular meetings with a professor and a PhD student, collaborating on several paper efforts - KIT: a MS student from KIT is working on one of our paper efforts as part of his thesis
Impact - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, Yi Chen, Felipe Huici, Nathan Dautenhahn, Pierre Olivier, "Assessing the Impact of Interface Vulnerabilities in Compartmentalized Software", Network and Distributed System Security (NDSS) Symposium, 2023. - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, Alexander Jung, ?tefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier, "FlexOS: Towards Flexible OS Isolation", 22nd Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2022. - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, ?tefan Teodorescu, Pierre Olivier, Tiberiu Mosnoi, Razvan Deaconescu, Felipe Huici, and Costin Raiciu, "FlexOS: Making OS Isolation Flexible", 18th Workshop on Hot Topics in Operating Systems (HotOS), 2021. - Publication: Alexander Jung, Hugo Lefeuvre, Charalampos Rotsos, Pierre Olivier, Daniel Onoro-Rubio, Mathias Niepert, and Felipe Huici, "Wayfinder: Towards Automatically Deriving Optimal OS Configurations", 12th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys), 2021 - Publication: Hugo Lefeuvre, Gaulthier Gain, Daniel Dinca, Alexander Jung, Simon Kuenzer, Vlad-Andrei Badoiu, Razvan Deaconescu, Laurent Mathy, Costin Raiciu, Pierre Olivier, and Felipe Huici, "Unikraft and the Coming of Age of Unikernels", USENIX ;login:, 2021 - Publication: Hugo Lefeuvre, "FlexOS: easy specialization of OS safety properties", ACM Middleware Doctoral Workshop, 2021. - Talk given at FOSSDEM'22 https://fosdem.org/2022/schedule/event/tee_flexos/ - Tutorial on Unikraft given at ASPLOS'22 https://asplos-conference.org/tutorials/#unikraft
Start Year 2020
 
Description Collaboration with the Unikraft community 
Organisation Polytechnic University of Bucharest
Country Romania 
Sector Academic/University 
PI Contribution We are collaborating with the Unikraft (https://unikraft.org/) community on several research avenues. 3 papers, in which Manchester is a major player (the PI's PhD student is 1st author) are accepted in top-tier systems conferences, and more papers are under submission/in the process of being written, with high degrees of involvment from Manchester.
Collaborator Contribution It is variable depending on the partners: - NEC/Unikraft.io: we have regular (weekly) meetings with the CEO and engineers from Unikraft.io, helping to drive the research and participating in paper writing. We also have an engineer from NEC working on a paper effort. - Polytechnic Bucharest: regular meetings with a professor and a PhD student, collaborating on several paper efforts - Lancaster University: regular meetings with a professor and a PhD student, collaborating on several paper efforts - KIT: a MS student from KIT is working on one of our paper efforts as part of his thesis
Impact - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, Yi Chen, Felipe Huici, Nathan Dautenhahn, Pierre Olivier, "Assessing the Impact of Interface Vulnerabilities in Compartmentalized Software", Network and Distributed System Security (NDSS) Symposium, 2023. - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, Alexander Jung, ?tefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier, "FlexOS: Towards Flexible OS Isolation", 22nd Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2022. - Publication: Hugo Lefeuvre, Vlad-Andrei Badoiu, ?tefan Teodorescu, Pierre Olivier, Tiberiu Mosnoi, Razvan Deaconescu, Felipe Huici, and Costin Raiciu, "FlexOS: Making OS Isolation Flexible", 18th Workshop on Hot Topics in Operating Systems (HotOS), 2021. - Publication: Alexander Jung, Hugo Lefeuvre, Charalampos Rotsos, Pierre Olivier, Daniel Onoro-Rubio, Mathias Niepert, and Felipe Huici, "Wayfinder: Towards Automatically Deriving Optimal OS Configurations", 12th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys), 2021 - Publication: Hugo Lefeuvre, Gaulthier Gain, Daniel Dinca, Alexander Jung, Simon Kuenzer, Vlad-Andrei Badoiu, Razvan Deaconescu, Laurent Mathy, Costin Raiciu, Pierre Olivier, and Felipe Huici, "Unikraft and the Coming of Age of Unikernels", USENIX ;login:, 2021 - Publication: Hugo Lefeuvre, "FlexOS: easy specialization of OS safety properties", ACM Middleware Doctoral Workshop, 2021. - Talk given at FOSSDEM'22 https://fosdem.org/2022/schedule/event/tee_flexos/ - Tutorial on Unikraft given at ASPLOS'22 https://asplos-conference.org/tutorials/#unikraft
Start Year 2020
 
Title Loupe 
Description Loupe is a tool designed to help you analyze the system call usage of applications. Loupe can do primarily two things: (1) collect data about the system call usage of a given set of applications, and (2) analyse the data collected for these applications. It can tell you what system calls a custom OS needs to implement to run the applications, and visualise these numbers in a variety of plots. 
Type Of Technology Software 
Year Produced 2023 
Open Source License? Yes  
Impact Loupe is actively used by the company Unikraft GmbH (https://unikraft.io/). Other entities (including the company Huawei and the open source community behind the ReactOS operating system) also reached out for information about Loupe. 
URL https://github.com/unikraft/loupe
 
Title Ports of the Unikraft/FlexOS operating systems to the ARM Morello platform 
Description This is our ports of the Unikraft and FlexOS operating systems to the ARM Morello platform, using its hybrid capability mode. This software was developed as part of the research effort that led to this paper: John Alistair Kressel, Hugo Lefeuvre, and Pierre Olivier, "Software Compartmentalization Trade-Offs with Hardware Capabilities", Workshop on Programming Languages and Operating Systems (PLOS), 2023 For more information see the project's website https://olivierpierre.github.io/project-flexcap/ 
Type Of Technology Software 
Year Produced 2023 
Open Source License? Yes  
Impact We published a paper in the PLOS'23 workshop. We were invited to give a talk presenting that research at the CHERITech'24 workshop. 
URL https://olivierpierre.github.io/project-flexcap/
 
Description Invited talk at Heriot-Watt University 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Dr. Olivier gave an invited talk at Heriot-Watt University's LAIV Seminar on the topic of Compatibility and Isolation in Specialised Operating Systems
Year(s) Of Engagement Activity 2023
URL https://laiv.uk/laiv-seminars/
 
Description Keynote at the Crackchester Student Association's Hackers' Hub event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact Pierre Olivier gave a keynote on the topic of systems security at the Crackchester Student Association's Hackers' Hub event
Year(s) Of Engagement Activity 2022
URL https://www.facebook.com/events/534647688274225/?ref=newsfeed
 
Description Short talk at a meeting of the Academic 'Cybersecurity Partnership project' - Nord France Invest and Midlands Cyber 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dr. Olivier gave a short talk presenting his research on the topic of cyber security at the Academic 'Cybersecurity Partnership project' - Nord France Invest and Midlands Cyber. Academics, professional practitioners, and policy markers from France and the UK participated in the meeting.
Year(s) Of Engagement Activity 2023
URL https://www.midlandscyber.com/cyber-events/academic-%E2%80%99cybersecurity-partnership-project-nord-...
 
Description Talk at FOSDEM'23 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Pierre Olivier gave a talk at FOSDEM'23 regarding Loupe (ASPLOS'24 publication).
Year(s) Of Engagement Activity 2023
URL https://archive.fosdem.org/2023/schedule/event/loupe/
 
Description Talk at the CheriTech Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact An Mphil student involved in the project presented the progress
Year(s) Of Engagement Activity 2023
URL https://www.dcs.gla.ac.uk/~jsinger/cheritech23.html
 
Description Talk at the Crackchester cyber security student association 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact Dr. Olivier gave a talk at the student association on the topic of software compartmentalisation.
Year(s) Of Engagement Activity 2023
 
Description Talk at the Huawei Future Device Technology Summit 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Presentation of FlexOS (ASPLOS'22) paper. Some members of the audience reached out afterwards and indicated they may use the software artifacts.
Year(s) Of Engagement Activity 2023
URL http://haemod.uk/documents/conference/Huawei_Future_Device_Summit_Helsinki.pdf
 
Description Talk at the Software Systems Security Seminar in Rennes, France 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The talk covered our work on software compartmentalisation (NDSS'23, KISV'23, PLOS'23 papers).
Year(s) Of Engagement Activity 2023
URL https://seminaires-dga.inria.fr/en/seances-de-lannee/