A new high-performance low-cost runtime for serverless computing
Lead Research Organisation:
Imperial College London
Department Name: Computing
Abstract
Serverless computing requires efficiently executing untrusted code from many users at once. Suitable technology for implementing such a system does not exist, with current approaches sacrificing one or more of security, efficiency or speed. My work employs novel isolation techniques, high-speed data sharing and new programming abstractions to avoid such compromises. Through this work I hope to enable innovative uses for serverless technology in tackling large-scale scientific problems and machine learning.
Machine Learning and large-scale computing
Machine Learning and large-scale computing
Organisations
People |
ORCID iD |
| Peter Pietzuch (Primary Supervisor) | |
| Simon Shillaker (Student) |
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| EP/N509486/1 | 01/10/2016 | 31/03/2022 | |||
| 1973141 | Studentship | EP/N509486/1 | 01/10/2017 | 31/03/2021 | Simon Shillaker |
| Description | (CloudButton) - Serverless Data Analytics Platform |
| Amount | € 4,277,507 (EUR) |
| Funding ID | 825184 |
| Organisation | European Commission |
| Sector | Public |
| Country | European Union (EU) |
| Start | 01/2019 |
| End | 12/2021 |
| Description | CloudButton (EU Horizon 2020 project) |
| Organisation | Atos |
| Department | Atos Spain |
| Country | Spain |
| Sector | Private |
| PI Contribution | CloudButton is an EU Horizon 2020 project and Imperial is one of three of the academic partners. The team at Imperial is my supervisor and me. I wrote part of Imperial's contribution to the original proposal, contributed artifacts to all the deliverables so far, made presentations at group meetings, and collaborated with others to implement the genomics use-case. |
| Collaborator Contribution | The project is a large 3-year undertaking involving 3 academic institutions, 3 industry partners and four use-case partners. The focus is on producing a usable, complete system for running big data and scientific workloads in a serverless context. Several publications and open source software projects have been produced, detailed of which are on the project website. |
| Impact | As mentioned above, this is a large EU project focused primarily around serverless computing, but bringing in researchers and use-cases from the world of genomics, metabolomics and geospatial data processing. |
| Start Year | 2019 |
| Description | CloudButton (EU Horizon 2020 project) |
| Organisation | European Molecular Biology Laboratory |
| Country | Germany |
| Sector | Academic/University |
| PI Contribution | CloudButton is an EU Horizon 2020 project and Imperial is one of three of the academic partners. The team at Imperial is my supervisor and me. I wrote part of Imperial's contribution to the original proposal, contributed artifacts to all the deliverables so far, made presentations at group meetings, and collaborated with others to implement the genomics use-case. |
| Collaborator Contribution | The project is a large 3-year undertaking involving 3 academic institutions, 3 industry partners and four use-case partners. The focus is on producing a usable, complete system for running big data and scientific workloads in a serverless context. Several publications and open source software projects have been produced, detailed of which are on the project website. |
| Impact | As mentioned above, this is a large EU project focused primarily around serverless computing, but bringing in researchers and use-cases from the world of genomics, metabolomics and geospatial data processing. |
| Start Year | 2019 |
| Description | CloudButton (EU Horizon 2020 project) |
| Organisation | IBM |
| Department | IBM Research Haifa |
| Country | Israel |
| Sector | Private |
| PI Contribution | CloudButton is an EU Horizon 2020 project and Imperial is one of three of the academic partners. The team at Imperial is my supervisor and me. I wrote part of Imperial's contribution to the original proposal, contributed artifacts to all the deliverables so far, made presentations at group meetings, and collaborated with others to implement the genomics use-case. |
| Collaborator Contribution | The project is a large 3-year undertaking involving 3 academic institutions, 3 industry partners and four use-case partners. The focus is on producing a usable, complete system for running big data and scientific workloads in a serverless context. Several publications and open source software projects have been produced, detailed of which are on the project website. |
| Impact | As mentioned above, this is a large EU project focused primarily around serverless computing, but bringing in researchers and use-cases from the world of genomics, metabolomics and geospatial data processing. |
| Start Year | 2019 |
| Description | CloudButton (EU Horizon 2020 project) |
| Organisation | IMT Atlantique |
| Country | France |
| Sector | Academic/University |
| PI Contribution | CloudButton is an EU Horizon 2020 project and Imperial is one of three of the academic partners. The team at Imperial is my supervisor and me. I wrote part of Imperial's contribution to the original proposal, contributed artifacts to all the deliverables so far, made presentations at group meetings, and collaborated with others to implement the genomics use-case. |
| Collaborator Contribution | The project is a large 3-year undertaking involving 3 academic institutions, 3 industry partners and four use-case partners. The focus is on producing a usable, complete system for running big data and scientific workloads in a serverless context. Several publications and open source software projects have been produced, detailed of which are on the project website. |
| Impact | As mentioned above, this is a large EU project focused primarily around serverless computing, but bringing in researchers and use-cases from the world of genomics, metabolomics and geospatial data processing. |
| Start Year | 2019 |
| Description | CloudButton (EU Horizon 2020 project) |
| Organisation | James Hutton Institute |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | CloudButton is an EU Horizon 2020 project and Imperial is one of three of the academic partners. The team at Imperial is my supervisor and me. I wrote part of Imperial's contribution to the original proposal, contributed artifacts to all the deliverables so far, made presentations at group meetings, and collaborated with others to implement the genomics use-case. |
| Collaborator Contribution | The project is a large 3-year undertaking involving 3 academic institutions, 3 industry partners and four use-case partners. The focus is on producing a usable, complete system for running big data and scientific workloads in a serverless context. Several publications and open source software projects have been produced, detailed of which are on the project website. |
| Impact | As mentioned above, this is a large EU project focused primarily around serverless computing, but bringing in researchers and use-cases from the world of genomics, metabolomics and geospatial data processing. |
| Start Year | 2019 |
| Description | CloudButton (EU Horizon 2020 project) |
| Organisation | Red Hat UK |
| Country | United Kingdom |
| Sector | Private |
| PI Contribution | CloudButton is an EU Horizon 2020 project and Imperial is one of three of the academic partners. The team at Imperial is my supervisor and me. I wrote part of Imperial's contribution to the original proposal, contributed artifacts to all the deliverables so far, made presentations at group meetings, and collaborated with others to implement the genomics use-case. |
| Collaborator Contribution | The project is a large 3-year undertaking involving 3 academic institutions, 3 industry partners and four use-case partners. The focus is on producing a usable, complete system for running big data and scientific workloads in a serverless context. Several publications and open source software projects have been produced, detailed of which are on the project website. |
| Impact | As mentioned above, this is a large EU project focused primarily around serverless computing, but bringing in researchers and use-cases from the world of genomics, metabolomics and geospatial data processing. |
| Start Year | 2019 |
| Description | CloudButton (EU Horizon 2020 project) |
| Organisation | Rovira i Virgili University |
| Country | Spain |
| Sector | Academic/University |
| PI Contribution | CloudButton is an EU Horizon 2020 project and Imperial is one of three of the academic partners. The team at Imperial is my supervisor and me. I wrote part of Imperial's contribution to the original proposal, contributed artifacts to all the deliverables so far, made presentations at group meetings, and collaborated with others to implement the genomics use-case. |
| Collaborator Contribution | The project is a large 3-year undertaking involving 3 academic institutions, 3 industry partners and four use-case partners. The focus is on producing a usable, complete system for running big data and scientific workloads in a serverless context. Several publications and open source software projects have been produced, detailed of which are on the project website. |
| Impact | As mentioned above, this is a large EU project focused primarily around serverless computing, but bringing in researchers and use-cases from the world of genomics, metabolomics and geospatial data processing. |
| Start Year | 2019 |
| Title | Faabric |
| Description | Serverless messaging and state middleware |
| Type Of Technology | Software |
| Year Produced | 2020 |
| Open Source License? | Yes |
| Impact | It will form the base of my next publication. |
| URL | https://github.com/faasm/faabric |
| Title | Faasm |
| Description | Faasm is a high-performance stateful serverless runtime. It provides multi-tenant isolation, but also lets functions share regions of memory. These shared memory regions give low-latency concurrent access to data, and are synchronised globally to support large-scale parallelism. |
| Type Of Technology | Software |
| Year Produced | 2018 |
| Open Source License? | Yes |
| Impact | Faasm has been used as part of the EU CloudButton project, providing a high-performance serverless runtime that did not previously exist. |
| URL | https://github.com/lsds/Faasm |