ABC: Adaptive Brokerage for the Cloud

Lead Research Organisation: Lancaster University
Department Name: Computing & Communications

Abstract

The answer to both these questions is usually 'No' even by those familiar with cloud computing. Such uncertainty is caused by 3 main reasons:

1) A bewildering choice of service offerings by the many service providers (e.g. Amazon, Google, Microsoft, etc.).
2) Difficulty in comparison between offered services due to non-uniform description of specifications.
3) High time and monetary costs associated with continuous monitoring of the services on offer from various providers.

This project will introduce more certainty into the selection of cloud services through the introduction of a smart and continuously adaptive cloud broker. The broker will act as an intermediary between end users and cloud service providers in order to enhance service delivery and service value. This brokerage service will be designed to manage heterogeneous cloud offerings, including public, private or hybrid environments. Such brokerage will open up an entirely new multi-cloud marketplace, allowing applications to be simply deployed to the optimal provider and resource type, reducing complexity, vendor lock-in and computational running costs.

Our research will result in a number of fundamental contributions to the cloud computing field. First we will address the problem of how to define, schedule and enforce user-defined Service Level Objectives (SLOs): high-level intentions, which specify the desired end goal of a deployment for applications that span multiple cloud providers with complex inter-dependencies. This will allow users to focus on what (e.g., failure tolerance) needs to be achieved, rather than low-level specifics about how (e.g., deploy to Amazon compute optimised VM) applications are deployed. This automation will in turn help abstract many of the complexities associated with low-level configuration from the user.

Second, we will develop novel lightweight container-based benchmarking techniques, which can gather cloud-level performance metrics in near real-time in a multi-cloud environment. These techniques will be general in scope and allow users to obtain a near real time perspective of the 'weather', or current state across a range of cloud providers.

Third, we will develop adaptive machine learning strategies for the autonomic and pro-active management of cloud-based applications. The application of machine learning will aid decisions about which providers and resource configuration meet the requirements specified in the SLO, how these trade off against cost, and when to redeploy to different providers, or instance types based on active management, etc.

We are confident that this project will address an urgent and fundamental question: how to leverage cloud infrastructure to quickly, cheaply and efficiently perform vital computational workloads. Solving this problem is crucial to the UK digital economy, which is increasingly reliant on the cloud. The developed smart brokerage framework will enable digital economy stakeholders to optimise their use of cloud resources. This is beneficial to all areas of business, including start-ups and micro-businesses who can benefit greatly from the flexibility created by platform independence and adaptive management strategies.

Related Projects

Project Reference Relationship Related To Start End Award Value
EP/R010889/1 01/01/2018 30/03/2021 £388,701
EP/R010889/2 Transfer EP/R010889/1 30/06/2021 31/12/2022 £117,047
 
Description We have carried out a systematic survey of the state of the art on cloud brokerage, identifying the different approaches and techniques used and the shortcomings thereof. This study has helped us build a foundational understanding that will be very important going forwards. We have then designed a basic framework for an adaptive cloud broker. Next, we developed a modelling language to be used for the specification of cloud SLOs (Service Level Objectives). All these findings and other minor ones have been published as papers.
Exploitation Route Based on our investigation and reflection, we have identified a number of future avenues in the field of cloud brokerage, namely: Customer Assistance, Adaptive and Fluid Deployment, and Intelligent Decision Making. These areas need more work by the community, and we clarify exactly how and why in each case.
Sectors Digital/Communication/Information Technologies (including Software)

URL https://doi.org/10.1145/3274657
 
Description ABC: Adaptive Brokerage for the Cloud
Amount £117,047 (GBP)
Funding ID EP/R010889/2 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 06/2021 
End 03/2022
 
Description Extension of the SLO-ML work 
Organisation Deloitte Touche Tohmatsu
Country United States 
Sector Private 
PI Contribution Abdessalam Elhabbash and Yehia Elkhatib have engaged with them for extending the SLO-ML work, with the results to also be released as open source.
Collaborator Contribution They have started working on the source code for the extension.
Impact No outcomes yet.
Start Year 2020
 
Description Portsmouth 
Organisation University of Portsmouth
Country United Kingdom 
Sector Academic/University 
PI Contribution Yehia Elkhatib has visited them for collaboration discussions, and also given a seminar about the results of the ABC project.
Collaborator Contribution They have invited me to give the talk and engaged in discussions about potential collaborations.
Impact We have started a collaborative draft for a follow up research grant.
Start Year 2020
 
Title SLO-ML 
Description SLO-ML extends cloud modelling languages by providing concepts for modelling service level objectives. It is developed as part of the ABC project which focuses on developing an adaptive cloud brokerage framework to act as an intermediary between end users and cloud service providers in order to enhance service delivery and value. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact Conference paper, and follow up work with partners. 
URL https://github.com/AbdessalamElhabbash/SLO-ML
 
Description Workshop at RAL 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Third sector organisations
Results and Impact A number of representatives of RAL at STFC, CEH, and Oxford University met with us to discuss the requirements of scientific application users for cloud brokerage solutions. During the day-long workshop, we explored a wide array of scientific applications and their operational requirements. We also presented our work to date on cloud brokerage, giving an overview of the state of the art and explaining areas of deficiency.
Year(s) Of Engagement Activity 2018