EARL: sdn EnAbled MeasuRement for alL

Lead Research Organisation: Queen Mary University of London
Department Name: Sch of Electronic Eng & Computer Science

Abstract

Internet eXchange Points (IXPs) have become a critical element of the Internet, as they provide the physical locations where networks interconnect and exchange traffic. IXPs carry huge traffic volumes, reduce interconnection costs, and hence make national Internet access affordable. Despite the growth of these infrastructures, the rapid evolution of the Internet poses new challenges.

Reacting as soon as possible to the highly dynamic Internet environment has always been the first priority for Network Operators. Unfortunately, state-of-the-art techniques are extremely limited. Networks use the Border Gateway Protocol (BGP) to inform each other of which destinations are reachable. Accordingly, network operators (ab)use BGP Traffic Engineering (TE) to tweak traffic paths. TE is a network-management tool allowing a network to adapt events ranging from a change in customer location to mitigating dramatically large traffic outbursts of a malicious Distributed Denial of Service (DDoS) attack. However, BGP-TE lacks programmability and dynamism: once BGP preferences are set up, they cannot react in real-time to network events.

With a high-fidelity measurement-focused approach, a network could implement more sophisticated traffic management techniques. For example, any network connected through an IXP must implement ingress traffic filtering to avoid receiving undesirable traffic (e.g., DDoS attacks or resulting from misconfigurations). However, correctly controlling ingress filters is complex. Thus, most IXP customers unrealistically expect the organisations originating the traffic to manage any problem. TE limitations result from the inability of current Internet monitoring techniques to cope with the wide range of granularities of network events. While control plane related events (those concerned with the selection of paths/routes, such as BGP updates) happen at a time scale of minutes, data plane events (packet processing) occur at time-scales of micro-seconds. While control plane monitoring is relatively easy, data plane observability is poor, relies on expensive equipment, and does not scale.

EARL addresses this imbalance between the ability to observe control and data plane, and the consequent limits on the detection and reaction to network events. EARL is a novel integration of monitoring mechanisms and reactive network management. EARL enables a prompt reaction to network events with its Software Defined Networking (SDN) approach. Because of the IXP's central role on the Internet and the critical nature at the national level, we believe that they are the ideal place to explore EARL's ideas. We will demonstrate how measurement-assisted network management permits new Internet-wide services and, enables the provision of services hitherto considered impossible or too costly to deploy. Our goal for the EARL project is to pioneer SDN enabled measurement-based network management to enhance the Internet infrastructure. This will lead to relevant tools and data for the larger researcher and practitioner communities. To this aim, we will create a new research instrument, EARLnet: an operational, research-centered, Autonomous System (AS) directly connected to our partners, providing a new and unique real-world environment for the real-time monitoring of network status and SDN-oriented research. EARLnet will serve also as a test-bed to develop and evaluate novel reactive network management solutions.

The EARL project has the potential to revolutionise current Internet network management through new fine-grained and reactive TE policies. EARL will not only create new mechanisms, but also translate the blind, legacy BGP-based, TE into measurement-assisted SDN techniques. Furthermore, through our partner institution, the Cambridge Cloud Cybercrime Centre (CCCC), EARLnet will provide valuable data to a large community of researchers and practitioners.

Planned Impact

EARL will reach and influence at least three main constituencies: (1) industry, (2) standardization bodies, and (3) the research community.

(1) Impact on industry. EARL will have a clear and positive impact on Internet eXchange Points (IXPs) as well as the networks there connected, which will ultimately benefit the national Internet access. With the UK housing some of the largest and most globally important IXPs, the direct impact through them and the networks connected there will be magnified. The letters of support accompanying this proposal indicate the serious and deep interest in the EARL project from a broad array of companies: LINX is a London based IXP and one of the largest worldwide, providing interconnection to over 700 Autonomous Systems (ASes). ESnet is a high-speed computer network serving the United States Department of Energy (DOE) scientists and their collaborators worldwide. GEANT is the pan-European research and education backbone network, a critical infrastructure interconnecting National Research and Education Networks (NRENs) across Europe. Google is one of the largest content service providers worldwide. As EARL impact will span beyond the providers of the Internet infrastructure, we will also work together with those companies providing the key elements to their operations. In particular, we will work with some of the main industrial providers of network equipment. Corsa is an SDN networking company focused on delivering programmable switches, routers and Layer4-7 capabilities for SDN and Network Function Virtualization (NFV). Cisco Systems is a leading networking company which designs, manufactures, and sells network equipment. These companies, which represent the key elements for SDN-enabled monitoring-assisted innovation, are eager to provide support and benefit from the outputs of our research.

(2) Impact on open standards. The EARL project will significantly contribute to open standards for programmable networking and monitoring. Standardisation efforts in SDN is currently thriving with a broad commercial and academic support. We will take part in standardisation activities and working groups such as the Open Networking Foundation (ONF) and Internet Research Task Force (IRTF) to emphasise the importance of measurement assisted network management in the fabrics of the future. The clear support from the ONF for this project opens the door for a fruitful collaboration towards new open standards.

(3) Impact on research community. Through our partner institution, the Cambridge Cloud Cybercrime Centre (CCCC), EARLnet will provide valuable data to a large community of researchers and practitioners. CCCC provides the ideal vehicle for data distribution with in-place legal frameworks and extensive relevant experience. This collaboration will amplify EARL's impact: the EARL project will provide to the CCCC essential technical data for CCCC to infer malicious security threats and CCCC will utilise its existing frameworks to make the data available to the wider community of researchers and practitioners.

The novelty, timeliness, and relevance of the EARL project will ensure a widespread interest in its results from the international community (i.e., RIPE, NANOG, MENOG). There will also be abundant opportunities for the technology to be exploited by the industrial supporters. Through those supporters, the EARL project will provide significant benefits to the UK software industry, increasing its competitiveness in the growing global Internet marketplace. For the academic partners, the project will lead to high-profile publications and to exciting opportunities for future research. This will further the UK's aspirations for the development and use of SDN-enabled technologies.

Publications

10 25 50
publication icon
Ben Basat R (2020) PINT

publication icon
Bruyere M (2018) Umbrella

publication icon
Böttger T (2018) Looking for hypergiants in peeringDB in ACM SIGCOMM Computer Communication Review

publication icon
Böttger T (2018) Open Connect Everywhere A Glimpse at the Internet Ecosystem through the Lens of the Netflix CDN in ACM SIGCOMM Computer Communication Review

publication icon
Ding D (2020) An Incrementally-Deployable P4-Enabled Architecture for Network-Wide Heavy-Hitter Detection in IEEE Transactions on Network and Service Management

publication icon
Latif S (2020) Leveraging Data Science to Combat COVID-19: A Comprehensive Review in IEEE Transactions on Artificial Intelligence

 
Description Software Defined internet eXchange Points (SDXs) are a promising solution to the long-standing limitations and problems of interdomain routing. While proposed SDX architectures have improved the scalability of the control plane, these solutions have ignored the underlying fabric upon which they should be deployed. This work makes the case for a new fabric architecture that proposes stronger control and data plane separation. Umbrella, the lower-layer part of the EARL solution, provides a more scalable solution to manage the layer-2 interconnection of SDXs.

Also, important progress for scalable network monitoring thanks to Sketch data structures has also been made, thanks to the ElasticSketch work.
Exploitation Route Umbrella has been deployed in TOUSIX, a SDX in France.

ElasticSketch has been used by many other academics to improve scalable data plane monitoring.
Sectors Digital/Communication/Information Technologies (including Software)

 
Description Umbrella, one of the components from the EARL project, has been deployed in a real IXP, TOUSIX, in France, in their production network.
First Year Of Impact 2018
Sector Digital/Communication/Information Technologies (including Software)
 
Title ElasticSketch 
Description When network is undergoing problems such as congestion, scan attack, DDoS attack, etc., measurements are much more important than usual. In this case, traffic characteristics including available bandwidth, packet rate, and flow size distribution vary drastically, significantly degrading the performance of measurements. To address this issue, we propose the Elastic sketch. It is adaptive to currently traffic characteristics. Besides, it is generic to measurement tasks and platforms. We implement the Elastic sketch on six platforms: P4, FPGA, GPU, CPU, multi-core CPU, and OVS, to process six typical measurement tasks. Experimental results and theoretical analysis show that the Elastic sketch can adapt well to traffic characteristics. Compared to the state-of-the-art, the Elastic sketch achieves 44.6 ~ 45.2 times faster speed and 2.0 ~ 273.7 smaller error rate. 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact Multiple further works by the PI (but outside the EARL project) as well as by other researchers has built upon this contribution. Sketches for data monitoring has been highly impacted by this work. 
 
Title Umbrella 
Description Umbrella is a unified software defined development framework that provides a new set of APIs for implementing of SDN applications, keeping the abstractions independent of the Northbound APIs used by specific SDN controllers. The main design goals are: Design and implement a development framework that provides a new set of abstractions for SDN applications, keeping the abstractions independent of the NB APIs used by specific SDN controllers. Design and implement a set of modules that use the proposed abstractions to provide information needed by SDN applications, such as topology, network statistics, and real time topology changes. Increase portability of SDN applications across SDN controllers, and make it easy for a programmer to evaluate a specific application on multiple SDN controllers (e.g., to compare performance). Provide a software defined network programming framework that reduces programming complexity, allows a programmer to write SDN applications without requiring a programmer to master low-level details for each SDN controller, and avoids locking an application to a specific controller. Provide a framework uses a hybrid approach that utilizes both of reactive and proactive approach for managing and programming of SDN networks that offers better scalability than a completely reactive network management. 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact Deployed in production by TOUSIX in France.