CONCERT: A Context-Adaptive Content Ecosystem Under Uncertainty

Lead Research Organisation: University College London
Department Name: Electronic and Electrical Engineering

Abstract

The CONCERT objective is to develop a content ecosystem, encompassing all relevant players, which will be able to perform intelligent content and network adaptation in highly dynamic conditions under uncertainty. This ecosystem will have as basis emerging information-/content-centric networking technologies which support intrinsic in-network content manipulation. The project will consider uncertainty aspects in the following two application domains: a) social media networks based on user generated content and b) CDN-like professional content distribution. Three dimensions of uncertainties will be addressed: heterogeneous and changing service requirements by end users, threats that may have adverse impacts on the content ecosystem, as well as opportunities that can be exploited by specific players in order to have their costs reduced.

In order to manage and exploit these uncertainty aspects, CONCERT defines a two-dimensional content and network adaptation framework that operates both cross-layer and cross-player. First, the decision on any single adaptation action needs to take into account context information from both the content application layer and the underlying network. Second, we consider joint content and network adaptation in order to simultaneously achieve optimised service performance and network resource utilisation. Finally, some complex uncertainty scenarios require coordinated content and network adaptation across different ecosystem players. In this case, inconsistent or even conflicting adaptation objectives and different levels of context knowledge need to be reconciled and are key research issues.

In order to achieve adaptation solutions capable of coping with these different uncertainties, the project will develop advanced learning, decision-making and negotiation techniques. Learning is required for deriving accurate system behavioural patterns according to the acquired context knowledge. This will then drive decision-making functions for taking the most appropriate adaptation actions to address these uncertainties. Negotiation techniques are required for resolving potential tussles between specific content/network adaptation objectives by different players in the content ecosystem. The project will consider both centralised and distributed approaches in which learning and decision-making processes on adaptation actions can be performed either at the central adaptation domain controller or in a decentralised manner across multiple network elements. In the latter case, emerging information-/content-centric networks will become much more intelligent, with content-aware devices performing self-adaptation according to their own context knowledge but through coordination in order to achieve global near-optimality and stability.

Planned Impact

CONCERT addresses the next generation of intelligent and adaptive information-/content-centric networks. We envisage empowering it with cross-layer and cross-domain adaptation capabilities so that they can deal with uncertainties, anticipate potential problems, maximise user QoE and optimise resource usage while reducing operational costs. The envisaged content ecosystem will also provide the opportunity to both prosumers and network operators to play an active role in the future content market. The project is expected to lead to significant progress beyond the current state-of-the-art in a number of scientific areas as presented next.

a) Intelligent information- and content-centric networks. Such emerging networks exhibit limited intelligence apart from native in-network content resolution and content-based anycast routing. We intend to combine these with learning and self-adaptation mechanisms so that they are able to deal with uncertainties while always maintaining optimal resource utilisation and service performance. Both distributed and centralised decision making techniques utilising contextual information will be considered, making these networks truly adaptive and intelligent.

b) Holistic network control for multiple uncertainty dimensions. In current networks, traffic, device, energy, content popularity and other uncertainty aspects are dealt with through separate approaches/techniques, which can hardly co-exist in an optimal manner. We intend to provide a systematic approach for network control that involves training/learning and sophisticated decision making exploiting user, traffic, and network behavioural patterns. This novel approach cuts across all the players participating in the content ecosystem, maintaining optimal performance and maximising user QoE.

c) Combined cross-domain content adaptation. In current networks, adaptation is performed in the application layer or in the network but in a non-coordinated fashion. Here we envisage a two-dimensional adaptation framework in which intelligent decision-making will choose where to perform adaptation "vertically", in the application layer or in the network, and "horizontally", across network devices or even across different player domains, e.g., at the prosumers' site. In addition, server selection and fetching from router caches will be combined with content adaptation decisions, hence maintaining optimal media streaming quality.

The commercial/market aspects of the proposed content ecosystem will be manifold. The current content market is oriented towards CDNs and is mostly dominated by large US companies. The advent of the proposed information-centric context-adaptive content ecosystem will unlock its potential and redefine associated business models, favouring SMEs or even individual prosumers. The emerging intelligent context-adaptive information-centric network will effectively behave as a very large CDN, enabling small content providers with innovative ideas to enter the market without the large costs incurred by the current CDN providers. The latter will have to reinvent themselves, offering added-value CDN services by exploiting the networks' increased capabilities. Cloud service providers will also benefit, offering new enhanced services.
 
Description 1) We designed a theoretical framework for modelling path-based information dissemination, as opposed to the conventional contact-based infection in classical epidemic models, whereby "information" can both be desirable objects, e.g., data, content, news updates, context information etc., and undesirable ones, e.g., Internet viruses / malware, gossip etc. The model extends the state-of-the-art in epidemic theory and describes the path-based contagion dynamics in a generic manner that facilitates application to various contexts; the latter are elaborated as use cases in the project.

From our analysis, we found that the information spreading capacity is dependent on (i) the path construction, which in the context of the Internet relates to routing protocols, and (ii) the traffic intensity, i.e., the network traffic volume. We discovered a critical threshold below which spreading will die off and which can be computed by finding the spectral radius (or the largest eigenvalue of) what we call an "infection characterisation matrix" (IEEE/ACM Transactions on Networking 2017 journal paper).

We subsequently investigated the modelling of the spreading process induced by agent flows in the CONCERT ecosystem. The agents traverse from sources to destinations following specific paths and infecting the sites they traverse with specific probability. In our investigation, we found that the spreading radius is wider when compared to the conventional contact-based spreading model. Another interesting finding is that, in contrast to the contact-based model where nodes located more centrally in a network are proportionally more prone to infection, the agent assisted spreading model shows no such strict correlation, i.e. nodes may not be highly susceptible even located at the heart of the network and vice versa. (IEEE Transactions on Network Science and Engineering 2018 journal paper).

2) In the context of large-scale Internet-wide information propagation and maintenance, we studied the scalability and feasibility of bloom filter based inter-domain name resolution schemes based on ICN principles. Scalability of global name resolution is an open issue and bloom filters have been considered as a potential solution to the scalability concerns that stem from the enormous size of the information namespace. Our findings show that the skewed distribution of the name resolution state across the inter-domain Internet topology renders the selection of a specific bloom filter configuration particularly problematic, since domains are heterogeneous and can have very different requirements and capabilities. We concluded that bloom filters alone do not provide a sufficient answer to this problem (ACM ICN'2015 conference paper).

3) We also investigated specific issues concerning content spreading in networks with uncertainties. We focused on opportunistic device-to-device networks with are prone to perturbations (e.g., disconnections, limited communication capabilities, etc.) and investigated a content dissemination approach based on ICN principles. Our approach complements ICN with fountain coding to exploit the benefits of multi-source multi-path forwarding. From our evaluations, we showed that our proposal can minimise redundancy, and the resulting waste of network resources, without requiring any centralised coordination mechanism (Elsevier Computer Communications 2017 journal paper).

4) We also addressed the problem of dynamic workload (as one of the uncertainties considered in CONCERT) in the context of the emerging mobile multimedia applications which have stringent latency requirements as well as high computational cost (e.g., augmented reality applications). We investigated an approach to bring resources closer to users by exploiting network function virtualization (NFV) and mobile edge-cloud (MEC). We take a longitudinal view and investigate not only how service-hosting nodes should be instantiated but also when this should happen while explicitly taking into account the added help provided by standard tools such as auto-scaling and load balancing mechanisms. According to our performance evaluation, our solution always manages to allocate sufficient resources on time to guarantee continuous satisfaction of the application latency requirements under changing workload while incurring up to 40% less cost in comparison to existing over-provisioning approaches (IEEE CloudNet'2016 conference paper).

We have then extended the above work with a holistic dynamic resource allocation framework that consists of a fast heuristic-based incremental allocation mechanism that dynamically performs resource allocation and a re-optimization algorithm that periodically adjusts allocation to maintain a near-optimal MEC operational cost over time for supporting low latency mobiles services. We show through extensive simulations that our flexible framework always manages to allocate sufficient resources in time to guarantee continuous satisfaction of applications' low latency requirements. (IEEE Transactions on Network and Service Management 2018 journal paper).

We have further considered the fault tolerance of stateful virtualised network functions (VNFs) within the concept of NFV. VNFs are vulnerable to various faults such as software and hardware failures. We study the fault-tolerant VNF placement problem with the optimisation objective of admitting as many requests as possible. In particular, the VNF placement of active/stand-by instances, the request routing paths to active instances, and state transfer paths to stand-by instances are jointly considered. We devise an efficient heuristic algorithm to solve this problem, and propose a bi-criteria approximation algorithm with performance guarantees for a special case of the problem. Simulations with realistic settings show that our algorithms can significantly improve the request admission rate compared to conventional approaches (IEEE ICC 2018 conference paper).

Following this theme, we propose a set of practical, uncoordinated strategies for service placement in edge-clouds. Through extensive simulations using both synthetic and real-world trace data, we demonstrate that uncoordinated strategies can perform comparatively well with the optimal placement solution, which satisfies the maximum amount of user requests (IEEE Cloudcom 2017 conference paper).

5) Another area addressed relates to resilience in networks under uncertainties. We investigated cascading failures in the power grid domain where the operation of the power grid is interdependent with the communication network and a failure in one may cause disruption the other and vice versa. This then forms an iterative failure pattern that causes catastrophic impact to the interdependent network. We used real grid data to quantify the precise impact of such failures (IFIP Networking 2016 conference paper).

6) Related to our studies on information spreading (see point (1) above), we have also investigated the problem of distributed resilient caching. We study the problem of optimally allocating content over a resilient caching network, in which each cache may fail under some situations. Given content request rates and multiple routing paths, we formulate an optimization problem to maximize the expected caching gain, i.e., the reduction of latency due to intermediate caching. The offline version of this problem is NP-hard. We first propose a centralised, offline algorithm and show that a solution with (1-1/e) approximation ratio to the optimal can be constructed. We then propose a distributed ascent algorithm based on the concave relaxation of the expected gain. Informed by the results of our analysis, we finally propose a distributed resilient caching algorithm (DR-Cache) that is simple and adaptive to network failures. We show numerically that DR-Cache significantly outperforms other candidate algorithms under synthetic requests, as well as real world traces over a class of network topologies (IEEE INFOCOM 2018 conference paper).
Exploitation Route Our work ranges from theoretical development to more practical engineering solutions. Our framework for modelling path-based information spreading has the potential to be applied to wide ranging areas and scenarios. In CONCERT in particular, this work is being used for realising an adaptive context-aware content network environment with emphasis on the Internet and online social networks. Its applicability encompasses Internet ISPs, mobile network operators, social network providers and content providers, who should all achieve economies of scale. In addition, end users should get better "quality of experience".

Our study of bloom filter based Internet-wide information propagation and maintenance has already been used as a "road sign" in the IRTF ICNRG, stating that this path, although intuitively promising, is inadequate and should be treated with care. For the potential future evolution of the Internet towards Information Centric Networking (ICN), its scalability is crucial and our work represents a step forward in this direction.

The impending arrival of 5G and beyond networks which promise ever-higher data rates and shorter latencies has fostered the emergence of novel applications, such as virtual / augmented reality, healthcare, autonomous vehicles, etc., which require rapid and timely response to information requests regardless of device capabilities and network conditions. Our work on mobile edge cloud addresses exactly this challenge, proposing a framework for optimally allocating resources to different network locations in order to satisfy mobile users who are running applications that are highly resource-hungry and also have stringent response time requirements. Since 2020 the project team started to extend the implemented platform to further support the immersive Holographic Teleportation applications beyond the conventional MPEG video content. It is expected that the extended framework will enable Internet-scale teleportation of live hologram objects with end-to-end user Quality of Experience assurance against dynamic and uncertain network and traffic conditions.
Sectors Chemicals,Digital/Communication/Information Technologies (including Software),Energy,Transport

URL http://www.concert-project.org/
 
Description We have designed a theoretical framework for modelling path-based information dissemination as opposed to the conventional contact-based infection in classical epidemic models, whereby "information" can both be desirable objects, e.g., data, content, news updates, context information etc., and undesirable ones, e.g., Internet viruses / malware, gossip etc. The model extends the state-of-the-art in epidemic theory and describes the path-based contagion dynamics in a generic manner that facilitates application to various contexts; the latter were elaborated as use cases in the project. Our framework is generic in nature and, as such, it is applicable to various network types in different scientific disciplines. Therefore, we see potentially a very wide spectrum of applications. In CONCERT, this framework is being used in the context of communication networks that include the Internet, online social networks and mobile wireless networks. However, it can be also used in totally different contexts. For example, we have examined its applicability to energy power grids and also to transport networks. In addition, we have conducted a feasibility study on Internet-wide information propagation and maintenance. In the context of information-centric networking (ICN), we analysed the scalability of inter-domain name resolution schemes using bloom-filters which have the potential of significant reduction of state maintenance requirements due to their property of enabling compact state representation. We found that there is no specific bloom filter configuration that can satisfy the requirements and capabilities of various autonomous systems. This is mainly due to the highly skewed distribution of the name resolution states. Our study on bloom-filter-based Internet-wide information propagation and maintenance has already been used as a "road sign" in the IRTF ICNRG that such path, although intuitively promising, is inadequate and should to be treated with care. This research work in the ICN area has led to an industrial project on scalable name-based routing in Hybrid ICN which started in 2020. With the global increased use of communication devices (smart phones, tablets, sensors etc.), device-to-device communication is becoming common but at the same time it faces many challenges. In this context, we have tackled the issue of efficient content dissemination by exploiting a combination of ICN and fountain-coding concepts. Finally, existing 5G and beyond networks(i.e. 6G) provide ever-increasing data rates and shorter latencies and these have fostered the emergence of novel applications which require rapid and timely response to information requests regardless of device capabilities and network conditions e.g. virtual / augmented reality, healthcare, autonomous vehicles, etc. Our work on mobile edge cloud addresses exactly this challenge, proposing a framework for optimally allocating resources to different network locations in order to satisfy mobile users who are running applications that are highly resource-hungry and also have stringent response time requirements. Since 2020 the project team started to extend the implemented platform to further support the immersive Holographic Teleportation applications beyond the conventional MPEG video content. It is expected that the extended framework will enable Internet-scale teleportation of live hologram objects with end-to-end user Quality of Experience assurance against dynamic and uncertain network and traffic conditions.
First Year Of Impact 2015
Sector Digital/Communication/Information Technologies (including Software),Energy,Transport
Impact Types Societal,Economic

 
Description ICN2020: Advancing ICN Towards Real-World Deployment Through Research, Innovative Applications, And Global Scale Experimentation
Amount € 1,300,000 (EUR)
Funding ID 723014 
Organisation European Commission 
Department Horizon 2020
Sector Public
Country European Union (EU)
Start 07/2016 
End 06/2019
 
Description INTENT: Information-Centric Network Management and Traffic Engineering
Amount € 221,606 (EUR)
Funding ID 628360 
Organisation European Commission 
Department Seventh Framework Programme (FP7)
Sector Public
Country European Union (EU)
Start 01/2015 
End 12/2016