AMPS: Analysis of Massively Parallel Stochastic Systems
Lead Research Organisation:
Imperial College London
Department Name: Computing
Abstract
A fast response time and a reliable service are important quality of service criteria for almost all computer and communication systems. Indeed, a system which does not meet its performance and dependability requirements - its system performability - is, in practical terms, as ineffectual as a system that does not meet its correctness requirements.This proposal aims to develop a range of novel techniques and software tools for the performability analysis of massively parallel systems, that is systems which are constructed from large groups of identical interdependent distributed components. The analysis of these systems is a difficult, yet practically important problem in many diverse fields, including the biological sciences - e.g. the dynamics of insect colonies and rate of spread of infections, environmental sciences - e.g. population growth and crowd dynamics, economics - e.g. fluctuations of financial markets as a result of individual trader behaviour, and computer science - e.g. diffusion of computer worms and viruses, dynamic load modelling in computational Grids, peer-to-peer networks, and mobile ad hoc networks.From this wide range of possible application areas, we will focus on massively parallel computer-communication systems. Many such systems, such as distributed mobile publish-subscribe architectures, peer-to-peer filesharing networks and network worm infestations, are having an increasing economic, social and technological impact on our society. Yet, while great strides have been made in our ability to concisely describe these systems and their dynamic behaviour by means of compositional modelling formalisms, much less progress has been made on our ability to analyse these systems quantitatively. This is because almost all traditional performance analysis techniques are based on the idea of (explicitly or implicitly) constructing and solving a Markov chain made up of all possible system behaviours or states. Because the number of states explodes combinatorially with an increasing number of components, the number of states found in a massively parallel system is typically far beyond the feasible limit of direct quantitative analytical study (currently the state of the art is of the order 100 million states). This means that usually the only practical alternative for tackling such models is discrete-event simulation. However, for large models, even long-running simulations often suffer from low state-space coverage, which in turn leads to problems with stability and accuracy of performance metrics (especially where rare-events are not taken into account).An exciting recent development in the performance analysis of such massively parallel systems when represented in stochastic process algebras (such as PEPA), is to use a fluid approximation of the state space. We aim to significantly develop this new paradigm, in collaboration with compositional techniques, by exploring how the fluid approximations can be captured precisely using ordinary and stochastic differential equations (ODEs and SDEs). This gives us the potential to explore the emergent behaviour of a massively parallel system based on the discrete agent description of the underlying components.
Organisations
Publications
Bortolussi L
(2013)
Bounds on the deviation of discrete-time Markov chains from their mean-field model
in Performance Evaluation
Bradley J
(2012)
Resilience Assessment and Evaluation of Computing Systems
Bradley J
(2011)
Invited Response to Computer Journal Lecture by Prof. Jane Hillston
in The Computer Journal
De Jager D
(2010)
Extracting state-based performance metrics using asynchronous iterative techniques
in Performance Evaluation
Guenther M
(2011)
Passage-time computation and aggregation strategies for large semi-Markov processes
in Performance Evaluation
Haggarty O
(2009)
Distributed Response Time Analysis of GSPN Models with MapReduce
in SIMULATION
Hayden R
(2012)
Fluid computation of passage-time distributions in large Markov models
in Theoretical Computer Science
Hayden R
(2010)
A fluid analysis framework for a Markovian process algebra
in Theoretical Computer Science
Hayden R
(2010)
Evaluating fluid semantics for passive stochastic process algebra cooperation
in Performance Evaluation
Hayden R
(2013)
Performance Specification and Evaluation with Unified Stochastic Probes and Fluid Analysis
in IEEE Transactions on Software Engineering
Description | We have discovered how to provide real-time performance metrics for complex massively parallel systems - such as big data systems, security monitoring software, energy-aware systems, autonomous systems and wireless sensor networks. We have written an open-source tool to support our conceptual developments (GPAnalyser). |
Exploitation Route | Use project tools and concepts to develop innovative real-time predictions of capacity and resource requirements for very large systems eg for a web service in a virtualised environment such as twitter. |
Sectors | Construction Digital/Communication/Information Technologies (including Software) Energy Healthcare Manufacturing including Industrial Biotechology |
URL | http://code.google.com/p/gpanalyser/ |
Description | Research translated into a startup company which was used to prevent web crawling to steal valuable datasets. Company was subsequently bought out by Google. |
First Year Of Impact | 2013 |
Sector | Digital/Communication/Information Technologies (including Software),Healthcare,Manufacturing, including Industrial Biotechology |
Impact Types | Economic |