Square Kilometre Array Project 2016/17

Lead Research Organisation: University of Cambridge
Department Name: Physics

Abstract

This is a request for additional funds for industrial engagement for the SKA Science Data Processor Work led from Cambridge. The objective is to construct a prototype of the hardware and middleware architecture for the SDP. The prototype will be based of course on current architecture but will test a number of critical elements of the design in particular the crucial element of multiple Data Islands each with their own shared file system but no global file system. The prototype will also include prototyping of middleware layers based on OpenStack.

Planned Impact

The SKA is a major new global astronomy facility. It will be operated as an observatory with time awarded via competitive application for observing time. At the same time, the SKA project is a major technical challenge especially within the area of High-performance computing, "Big Data" technologies and energy efficient computing.

The output of the work to be supported by the application will be detailed design and prototyping work for the SKA through to Critical Design review. The delivery of the SKA itself which will follow from this design work will benefit the full academic astronomy community within the partner countries and more widely. In addition the key science goals for the instrument are at the core of modern cosmology and physics and the results will have significant impact beyond the astronomical community both within scientific community more widely but also to Society via our better understanding of the Universe and fundamental physics. We note in particular that SKA phase 1 will provide un-paralled views of the Epoch of deionisation when the first objects form and studies of pulsars within our Galaxy will enable theories of gravity to be tested in extreme conditions and detection of the expected stochastic background of gravitational waves. The UK leadership in key design areas will ensure the UK astronomical community is ideally placed to maximise return from the SKA when operational.

The design work will also have a much wider impact. The SKA is widely recognised as the next BigData challenge within science. As such the required design is pushing the boundaries of those technologies needed to process, analyse and store streaming data. These technologies are critical to the SKA, but are also of wide and general applicability across a broad range of market sectors from standard Data Centres to the analysis and management of data from sensor networks such as those associated with Smart Cities. Via collaboration with major industry partners in the ICT sector and the targeted contracts to be managed as part of this work, the design work to be undertaken will have much wider applicability across the IT market sector. In particular the SKA acts as a single point of focus for many companies both in terms of their own development work, but also as a project for wider collaboration within the sector. In these ways the ICT sector will be a direct beneficiary of the SKA design work. Areas where IP is likely to be developed as part of the design work include new software for file and object stress, new software management layers for Data Centres/HPC facilities especially for BigData challenges, new tools for data management, new analytical approaches for BigData. For these beneficiaries we foresee that the impact may occur in a a number of ways:
1. IP will be developed via this work either jointly or within the industry partners. The IP arrangements in place enable industry partners to exploit this IP while retaining necessary rites for the delivery of the SKA. If the IP is jointly developed appropriate sharing of the results of the impact will be put in place following standard procedures established via Cambridge Enterprise.
2. The SKA design work will provide a focus for some industry partners to develop modified or new products and services which will have wider impact.
3. Direct funding of design work will be undertaken as part of this work. Exploitation of IP derived in this way will be determined by the contractual arrangements with those companies winning competitive tenders while ensuring the delivery of the SKA design according to the SKA IP policy.

Further IP generated wholly within Cambridge will be exploited appropriately via interaction with the University's wholly owned Cambridge Enterprise subsidiary.

Publications

10 25 50
 
Description This award was to build an experimental cluster as a prototype for the SKA data processing system and to use it to experiment on using different approaches to obtaining high performance for advanced data analytics

The key findings of this research are:

a) The design required for a high performance data analytics engine including especially the configuration of the storage and I/O system
b) The development of a high-performance software platform, built on top of and modified from OpenStack, termed scientific openstack
Exploitation Route An independent company, StackHPC is now using these findings in a commercial context
Sectors Digital/Communication/Information Technologies (including Software)

URL http://ska-sdp.org
 
Description This award enabled the development of a test SKA data processing platform. The main findings are now being used by StackHPC an independent company, in a commercial context. In particular the findings on how to design a high performance data analytics engine are finding wide applicability.
First Year Of Impact 2018
Sector Digital/Communication/Information Technologies (including Software)
Impact Types Economic

 
Description SKA SDP Consortium 
Organisation SKA Square Kilometre Array
Department SKA Science Data Processing Consortium
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Cambridge led this consortium with Alexander as Lead, Coles as Project Manager, Nikolic as project Engineer, Ashdown as project scientist and Wortmann as project architect. The consortium delivered the design of the SKA Science Data Processor through to Critical Design Review for the SKA. The SDP is a new high performance data analytics system capable of scaling to at least 250 PFlop performance
Collaborator Contribution Cambridge led this consortium with Alexander as Lead, Coles as Project Manager, Nikolic as project Engineer, Ashdown as project scientist and Wortmann as project architect.
Impact The consortium delivered the design of the SKA Science Data Processor through to Critical Design Review for the SKA. The SDP is a new high performance data analytics system capable of scaling to at least 250 PFlop performance. This is multi disciplinary including physicists, astronomers, computer scientists and software engineers as well as project management and system engineering expertise
Start Year 2013
 
Title SKA SDP Algorithm Reference Library 
Description The software is a reference implementation of the imaging and calibration algorithms required for SKA imaging. It is written in python and will be delivered to the SKA Organisation as part of the deliverables form the design work. The latest version of the library has also significant performance and scalability itself in conjunction with the DASK execution framework / environment 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact This will be the first end-to-end reference library for radio astronomy software. It enables us to engage with industry in considering the next construction phase of the programme providing a reference for critical software elements. 
URL http://ska-sdp.org
 
Title SKA Science Data Processor Architecture 
Description This is a software architecture and associated prototype code for the SKA Science Data Processor. The software implements a novel architecture for a highly scalable data analytics engine able to scale to at least 250 PFlop performance. The design is highly innovative and supports extensibility and maintainability as key drivers as well as performance. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact Although this product has been developed for the SKA it has wide potential applicability to any problem requiring at-scale data analytics 
URL http://ska-sdp.org
 
Title SKALA4 
Description The SKALA is a novel antenna design for low-frequency radio astronomy offering excellent performance over a wide bandwidth from 40-800MHz. The antenna is optimised for low-noise performance and sky coverage. The latest version of the antenna has significant design improvements including a flatter bandpass and better design for mass manufacture 
Type Of Technology Systems, Materials & Instrumental Engineering 
Year Produced 2018 
Impact This antenna design is currently the favoured design of antenna for the SKA low-freqency aperture arrays. In the final SKA approximately 3,000,000 antennas of this sort will be deployed. The antenna will also likely be adopted (or variants of it) for a 
 
Title Scientific OpenStack 
Description The software has been developed as part of the SKA Science Data Processor Platform. It adds new functionality to the OpenStack platform to enable high performance workflows and other monitoring. The software has been added back into the main OpenStack repository for general use. 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact This software has wide applicability outside of radio astronomy and the SKA project for which it is developed and at least one company, StackHPC are taking this forward in a commercial context 
URL http://ska-sdp.org
 
Company Name CAMBRIDGE ELECTROMAGNETIC TECHNOLOGY LTD 
Description This is a spin out company formed by three researchers from University of Cambridge (de Lera Acedo, Alexander and Faulkner) to exploit know-how and IP generated during the development phase of the SKA 
Year Established 2019 
Impact The turnover for the first nine months of the company exceeds £100k