The DiRAC 2.5x Facility

Lead Research Organisation: University of Leicester
Department Name: Physics and Astronomy

Abstract

Physicists across the astronomy, nuclear and particle physics communities are focussed
on understanding how the Universe works at a very fundamental level. The distance scales
with which they work vary by 50 orders of magnitude from the smallest distances probed
by experiments at the Large Hadron Collider, deep within the atomic
nucleus, to the largest scale galaxy clusters discovered out in space. The Science challenges,
however, are linked through questions such as: How did the Universe begin and how is it evolving?
and What are the fundamental constituents and fabric of the Universe and how do they interact?

Progress requires new astronomical observations and experimental data but also
new theoretical insights. Theoretical understanding comes increasingly from large-scale
computations that allow us to confront the consequences of our theories very accurately
with the data or allow us to interrogate the data in detail to extract information that has
impact on our theories. These computations test the fastest computers that we have and
push the boundaries of technology in this sector. They also provide an excellent
environment for training students in state-of-the-art techniques for code optimisation and
data mining and visualisation.

The DiRAC2 HPC facility has been operating since 2012, providing computing resources for theoretical research
in all areas of particle physics, astronomy, cosmology and nuclear physics supported by STFC. It is a highly productive
facility, generating more than 250 papers annually in international, peer-reviewed journals. However, the DiRAC2 hardware is now at least 5 years old and is therefore at significant risk of failure. The loss of any one of the DiRAC2 services
would have a potentially disastrous impact on the research communities which rely on it to deliver their scientific research. The main
purpose of the requested funding for the DiRAC2.5x project is to replace the ageing DiRAC2 hardware at Durham, Edinburgh and Leicester
while taking advantage of recent hardware advances to provide some new capabilities (e.g. i/o acceleration using flash storage) as prototypes for
the proposed DiRAC3 services.

DiRAC2.5x builds on the success of the DiRAC HPC facility and will provide the resources needed to support cutting-edge research
during 2018 in all areas of science supported by STFC. While the funding is required to "keep the lights on", the science programme will continue to be
world-leading. Examples of the projects which will benefit from this investment include:

(i) lattice quantum chromodynamics (QCD) calculations of the properties of fundamental particles from first principles;
(ii) improving the potential of experiments at CERN's Large Hadron Collider for discovery of new physics by increasing the accuracy of theoretical predictions for rare processes involving the fundamental constituents of matter known as quarks;
(iii) simulations of the merger of pairs of black holes which generate gravitational waves such as those recently discovered by the LIGO consortium;
(iv) the most realistic simulations to date of the formation and evolution of galaxies in the Universe;
(v) the accretion of gas onto supermassive black holes, the most efficient means of extracting energy from matter and the engine which drives galaxy formation and evolution;
(vi) new models of our own Milky Way galaxy calibrated using new data from the European Space Agency's GAIA satellite;
(vii) detailed simulations of the interior of the sun and of planetary interiors;
(viii) the formation of stars in clusters - for the first time it will be possible to follow the formation of stars many times more massive than the sun.

Planned Impact

The anticipated impact of the DiRAC2.5x HPC facility aligns closely with the recently published UK Industrial Strategy. As such, many of our key impacts will be driven by
our engagements with industry. Each service provider for DiRAC2.5x has a local industrial strategy to deliver increased levels of industrial returns over the next three years.
The "Pathways to impact" document which is attached to this proposal describes the overall industrial strategy for DiRAC2.5x, including our strategic goals and key performance indicators.

Publications

10 25 50
publication icon
Adamek Julian (2019) Safely smoothing spacetime: backreaction in relativistic cosmological simulations in Classical and Quantum Gravity

publication icon
Ali A (2019) Massive star feedback in clusters: variation of the FUV interstellar radiation field in time and space in Monthly Notices of the Royal Astronomical Society

publication icon
Arthur Jake (2017) nIFTy galaxy cluster simulations - V. Investigation of the cluster infall region in Monthly Notices of the Royal Astronomical Society

publication icon
Ballabio G. (2018) Enforcing dust mass conservation in 3D simulations of tightly coupled grains with the PHANTOM SPH code in Monthly Notices of the Royal Astronomical Society

publication icon
Bate M (2019) The statistical properties of stars and their dependence on metallicity in Monthly Notices of the Royal Astronomical Society

publication icon
Beckmann R. S. (2019) Zooming in on supermassive black holes: how resolving their gas cloud host renders their accretion episodic in Monthly Notices of the Royal Astronomical Society

publication icon
Beckmann R. S. (2018) Bondi or not Bondi: the impact of resolution on accretion and drag force modelling for supermassive black holes in Monthly Notices of the Royal Astronomical Society

publication icon
Bolton James S. (2017) The Sherwood simulation suite: overview and data comparisons with the Lyman a forest at redshifts 2 = z = 5 in Monthly Notices of the Royal Astronomical Society

 
Description Many new discoveries about the formation and evolution of galaxies, star formation, planet formation have been made possible by the award.
Exploitation Route Many international collaborative projects are supported by the HPC resources provided by DiRAC.
Sectors Digital/Communication/Information Technologies (including Software),Education

URL http://www.dirac.ac.uk
 
Description Major co-design project with Hewlett-Packard Enterprise, including partnership in the HPE/Arm/Suse Catalyst UK programme.
First Year Of Impact 2017
Sector Digital/Communication/Information Technologies (including Software)
Impact Types Societal

 
Description DiRAC 2.5x Project Office 2017-2020
Amount £300,000 (GBP)
Organisation Science and Technologies Facilities Council (STFC) 
Sector Academic/University
Country United Kingdom
Start 02/2018 
End 03/2020
 
Title Citation analysys and Impact 
Description Use of IT to determineacademic impact of eInfrastructure 
Type Of Material Improvements to research infrastructure 
Year Produced 2017 
Provided To Others? Yes  
Impact Understood emerging trends in DiRAC Science and helped decide the scale and type of IT investments and direct us to develop new technologies 
URL http://www.dirac.ac.uk
 
Description Co-design project with Hewlett Packard Enterprise 
Organisation Hewlett Packard Enterprise (HPE)
Country United Kingdom 
Sector Private 
PI Contribution Technical support and operations costs for running the hardware. Research workflows to test the system performance, and investment of academic time and software engineering time to optimise code for new hardware. Project will explore suitability of hardware for DiRAC workflows and provide feedback to HPE.
Collaborator Contribution In-kind provision of research computing hardware. Value is commercially confidential.
Impact As this collaboration is about to commence, there are no outcomes to report at this point.
Start Year 2018
 
Description DiRAC 
Organisation Science and Technologies Facilities Council (STFC)
Department Distributed Research Utilising Advanced Computing
Country United Kingdom 
Sector Public 
PI Contribution I am the PI for two research grants for the procurement and running of the Complexity@DiRAC High Performance Computing cluster at the University of Leicester. This cluster is now in active operation as a national HPC facility.
Collaborator Contribution DiRAC is the facility which provides HPC resources for the theoretical astrophysics and particle physics communities within STFC.
Impact The establishment and running of a new HPC cluster at the University of Leicester as part of the DiRAC national facility.
Start Year 2011
 
Description STFC Centres for Doctoral Training in Data Intensive Science 
Organisation University of Leicester
Department STFC DiRAC Complexity Cluster (HPC Facility Leicester)
Country United Kingdom 
Sector Academic/University 
PI Contribution Support for STFC Centres for Doctoral Training (CDT) in Data Intensive Science - DiRAC is a partner in five of the eight of the newly established STFC CDTs, and is actively engaged with them in developing industrial partnerships. DiRAC is also offering placements to CDT students interested in Research Software Engineering roles.
Collaborator Contribution Students to work on interesting technical problems for DiRAC
Impact This is the first year
Start Year 2017
 
Description Surrey-Saclay 
Organisation Saclay Nuclear Research Centre
Country France 
Sector Public 
PI Contribution Provided codes and know-how to develop GF Gorkov formalism and implementation.
Collaborator Contribution Help spreading and advertise my research
Impact Presentation of preliminary results at conference. Grant still ongoing. Results being written up. Output will be first ab-initio calculation of fully open shells.
Start Year 2010