Edinburgh DiRAC Resource Grant

Lead Research Organisation: University of Edinburgh

Department Name: Sch of Physics and Astronomy

Abstract

DiRAC (Distributed Research utilising Advanced Computing) is the integrated supercomputing facility for theoretical modelling and HPC-based research in particle physics, nuclear physics, astronomy and cosmology, areas in which the UK is world-leading. It was funded as a result of investment of £12.32 million, from the Government's Large Facilities Capital Fund, together with investment from STFC and from universities. In 2012, the DiRAC facility was upgraded with a further £15 million capital investment from government (DiRAC-2).

The DiRAC facility provides a variety of computer architectures, matching machine architecture to the algorithm design and requirements of the research problems to be solved. The science facilitated includes: using supercomputers to enable scientists to calculate what theories of the early universe predict and to test them against observations of the present universe; undertaking lattice field theory calculations whose primary aim is to increase the predictive power of the Standard Model of elementary particle interactions through numerical simulation of Quantum Chromodynamics; carrying out state-of-the-art cosmological simulations, including the large-scale distribution of dark matter, the formation of dark matter haloes, the formation and evolution of galaxies and clusters, the physics of the intergalactic medium and the properties of the intracluster gas.

This grant is to support the continued operation of the DiRAC facilities until 2017 to ensure that the UK remains one of the world-leaders of theoretical modelling in particle physics, astronomy and cosmology.

Planned Impact

The high-performance computing applications supported by DiRAC typically involve new algorithms and implementations optimised for high energy efficiency which impose demands on computer architectures that the computing industry has found useful for hardware and system software design and testing.

DiRAC researchers have on-going collaborations with computing companies that maintain this strong connection between the scientific goals of the DiRAC Consortium and the development of new computing technologies that drive the commercial high-performance computing market, with economic benefits to the companies involved and more powerful computing capabilities available to other application areas including many that address socio-economic challenges.

Funded Value:

£1,562,880

Funded Period:

Mar 14 - Sep 17

Funder:

STFC

Project Status:

Closed

Project Category:

Research Grant

Project Reference:

ST/M006530/1

Principal Investigator:

Richard Kenway

Research Subject:

Particle physics - theory (100%)

Research Topic:

Lattice QCD (50%)

Phenomenology (25%)

The Standard Model (25%)

Organisations

People	ORCID iD
Richard Kenway (Principal Investigator)
Peter Boyle (Co-Investigator)

Publications

Author Name Title

Publication Date Published

|< < 1 2 3 4 5 6 7 8 9 10 > >|

10 25 50

Srinivasan S (2021) Cosmological gravity on all scales. Part II. Model independent modified gravity N-body simulations in Journal of Cosmology and Astroparticle Physics

DeGraf C (2020) Cosmological simulations of massive black hole seeds: predictions for next-generation electromagnetic and gravitational wave observations in Monthly Notices of the Royal Astronomical Society

Hands S (2020) Critical behavior in the single flavor Thirring model in 2 + 1 D in Physical Review D

Hands S (2022) Critical behaviour in the single-flavor Planar Thirring Model

Anisman L (2022) Cross-sections for heavy atmospheres: H 2 O continuum in Journal of Quantitative Spectroscopy and Radiative Transfer

Jackson R (2021) Dark matter-deficient dwarf galaxies form via tidal stripping of dark matter in interactions with massive companions in Monthly Notices of the Royal Astronomical Society

Ruiz-Bonilla S (2022) Dealing with density discontinuities in planetary SPH simulations in Monthly Notices of the Royal Astronomical Society

Woss A (2021) Decays of an exotic 1 - + hybrid meson resonance in QCD in Physical Review D

Patsourakos S (2020) Decoding the Pre-Eruptive Magnetic Field Configurations of Coronal Mass Ejections in Space Science Reviews

Fumagalli M (2020) Detecting neutral hydrogen at z ? 3 in large spectroscopic surveys of quasars in Monthly Notices of the Royal Astronomical Society

Campargue A (2020) Detection of electric-quadrupole transitions in water vapour near 5.4 and 2.5 µm. in Physical chemistry chemical physics : PCCP

Ray G (2022) Determination of hybrid charmonium meson masses

Hatton D (2021) Determination of m ¯ b / m ¯ c and m ¯ b from n f = 4 lattice QCD + QED in Physical Review D

Shanahan PE (2015) Determination of the strange nucleon form factors. in Physical review letters

Grand R (2021) Determining the full satellite population of a Milky Way-mass halo in a highly resolved cosmological hydrodynamic simulation in Monthly Notices of the Royal Astronomical Society

Sykes C (2020) Determining the primordial helium abundance and UV background using fluorescent emission in star-free dark matter haloes in Monthly Notices of the Royal Astronomical Society

Gurung-López S (2021) Determining the systemic redshift of Lyman a emitters with neural networks and improving the measured large-scale clustering in Monthly Notices of the Royal Astronomical Society

Reid J (2020) Determining whether the squashing factor, Q , would be a good indicator of reconnection in a resistive MHD experiment devoid of null points in Astronomy & Astrophysics

Abbott R (2020) Direct C P violation and the ? I = 1 / 2 rule in K ? p p decay from the standard model in Physical Review D

MacTaggart D (2021) Direct evidence that twisted flux tube emergence creates solar active regions. in Nature communications

Chambers A (2015) Disconnected contributions to the spin of the nucleon in Physical Review D

Changeat Q (2022) Disentangling atmospheric compositions of K2-18 b with next generation facilities. in Experimental astronomy

Raj A (2021) Disk Tearing: Implications for Black Hole Accretion and AGN Variability in The Astrophysical Journal

Raj A (2021) Disk Tearing: Numerical Investigation of Warped Disk Instability in The Astrophysical Journal

Cheung G (2021) DK I = 0, $$ D\overline{K} $$ I = 0, 1 scattering and the $$ {D}_{s0}^{\ast } $$(2317) from lattice QCD in Journal of High Energy Physics

Gonzalez-Perez V (2020) Do model emission line galaxies live in filaments at z ~ 1? in Monthly Notices of the Royal Astronomical Society

Wurster J (2021) Do we need non-ideal magnetohydrodynamic to model protostellar discs? in Monthly Notices of the Royal Astronomical Society

Boyle P (2015) Domain Wall Charm Physics with Physical Pion Masses: Decay constants, bag and $?$ parameters

Boyle Peter (2016) Domain Wall Charm Physics with Physical Pion Masses: Decay Constants, Bag and $?$ Parameters in PoS

Khamseh A (2016) Domain Wall Charm Physics with Physical Pion Masses: Decay constants, bag and $\xi$ parameters

Boyle P. (2015) Domain wall charm physics with physical pION masses: Decay constants, bag and ? parameters in Proceedings of Science

Hands S (2015) Domain wall fermions for planar physics in Journal of High Energy Physics

Blum T (2016) Domain wall QCD with physical quark masses in Physical Review D

Gronow S (2021) Double detonations of sub-M Ch CO white dwarfs: variations in Type Ia supernovae due to different core and He shell masses in Astronomy & Astrophysics

Collins C (2022) Double detonations: variations in Type Ia supernovae due to different core and He shell masses - II. Synthetic observables in Monthly Notices of the Royal Astronomical Society

Hughes D (2021) Double-diffusive Magnetic Layering in The Astrophysical Journal

Key Findings
Collaboration


Description	In December 2009, the STFC Facility, DiRAC, was established to provide distributed High Performance Computing (HPC) services for theoretical modelling and HPC-based research in particle physics, astronomy and cosmology. DiRAC provides a variety of computer architectures, matching machine architecture to the algorithm design and requirements of the research problems to be solved. This grant funds the continued operation of the 1.3Pflop/s Blue Gene/Q system at the University of Edinburgh, which was co-developed by Peter Boyle (University of Edinburgh) and IBM to run with high energy efficiency for months at a time on a single problem to solve some of the most complex problems in physics, particularly the strong interactions of quarks and gluons. The DiRAC Facility supports over 250 active researchers at 27 UK HEIs. This includes the research projects of 100 funded research staff (PDRAs and Research Fellows), over 50 post-graduate projects, and £1.6M of funded research grants.
Exploitation Route	Theoretical results obtained input to a range of experimental programmes aiming to increase our understanding of Nature. Algorithms and software developed provide input to computer design.
Sectors	Digital/Communication/Information Technologies (including Software)
URL	http://dirac.ac.uk/


Description	Intel IPAG QCD codesign project
Organisation	Intel Corporation
Department	Intel Corporation (Jones Farm)
Country	United States
Sector	Private
PI Contribution	We have collaborated with Intel corporation since 2014 with $720k of total direct funding, starting initially as an Intel parallel computing centre, and expanding to direct close collaboration with Intel Pathfinding and Architecture Group.
Collaborator Contribution	We have performed detailed optimisation of QCD codes (Wilson, Domain Wall, Staggered) on Intel many core architectures. We have investigated the memory system and interconnect performance, particularly on Intel's latest interconnect hardware called Omnipath. We found serious performance issues and worked with Intel to plan a solution and this has been verified and is available as beta software. It will reach general availability in the Intel MPI 2019 release, and allow threaded concurrent communications in MPI for the first time. A joint paper on the resolution to this was written with the Intel MPI team, and the application of the same QCD programming techniques to machine learning gradient reduction was applied in the paper to the Baidu Research all reduce library, demonstrating a 10x gain for this critical step in machine learning in clustered environments. We are also working with Intel verifying future architectures that will deliver the exascale performance in 2021.
Impact	We have performed detailed optimisation of QCD codes (Wilson, Domain Wall, Staggered) on Intel many core architectures. We have investigated the memory system and interconnect performance, particularly on Intel's latest interconnect hardware called Omnipath. We found serious performance issues and worked with Intel to plan a solution and this has been verified and is available as beta software. It will reach general availability in the Intel MPI 2019 release, and allow threaded concurrent communications in MPI for the first time. A joint paper on the resolution to this was written with the Intel MPI team, and the application of the same QCD programming techniques to machine learning gradient reduction was applied in the paper to the Baidu Research all reduce library, demonstrating a 10x gain for this critical step in machine learning in clustered environments. This collaboration has been renewed annually in 2018, 2019, 2020. Two DiRAC RSE's were hired by Intel to work on the Turing collaboration.
Start Year	2016

Abstract

Planned Impact

Organisations

People

ORCID iD

Publications