Software development support for DiRAC

We propose to partially mitigate the shortfall in projected DiRAC computing resources over the next year, arising from the delay to DiRAC-3, by continuing a code optimisation effort to accelerate DiRAC's scientific codes on current architectures, and which will also give a sustained long term benefit to scientific throughput on future machines. The effort will be used to provide profiling, benchmarking and optimisation services to maximise scientific return from the DiRAC by ensuring the most efficient use of computing resources possible. This represents continuation of a successful first year of such effort supported by the DiRAC Technical Working Group grant and with additional support from the University of Edinburgh. The first year of effort focused on the representation of the DiRAC workload as benchmarks to assess future supercomputing needs, and there have been significant benefits to the community arising from this activity. These included discovering vectorisation, caching optimisation, and MPI parallelisation strategies for codes of the VIRGO, UKQCD, UKMHD and Planck cosmology collaborations as examples.

DiRAC contains some areas of key international strength in high performance software engineering. DiRAC's expertise has been the foundation upon which the Alan Turing Institute has secured an internationally unique strategic partnership with Intel's HPC architecture group. There is an good opportunity for STFC to enable the rest of the DiRAC community to exploit the knowledge base.

The proposed work will disseminate best practice in High Performance Computing software engineering throughout the theoretical Particle Physics, Astronomy and Nuclear physics communities in the UK. This will encourage the development of skills in the research community that are highly transferrable to the computations required for simulations in advanced manufacturing and computer aided design. Some level of the young researchers in the community leave academia each year as a natural part of the turnover that occurs in university sector research, especially given the limited number of research positions. The transfer of a portion of the research communities to other jobs carrying these skills is a key element that feeds UK industry from the space and satellite sector, through hard engineering and the financial sectors.

The DIRAC project has close ties with the highest levels of research and development in the computing industry. We designed a key component on the IBM BlueGene/Q system, as part of a unique Academic-Industrial joint project with IBM Watson Laboratory. The design has powered leading scientific computing installations around the world, from laboratories in the USA, Japan, Italy and Germany to our own national Hartree centre.

More recently DiRAC has been competitively awarded three Intel Parallel Computing Centres, won several international supercomputing awards, and developed a close codesign project with Intel on future HPC architectures. This latter effort promises the ability to influence a vast swathe of modern computing over the next five years. Such improvements in computing would impact all consumers of computional hardware, and in particular those doing numerical simulation such as the advanced manufacturing sector (e.g. Rolls Royce, BAE, Mclaren, Shell, Jaguar Land Rover), in addition to much of academic research in the physical sciences.

The Intel-Alan Turing Institute strategic partnership, with an HPC architecture team embedded in the ATI is built on this foundation of a deep collaboration with theoretical particle physicists. DiRAC's technical director is the codesign leader for the ATI and will transfer DiRAC's best practice codesign techniques to a number of other subject domains covering HPC and Data Science.

The Alan Turing Insitute, with DiRAC's codesign leader playing a central role is actively engaging with external partners, such as Intel, the MET office, Shell, and even Mclaren Racing, propagating vectorisation techniques developed in QCD codes to the important area of Finite Elements Modelling.

The present proposal provides resources for the theoretical particle physics, astronomy and nuclear physics communities to interact with the codesign knowledge centre. The opportunity to influence with sensible engineering decisions that optimise codes for products and products for codes is real. One example is a recurring element of computer architecture involves the tradeoff between throughput and accuracy of vectorised reciprocal square root instructions. These are a key element of the inverse square law that dominates the gravitational element of astronomy simulations. We have the opportunity to give definitive statements about the right balance to Intel.

Similarly we hope to provide useful information on the requirements for data motion, from the complexities of cache organisation and algorithms to interconnect requirements. Nowhere is this more pressing than addressing the enormous challenges presented by the Square Kilometre Array, perhaps a leading Big Data problem in the near future of scientific endeavour.

Our work can also lead to cross fertilisation into the nascent fields of machine learning and data science through the Alan Turing Institute.