JADE: Joint Academic Data science Endeavour
Lead Research Organisation:
University of Oxford
Department Name: Mathematical Institute
Abstract
This proposal led by the University of Oxford, with support from the Alan Turing Institute (ATI), Bristol, Edinburgh, KCL, QMUL, Sheffield, Southampton and UCL is for a national GPU system that will support multidisciplinary science with a focus on machine learning and molecular dynamics. The architecture is based on ``fat'' GPU compute nodes, with 8 of NVIDIA's new Pascal GPUs, each with
a) 16GB 720GB/s HBM2 memory,
b) an 80GB/s NVlink interconnect to other GPUs,
c) 6GB/s bandwidth to main system memory,
d) 6GB/s bandwidth to the Infiniband external network.
Each server also has two 20-core Xeons, 512 GB DDR4 memory and 8TB SSD.
The motivation for selecting this architecture is the huge growth in research in machine learning and associated areas of data science within the UK, particularly within the universities which are members of the Alan Turing Institute, or SES. The same architecture is also ideally suited for molecular dynamics, medical imaging and a number of other application areas.
The system will be run as a national facility, similar to Archer in being free to all academic users with computing time available to all through a lightweight Resource Allocation Panel, with a top-level steering committee determining the policy on resource allocation between the different application areas (Machine Learning, Molecular Dynamics, Other).
a) 16GB 720GB/s HBM2 memory,
b) an 80GB/s NVlink interconnect to other GPUs,
c) 6GB/s bandwidth to main system memory,
d) 6GB/s bandwidth to the Infiniband external network.
Each server also has two 20-core Xeons, 512 GB DDR4 memory and 8TB SSD.
The motivation for selecting this architecture is the huge growth in research in machine learning and associated areas of data science within the UK, particularly within the universities which are members of the Alan Turing Institute, or SES. The same architecture is also ideally suited for molecular dynamics, medical imaging and a number of other application areas.
The system will be run as a national facility, similar to Archer in being free to all academic users with computing time available to all through a lightweight Resource Allocation Panel, with a top-level steering committee determining the policy on resource allocation between the different application areas (Machine Learning, Molecular Dynamics, Other).
Planned Impact
One outcome from this investment in a national GPU system will be an increased number of researchers leaving universities with excellent GPU computing skills. This will have a significant impact on the UK economy in a number of key market sections:
a) Machine Learning
Many companies such as Google, Microsoft, Baidu and Facebook use GPUs extensively for machine learning applications because they are the most cost-efficient hardware platform for such applications. Within the UK, Google Deepmind, which was originally a UCL spinoff company, is the most important of these, but machine learning is also being used in a wide variety of startups.
b) Computer Vision and Self-Driving Cars
This is a research area which is growing very rapidly and will have a huge impact in the near future. Again there is a mix of companies working the area, ranging from multi-nationals such as Audi to UK startups such as Oxbotica, an Oxford spinoff company.
c) Oil and gas sector
GPUs are used extensively for both seismic inversion (in the exploration phase) and oil reservoir simulation (to maximise the production yield), with over 500 GPUs being used for this purpose within the UK.
d) Pharmaceuticals
There has been some noteworthy use of GPUs in drug discovery, checking the potential of new drugs by comparing their characteristics to other similar chemical compounds.
e) Finance sector
GPUs are used heavily by the big investment banks, with over 2500 in use at various big banks in London.
f) Creative / media companies
These companies, which include film and computer games companies, contribute £2.5bn to the UK economy, and parts of the sector use GPUs extensively for games, and special effects in films and commercials.
a) Machine Learning
Many companies such as Google, Microsoft, Baidu and Facebook use GPUs extensively for machine learning applications because they are the most cost-efficient hardware platform for such applications. Within the UK, Google Deepmind, which was originally a UCL spinoff company, is the most important of these, but machine learning is also being used in a wide variety of startups.
b) Computer Vision and Self-Driving Cars
This is a research area which is growing very rapidly and will have a huge impact in the near future. Again there is a mix of companies working the area, ranging from multi-nationals such as Audi to UK startups such as Oxbotica, an Oxford spinoff company.
c) Oil and gas sector
GPUs are used extensively for both seismic inversion (in the exploration phase) and oil reservoir simulation (to maximise the production yield), with over 500 GPUs being used for this purpose within the UK.
d) Pharmaceuticals
There has been some noteworthy use of GPUs in drug discovery, checking the potential of new drugs by comparing their characteristics to other similar chemical compounds.
e) Finance sector
GPUs are used heavily by the big investment banks, with over 2500 in use at various big banks in London.
f) Creative / media companies
These companies, which include film and computer games companies, contribute £2.5bn to the UK economy, and parts of the sector use GPUs extensively for games, and special effects in films and commercials.
Publications
Gérard SF
(2020)
Structure of the Inhibited State of the Sec Translocon.
in Molecular cell
Wang Q
(2022)
A layer-level multi-scale architecture for lung cancer classification with fluorescence lifetime imaging endomicroscopy
in Neural Computing and Applications
Hernandez-Fernandez M
(2019)
Using GPUs to accelerate computational diffusion MRI: From microstructure estimation to tractography and connectomes.
in NeuroImage
Dämgen M
(2021)
State-dependent protein-lipid interactions of a pentameric ligand-gated ion channel in a neuronal membrane
in PLOS Computational Biology
Salvatori T
(2022)
Reverse Differentiation via Predictive Coding.
in Proceedings of the ... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence
Sha L
(2021)
Multi-type Disentanglement without Adversarial Training
in Proceedings of the AAAI Conference on Artificial Intelligence
Astley J
(2022)
Large-scale investigation of deep learning approaches for ventilated lung segmentation using multi-nuclear hyperpolarized gas MRI
in Scientific Reports
Dämgen MA
(2020)
A Refined Open State of the Glycine Receptor Obtained via Molecular Dynamics Simulations.
in Structure (London, England : 1993)
Epstein M
(2021)
Molecular determinants of binding of non-oxime bispyridinium nerve agent antidote compounds to the adult muscle nAChR.
in Toxicology letters
Reguly I
(2017)
Beyond 16GB
Description | The supercomputer facility JADE has demonstrated three key objectives: 1. The the operational costs of running such a machine can be offset by selling some of the computational capacity to commercial users (so a commercial user would pay to run there computer code on the machine and this would offset the costs of housing the machine, staffing costs to support the machine and the electricity bill associated with it). 2. That there is a clear need and demand for a computer specialised for Machine Learning and Artificial intelligence applications in the UK. This has been demonstrated by the demand for computing time on JADE and the fact that the machine runs at capacity (even after an upgrade that increased the performance by at least 50%). 3. That a partnership between academia (Lead by Oxford) and a commercial entity (Atos) can deliver a world leading computer facility that delivers world leading research. |
Exploitation Route | Our findings demonstrate the feasibility of offsetting the running costs of UK e-infrastructure though a public / private partnerships model. |
Sectors | Digital/Communication/Information Technologies (including Software) Education Energy Government Democracy and Justice |
Description | The Joint Academic Data Science Endeavour (JADE) is the largest GPU facility in the UK supporting world-leading research in machine learning and molecular dynamics research. The JADE consortium, led by the University of Oxford, was awarded £3m of funding by EPSRC in early 2017 to deliver this new resource. It has now been delivered as part of a combined investment of £20m by EPSRC in the UK's regional Tier 2 HPC facilities, bridging the gap between institutional and national resources. The JADE consortium includes some of the UK's world leading groups in machine learning - Oxford University, University of Edinburgh, KCL, QMUL, Sheffield University, UCL and the Alan Turing Institute. In molecular dynamics - University of Bristol and Southampton University. JADE was delivered via a joint venture between Atos and STFC's Hartree Centre to supply, support and operate the system as a managed service to the JADE consortium. JADE achieved impact in many research areas, notably in Machine Learning, Computer Vision, Self-Driving Cars and Pharmaceuticals. Industrial impact was achieved though the sale of computing time to companies such as Boeing, Unilever, Nestlé and JCB, enabling them to access a hardware platform dedicated to Machine Learning along with expertise to support there investigations. |
First Year Of Impact | 2019 |
Sector | Aerospace, Defence and Marine,Creative Economy,Education,Environment,Financial Services, and Management Consultancy,Healthcare,Government, Democracy and Justice,Manufacturing, including Industrial Biotechology,Pharmaceuticals and Medical Biotechnology,Transport |
Impact Types | Cultural Societal Economic Policy & public services |