Scheduler demonstrator for a parallel processor based computer cluster: STFC Global Challenges Concepts Fund

Lead Research Organisation: University of Liverpool
Department Name: Physics


There are increasingly large data sets available to researchers, both created by dedicated experiments and collected from
the internet as web and twitter textual archives. In both cases the rate of increase of data production is growing faster than
the resources to analyse them with standard CPU based computers. New computer models based on GPUs that were
developed for gaming primarily by NVidia and Xeon-phi systems from Intel add co-processor capabilities to computers that
allow much more rapid data processing using the hundreds of dedicated processors on each board.
Using the new computing models provided by these hardware developments means providing a way to use the computers
with these co-processor cards in clusters. By demonstrating the code changes required and requirements for host and coprocessor
resources allows a cluster scheduler to use these computers as part of a mixed cluster. With this scheduler in
place it will be simple to extend existing clusters to use predominately co-processor based systems if the problem is one
that they do accelerate. This will allow scientists and businesses looking to process very large data sets to optimise the
costs of the computing resources they must buy to achieve this.

Planned Impact

The impact of the case would be to allow groups working with big data samples to utilize their limited budgets to optimise
the amount of computing they can perform on their data sets.


10 25 50
Description The possibilities for a mixed cpu model cluster have been investigated.
Exploitation Route The Grid and other large scale cluster users should be able to utilize this research to inform large scale purchasing decisions for future compute clusters.
Sectors Financial Services, and Management Consultancy

Description LIV.DAT Centre of Doctoral Training 
Organisation Liverpool John Moores University
Country United Kingdom 
Sector Academic/University 
PI Contribution We have arranged a partnership with LJMU to run the LIV.DAT centre for doctoral research funded by STFC in part due to the development of parallel processing architectures for particle physics through the grant. Three PhD students are now working on LHCb as a result of this award, which is my primary experiment. We are teaching the PhDs, providing research projects, industrial placements, data intensive research and training.
Collaborator Contribution LJMU have provided similar resources to support students in the CDT and additional subject specific and data intensive research training.
Impact There will be a total of at least 26 PhD students who have completed both PhDs in STFC related science and both six months of training in data intensive research and a six month industrial placement.
Start Year 2017