Scheduler demonstrator for a parallel processor based computer cluster: STFC Global Challenges Concepts Fund

There are increasingly large data sets available to researchers, both created by dedicated experiments and collected from
the internet as web and twitter textual archives. In both cases the rate of increase of data production is growing faster than
the resources to analyse them with standard CPU based computers. New computer models based on GPUs that were
developed for gaming primarily by NVidia and Xeon-phi systems from Intel add co-processor capabilities to computers that
allow much more rapid data processing using the hundreds of dedicated processors on each board.
Using the new computing models provided by these hardware developments means providing a way to use the computers
with these co-processor cards in clusters. By demonstrating the code changes required and requirements for host and coprocessor
resources allows a cluster scheduler to use these computers as part of a mixed cluster. With this scheduler in
place it will be simple to extend existing clusters to use predominately co-processor based systems if the problem is one
that they do accelerate. This will allow scientists and businesses looking to process very large data sets to optimise the
costs of the computing resources they must buy to achieve this.

The impact of the case would be to allow groups working with big data samples to utilize their limited budgets to optimise
the amount of computing they can perform on their data sets.