Exploration of machine learning for pre-emptive scheduling in single-instruction multiple-data mega-kernel designs

This project is about exploring the question if we can use machine learning to make parallel computing units like graphics processing units (GPUs) more efficient. GPUs have become powerful parallel processors, but using this power is often difficult. Currently, GPUs are mostly used for algorithms that are easy to parallelize. However, I believe that it is possible for more algorithms to benefit from the power of GPUs, if we offer new ways for GPU programming and more intelligent task scheduling strategies.

In the past I have been researching novel frameworks to allow the parallelisation of computational methods that are usually hard to translate for an execution of parallel hardware like GPUs. For that I used a parallel programming concept called Mega-Kernel. The essence of this concept is that a few computing units out of several thousand are responsible for scheduling computational tasks with different priorities. The Mega-Kernel concept has been demonstrated to provide a powerful extension of conventional kernel based program execution management that can deliver significant performance enhancement from single instruction multiple data (SIMD) approaches. However, to date the queue optimisation capabilities that are at the core of the approach use static rule based decision processes and in particular do not provide optimal hardware utilization or automatic intelligent preemptive scheduling. There are many real world applications for which performance could potentially be transformed by more dynamically adaptive scheduling capabilities and the SIMD architecture itself provides an opportunity to realise this. In this project we will explore statistical machine learning at the parallel hardware level to automatically predict the priorities of tasks in complex real world data analysis tasks such as the reconstruction and motion correction of n-dimensional motion corrupted medical image data. In particular, we will explore the real-time capabilities of machine learning supported preemptively scheduled reconstruction for the direct integration of motion correction into the scan process of fetal magnetic resonance data. Motion correction is the only way to provide comprehensive investigation of all fetal organs at high resolution. However, the currently used algorithmic pipeline is unidirectional, slow and consists of error-prone post-scan-processing steps. The lack of interactivity makes manual corrections or an integration into the scan process impossible at the moment. Automatically prioritised preemptive scheduling of the algorithm's tasks on commodity hardware like GPUs will likely provide a way to introduce real-time capabilities for such extremely complex computing methods.

It will be feasible during the 14 months of this project to explore if machine learning could be an option to predict the hardware utilisation of the tasks of a complex example algorithm like motion correction with potential corrective user input. If successful, the results of this project are likely to introduce a new paradigm in high-performance computing and will contribute to a paradigm-shift in medical image acquisition of moving objects.

This project focuses on developing new software technologies for high-performance computing (HPC) with applications to medical image analysis.
The most immediate impact is expected through uptake of the developed fundamental methods by the HPC community and by researchers in medical image analysis. The PI will advance this uptake through direct collaboration with experts in HPC (Imperial College, MPI Saarbrücken) and medical image acquisition (King's College London) and is currently exploring the potential for collaboration with other research groups (academic visits for example to Vienna University of Technology, Austria, the HPC facility in Nottingham, UK and the University Hospital Leipzig, Germany).

Dynamic scheduling for GPUs provides a great number of possibilities. Exploring the integration of machine learning into these techniques can pave the way for new parallel programming models, following the idea that algorithms can be written more easily by defining work entities and their relations rather than execution steps. With the possibility to control execution states at runtime, we can provide an intelligent task manager for GPUs. We will also explore the application areas that can be tackled with advanced GPU scheduling strategies, including motion correction in n-dimensional magnetic resonance imaging (MRI), which includes research on computational intelligence and real-time scheduling for time critical processes.

At the same time we will work on the remaining features that hamper the straight forward mapping of complex algorithms for GPU execution. We will enable learned task priorities that completely hide task prioritisation from the programmer, allowing an arbitrary small granularity of priority levels. With the combination of all these efforts, we will make GPUs a more accessible device for the development of complex parallel algorithms.

The chosen example application will contribute significantly towards the development of better and more objective imaging biomarkers in medical imaging. Such markers are crucial for early or differential diagnosis. In addition they allow clinicians to reach informed decisions and viewpoints on potential treatments and therapies. Therefore, clinicians in medical imaging and fetal medicine will be also direct beneficiaries of this research.

Our novel fast and interactive motion correction methods will likely lead to novel work flows for screening, diagnostics and image guided intervention in the prenatal clinical practice. Together with our clinical partners we will work out new work flows that base on the unleashed power of intelligently scheduled parallel algorithms. In the long run our methods are generalizable and can be transferred to other hard to parallelise algorithms that require interactive prioritised input.

It is also very likely that the medical imaging technology industry and GPU technology industry will be highly interested in the proposed research. I will therefore keep my existing connections to these industries alive and discuss research results with them and potential new contacts permanently during the project. IP arising from this will be protected for commercialization (e.g. via patents) and exploited together with Imperial Innovations (which coordinates the activities of technology transfer, company incubation and investment for Imperial College London).

This project will also have an indirect impact on the next generation of computer scientists. Students at Imperial will indirectly benefit from the project by being close to most recent state of the art in HPC and parallel algorithm development. The PI is a dedicated lecturer and is keen to pass on his knowledge about these techniques to the next generation. Expertise about GPU programming and HPC is an important feature of this project. Parallel algorithm development requires special training and most Computer Science curricula take this into account by now.