EPSRC Centre for Doctoral Training in Machine Learning Systems
Lead Research Organisation:
University of Edinburgh
Department Name: Sch of Informatics
Abstract
Machine Learning (ML) already has a dramatic impact on our daily lives. ML developments in large language models and deep generative models cement that further. The recent explosion in ML, however, is built on the back of improved computer systems able to train and generate ever more powerful models.
Systems design fundamentally defines ML performance and capability. This is true for Internet-scale ML and artificial intelligence (AI). Yet, more recently, it is especially evident in distributed, efficient, device-oriented, secure, personalised, privacy-preserving ML. UK strength in this fast developing area is dependent on a skilled R\&D workforce.
Systems research and ML research are symbiotic. Current innovation in systems research is driven by the ubiquitous need for efficient and reliable ML. ML research, conversely, is steered by deployment capability and the economic and environmental impact of the resulting systems. Furthermore, systems research increasingly relies on ML methods to automate design, and ML research develops such methods.
Major gains are made when the development of ML and systems are co-developed and co-optimized. This is relevant across a broad spectrum of industries: in-car systems, medical devices, mobile phones, sensor networks, condition monitoring systems, high-performance compute and high-frequency trading. Yet PhD training that brings together systems and ML is rare; research training is often siloed in the individual sub-disciplines.
Instead, we need researchers trained in both fields and experienced in working across them. Hence:
The ML Systems CDT will train a new type of student -- the ML-systems researcher.
The ML Systems researcher is critically capable in both fields, and has collaborative research experience across the systems-ML stack.
An example concretises this. A company is developing and deploying wearable body monitors. Effective models must be learnt on collected data, but data must be privacy preserving and bandwidth minimized. This is then personalised to each individual, adaptable to circumstance while being battery efficient and not connection dependent. To manage such a project requires knowledge of effective data-efficient ML signal analysis methods, designed and optimized for low-power hardware, itself tailored for the purpose through ML optimization methods. Knowledge of personalisation methods and the payoffs of privacy preserving methods vitally complement this. The societal impact, e.g.\ on those who might be obsessive about their medical state must also be considered, and will impact development. This CDT will train individuals with cross-cutting capability in all these components.
Students must have broad understanding of different hardware designs, different platforms, different environments, different models, and different goals beyond their immediate research focus. This makes a cohort-based CDT vital. Standard PhD training in ML systems can result in research focus on a single ML technique and a single system. The CDT treats ML Systems as a holistic discipline. Cohort interaction, and integration gives students real experience across multiple systems, approaches and methodologies. Furthermore students will join together to contribute to a unified toolkit for the ML-Systems stack, and make use of others' contributions to that toolkit.
On leaving the CDT, our graduates will understand fully where to focus resources to best improve a company's real-world ML development - whether that be at the ML-algorithm level, the hardware level, the compiler, level or even the legal level. They will be able to evaluate work at every level. We expect our graduates to be the leading team managers in real-world cutting-edge company ML.
Systems design fundamentally defines ML performance and capability. This is true for Internet-scale ML and artificial intelligence (AI). Yet, more recently, it is especially evident in distributed, efficient, device-oriented, secure, personalised, privacy-preserving ML. UK strength in this fast developing area is dependent on a skilled R\&D workforce.
Systems research and ML research are symbiotic. Current innovation in systems research is driven by the ubiquitous need for efficient and reliable ML. ML research, conversely, is steered by deployment capability and the economic and environmental impact of the resulting systems. Furthermore, systems research increasingly relies on ML methods to automate design, and ML research develops such methods.
Major gains are made when the development of ML and systems are co-developed and co-optimized. This is relevant across a broad spectrum of industries: in-car systems, medical devices, mobile phones, sensor networks, condition monitoring systems, high-performance compute and high-frequency trading. Yet PhD training that brings together systems and ML is rare; research training is often siloed in the individual sub-disciplines.
Instead, we need researchers trained in both fields and experienced in working across them. Hence:
The ML Systems CDT will train a new type of student -- the ML-systems researcher.
The ML Systems researcher is critically capable in both fields, and has collaborative research experience across the systems-ML stack.
An example concretises this. A company is developing and deploying wearable body monitors. Effective models must be learnt on collected data, but data must be privacy preserving and bandwidth minimized. This is then personalised to each individual, adaptable to circumstance while being battery efficient and not connection dependent. To manage such a project requires knowledge of effective data-efficient ML signal analysis methods, designed and optimized for low-power hardware, itself tailored for the purpose through ML optimization methods. Knowledge of personalisation methods and the payoffs of privacy preserving methods vitally complement this. The societal impact, e.g.\ on those who might be obsessive about their medical state must also be considered, and will impact development. This CDT will train individuals with cross-cutting capability in all these components.
Students must have broad understanding of different hardware designs, different platforms, different environments, different models, and different goals beyond their immediate research focus. This makes a cohort-based CDT vital. Standard PhD training in ML systems can result in research focus on a single ML technique and a single system. The CDT treats ML Systems as a holistic discipline. Cohort interaction, and integration gives students real experience across multiple systems, approaches and methodologies. Furthermore students will join together to contribute to a unified toolkit for the ML-Systems stack, and make use of others' contributions to that toolkit.
On leaving the CDT, our graduates will understand fully where to focus resources to best improve a company's real-world ML development - whether that be at the ML-algorithm level, the hardware level, the compiler, level or even the legal level. They will be able to evaluate work at every level. We expect our graduates to be the leading team managers in real-world cutting-edge company ML.
