MOA: High Efficiency Deep Learning for Embedded and Mobile Platforms (Full EPSRC Fellowship Submission)

In just a few short years, breakthroughs from the field of deep learning have transformed how computers perform a wide-variety of tasks such as recognizing a face, tracking emotions or monitoring physical activities. Unfortunately, the models and algorithms used by deep learning typically exert severe energy, memory and compute demands on local device resources and this conventionally limits their adoption within mobile and embedded devices. Data perception and understanding tasks powered by deep learning are so fundamental to platforms like phones, wearables and home/industrial sensors, that we must reach a point where current -- and future -- innovations in this area can be simply and efficiently integrated within even such resource constrained systems. This research vector will lead directly to outcomes like: brand new types of sensor-based products in the home/workplace, as well as enabling increasing the intelligence within not only consumer devices, but also in fields like medicine (smart stethoscopes) and anonymous systems (robotics/drones).

The MOA fellowship aims to fund basic research, development and eventual commercialization (through collaborations with a series of industry partners) algorithms that aims to enable general support for deep learning techniques on resource-constrained mobile and embedded devices. Primarily, this requires a radical reduction in the resources (viz. energy, memory and computation) consumed by these computational models -- especially at inference (i.e., execution) time. The proposal seeks will have two main thrusts. First, build upon the existing work of the PI in this area towards achieving this goal which includes: sparse intra-model layer representations (resulting in small models), dynamic forms of compression (models that can be squeezed smaller or bigger as needed), and scheduling partitioned model architectures (splitting models and running parts of them on the processor that suits that model fraction best on certain processors found inside a mobile/embedded device). This thrust will re-examine these methods towards solving key remaining issues that would prevent such techniques from being used within products and as part of common practices. Second, investigate a new set of ambitious directions that seek to increase the utilization of emerging purpose-built small-form-factor hardware processor accelerators designed for deep learning algorithms (these accelerators are suitable for use within phones, wearables and drones). However, like any piece of hardware, it is still limited by how it is programmed - and software toolchains that map deep learning models to the accelerator hardware remain infancy. Our preliminary results show that existing approaches to optimizing deep models, conceived first for conventional processors (e.g., DSPs, GPUs, CPUs), poorly use the new capabilities of these hardware accelerators. We will examine the development of important new approaches that modify the representation and inference algorithms used within deep learning so that they can fully utilize the new hardware capabilities. Directions include: mixed precision models and algorithms, low-data movement representations (that can trade memory operations for compute), and enhanced parallelization.

Expanded discussion upon aspects of how MOA relates to national importance are contained within the Pathways to Impact document. But here summarize in the following paragraphs, the core aspects of importance related to specific EPSRC priority areas, the broader economy, societal impact and contributions to knowledge.
MOA fits within the 'Robotics and artificial intelligence systems' priority area of the UKRI call. It aims to produce technology for enabling machine learning models that otherwise need to run remotely in the cloud, to instead run directly within mobile and embedded systems like robots, drones and small-form-factor devices. As a result, it is an enabling technology useful in the development of new applications or systems. There is direct usage, for example, to enabling healthy/independent living as it would allow a device such as 'Alexa' to be a smarter and better companion to the elderly; or within safety, it can allow a cheap battery-powered workplace or home camera to better understand the semantics of what it captures and react calling the police if it observes dangerous activities. Significantly, the proposed research will likely amplify and extend existing or already on-going machine learning research - this is because it allows a machine learning model that perhaps today must reside in powerful cloud computers to run directly within a drones, robots or devices; this in turn opens up an extend range of new possible application scenarios and use cases for such models.
At a societal level MOA contributes to more ethical, safer and privacy preserving forms of machine learning. Because it enables the use of machine learning directly on limited platforms like phones and devices it reduces the need to transmit and process sensitive data on 3rd party cloud servers not controlled or owned by consumers. Through MOA outcomes, consumers will be able to demand devices (like next-generation medical instruments) that retain their data, yet still offer the benefits of the latest in machine learning.
From the perspective of knowledge, MOA seeks to develop truly innovative concepts in models that can best utilize the latest in commodity and accelerator processor architectures (see workpackage 2). MOA also aims to mature and further invest in studying the latest techniques in a brand-new academic area (efficient deep learning) within a commercial level of quality and rigour (see workpackage 1) that can transfer to offering commercial benefits to companies like Nokia and Samsung that have multiple UK based teams.