EDGE - Adaptive Deep Learning Hardware for Embedded Platforms

Deep learning (DL) is the key technique in modern artificial intelligence (AI), which has provided state-of-the-art accuracy on many machine-learning based applications. Today, although most of the computational loads of DL systems are still spent running neural networks in data centres, the ubiquity of smartphones, and the upcoming availability of self-contained wearable devices for augmented reality (AR), virtual reality (VR) and autonomous robot systems are placing heavy demands on DL-inference hardware with high energy and computing efficiencies along with rapid development of DL techniques. Recently, we have witnessed a distinct evolution in the types of DL architecture, with more sophisticated network architectures proposed to improve edge AI inference. This includes dynamic network architectures that change with each new input in a data-dependent way, where inputs and internal states are not fixed. Such new architectural concepts in DL are likely to affect the type of hardware architectures that will be required to deliver such capabilities in the future. This project precisely addresses this challenge and proposes to design a flexible hardware architecture that enables adaptive support for a variety of DL algorithms on embedded devices. Primarily, to produce lower cost, lower power and higher processing efficiency DL-inference hardware that can be configured adaptably for dedicated application specifications and operating environments, this will require radical innovation in the optimisation of both the software and the hardware of current DL techniques.

This work aims to perform fundamental research, development and practical demonstrator to enable general support for a variety of DL techniques on embedded edge devices with limited resource and latency budgets. Primarily, this requires radical innovation on the current DL architectures in terms of computing architecture, memory hierarchy and resource utilisation, as well as system latency and throughput: it is particularly important for the modern DL systems that the inference processes are dynamic, such as, the DL inference maybe input-dependent and resource-dependent. The proposal therefore seeks the following three thrusts: First, to build upon the existing work of the PI in optimising machine-learning models for resource-constrained embedded devices, towards achieving the goal that the network model could be dynamically optimised as needed through hardware-aware approximation techniques. Second, with newly-developed adaptive compute acceleration technology in programmable memory hierarchy and adaptive processing hardware, to seek a new ambitious direction to develop a set of context-aware hardware architectures to work closely with the approximation algorithms that can fully utilise the true hardware capabilities. Unlike traditional optimisation techniques for DL hardware inference engines, the proposed work will explore both software and hardware programmability of adaptive compute acceleration technology, in order to maximise the optimisation results for the target application scenarios. Third, this project will work closely with our industry and project partners to produce a practical demonstrator to showcase the effectiveness of the proposed DL framework versus traditional approaches, particularly, evaluating the effectiveness of the framework in real-world mission-critical applications.