Unlocking the Potential of Neural Architecture Search (UPNAS)
Lead Research Organisation:
University of Edinburgh
Department Name: Sch of Engineering
Abstract
Deep neural networks demonstrate strong performance on a range of challenging tasks such as segmenting objects in complex scenes, synthesising speech, and modelling protein folding. However, each task requires its own specialised neural architecture to excel. These architectures are almost always manually designed. This design is a costly process requiring the time and effort of a human expert, and has become increasingly more difficult as networks have got larger, and more complicated. This is a barrier to entry for deploying deep learning for novel applications that must be overcome.
Neural architecture search algorithms have the potential to remove this barrier; they automatically search for an appropriate neural architecture for a particular task. However, they are not without issues: (1) they are very expensive to run; (2) they search within a narrow space of architectures; (3) we don't know how robust they are. This project aims to unlock the potential of neural architecture search by tackling these issues.
Neural architecture search is slow and expensive because search algorithms rely on having to train architectures to determine how effective they are, to inform the search process. We will build upon our recent work to bypass this training by developing inexpensive proxy measures, and create search algorithms that are able to leverage these proxies to perform neural architecture search efficiently.
Search spaces are narrow because they only draw from a single family of network architectures, and use neural operations that aren't optimised for a target device, instead selecting from a predefined list. We will create search spaces that are able to incorporate multiple families of architectures and use operations that are tailored to run efficiently on a selected platform.
Neural architecture search is well-studied for e.g. image classification on benchmark datasets, but it is unclear how well it generalises outside of these benchmarks. We will explore how well neural architecture search works on tasks involving satellite and medical imagery, and explore failure modes that can occur and develop approaches to overcome these to make the search process more robust.
Neural architecture search algorithms have the potential to remove this barrier; they automatically search for an appropriate neural architecture for a particular task. However, they are not without issues: (1) they are very expensive to run; (2) they search within a narrow space of architectures; (3) we don't know how robust they are. This project aims to unlock the potential of neural architecture search by tackling these issues.
Neural architecture search is slow and expensive because search algorithms rely on having to train architectures to determine how effective they are, to inform the search process. We will build upon our recent work to bypass this training by developing inexpensive proxy measures, and create search algorithms that are able to leverage these proxies to perform neural architecture search efficiently.
Search spaces are narrow because they only draw from a single family of network architectures, and use neural operations that aren't optimised for a target device, instead selecting from a predefined list. We will create search spaces that are able to incorporate multiple families of architectures and use operations that are tailored to run efficiently on a selected platform.
Neural architecture search is well-studied for e.g. image classification on benchmark datasets, but it is unclear how well it generalises outside of these benchmarks. We will explore how well neural architecture search works on tasks involving satellite and medical imagery, and explore failure modes that can occur and develop approaches to overcome these to make the search process more robust.
Organisations
People |
ORCID iD |
| Elliot Crowley (Principal Investigator) |