Centre for Spatial Computational Learning
Lead Research Organisation:
Imperial College London
Department Name: Electrical and Electronic Engineering
Abstract
The rise of the Deep Neural Network as an increasingly universal paradigm of computation has been the defining feature across much of computing in the last few years. At the same time, the traditional von Neumann processor paradigm is being challenged by the rise of hardware spatial computational accelerators, such as FPGAs, which face serious usability and programmability challenges. The deep learning application domain is narrow enough to allow us to reconsider the entire stack of spatial compute, from arithmetic circuits to cloud-based systems, in an integrated and domain-specific manner. We have strong expertise across the breadth of this space. If successful, our joint research centre has the potential to put the UK's expertise at the centre of the technology revolutionising the way high performance and low energy computation is specified and delivered, opening opportunities from ultra-low energy machine learning in internet of things devices through to powering scientific discovery in high-end server farms.
This centre would mark a break-through in international coordination of research in the field. We will: (a) work together to evolve a joint research strategy, (b) deliver elements of that joint research strategy through the staff employed on this grant as well as through academic staff and PhD students in all institutions, (c) facilitate a managed researcher exchange programme through funding exchange / secondment expenses for investigators and visiting researchers, as well as supporting secondments to/from industry (d) hold annual showcase events of our work, and (e) deliver high-quality R&D and STEM advocacy outreach.
By the end of the three-year period, we expect to have a self-sustaining momentum of internationally-coordinated research, incorporating the initial investigators, but extending beyond to new groups and the network of SMEs developing in this area.
This centre would mark a break-through in international coordination of research in the field. We will: (a) work together to evolve a joint research strategy, (b) deliver elements of that joint research strategy through the staff employed on this grant as well as through academic staff and PhD students in all institutions, (c) facilitate a managed researcher exchange programme through funding exchange / secondment expenses for investigators and visiting researchers, as well as supporting secondments to/from industry (d) hold annual showcase events of our work, and (e) deliver high-quality R&D and STEM advocacy outreach.
By the end of the three-year period, we expect to have a self-sustaining momentum of internationally-coordinated research, incorporating the initial investigators, but extending beyond to new groups and the network of SMEs developing in this area.
Planned Impact
1. Economic Impact
Our strategy here will be to release software and programmable hardware-design tool and artefacts under a permissive open-source license such as the MIT Licence, allowing for future commercialisation, at the same time as exploring avenues for that exploitation through our advisory board. As is standard practice, background IP ownership will not be affected by this project; the PI will actively manage the project to minimise use of any background IP that is only available under restrictive licensing, and preference will always be given to commercialisable open-source licensed background in the event of various options being available. We expect existing industrial partners to benefit directly from this work, and will use the contacts of the ElecTech council, whose CEO will chair our advisory group, to ensure impact on UK SMEs beyond the initial set of partners.
2. Societal Impact
The nature of this research project - both the technical nature and the international nature - lends itself to some exciting outreach and public engagement possibilities. Both Constantinides and Merrett have strong records in outreach; Merrett will lead the outreach and public engagement programme of our project.
We aim to work with our industrial partners to engage the public in the following ways: (i) through participation in the Science Museum Lates series, through which we have had significant success in the past, (ii) through internal and external festivals / science fairs, which last year included a contribution from Bouganis on machine learning for drones, and (iii) through the production of public-facing high quality YouTube videos and Twitter, an approach taken by Merrett in the PRiME programme grant.
3. People Impact
One major impact of the proposed centre will be on the research staff and PhD students who will work under the centre's auspices. They will end their study / research contract immeasurably stronger in skill set and research depth for their exposure to the best international research environments in the area. Our programme of internships and placements will leave them as some of the most employable individuals within the high-tech world, whether in academia or in industry. Through participation in our annual retreats, they will have gained exposure to both academic and industrial thinking from the top players in three countries; investigators will track and mentor individuals in their career paths. No matter where these individuals move on to at the end of their project, this will leave the UK's ICT community immeasurably strengthened by having individuals with a strong positive experience of mentoring under the auspices of an EPSRC project, leaving key advocates in place for the importance of STEM research funding in the UK.
Our strategy here will be to release software and programmable hardware-design tool and artefacts under a permissive open-source license such as the MIT Licence, allowing for future commercialisation, at the same time as exploring avenues for that exploitation through our advisory board. As is standard practice, background IP ownership will not be affected by this project; the PI will actively manage the project to minimise use of any background IP that is only available under restrictive licensing, and preference will always be given to commercialisable open-source licensed background in the event of various options being available. We expect existing industrial partners to benefit directly from this work, and will use the contacts of the ElecTech council, whose CEO will chair our advisory group, to ensure impact on UK SMEs beyond the initial set of partners.
2. Societal Impact
The nature of this research project - both the technical nature and the international nature - lends itself to some exciting outreach and public engagement possibilities. Both Constantinides and Merrett have strong records in outreach; Merrett will lead the outreach and public engagement programme of our project.
We aim to work with our industrial partners to engage the public in the following ways: (i) through participation in the Science Museum Lates series, through which we have had significant success in the past, (ii) through internal and external festivals / science fairs, which last year included a contribution from Bouganis on machine learning for drones, and (iii) through the production of public-facing high quality YouTube videos and Twitter, an approach taken by Merrett in the PRiME programme grant.
3. People Impact
One major impact of the proposed centre will be on the research staff and PhD students who will work under the centre's auspices. They will end their study / research contract immeasurably stronger in skill set and research depth for their exposure to the best international research environments in the area. Our programme of internships and placements will leave them as some of the most employable individuals within the high-tech world, whether in academia or in industry. Through participation in our annual retreats, they will have gained exposure to both academic and industrial thinking from the top players in three countries; investigators will track and mentor individuals in their career paths. No matter where these individuals move on to at the end of their project, this will leave the UK's ICT community immeasurably strengthened by having individuals with a strong positive experience of mentoring under the auspices of an EPSRC project, leaving key advocates in place for the importance of STEM research funding in the UK.
Organisations
- Imperial College London (Lead Research Organisation)
- Cornell University (Collaboration)
- University of Toronto (Collaboration, Project Partner)
- University of California, Los Angeles (UCLA) (Collaboration)
- UNIVERSITY OF SYDNEY (Collaboration)
- KING'S COLLEGE LONDON (Collaboration)
- University of California, Los Angeles (Project Partner)
- ARM (United Kingdom) (Project Partner)
- Corerain Technologies (Project Partner)
- Imagination Technologies (United Kingdom) (Project Partner)
- Xilinx (United States) (Project Partner)
- Maxeler Technologies (United Kingdom) (Project Partner)
Publications
Voss N
(2021)
On Predictable Reconfigurable System Design
in ACM Transactions on Architecture and Code Optimization
Que Z
(2024)
LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics
in ACM Transactions on Embedded Computing Systems
Wang E
(2023)
Enabling Binary Neural Network Training on the Edge
in ACM Transactions on Embedded Computing Systems
Rasoulinezhad S
(2021)
Rethinking Embedded Blocks for Machine Learning Applications
in ACM Transactions on Reconfigurable Technology and Systems
Eldafrawy M
(2020)
FPGA Logic Block Architectures for Efficient Deep Learning Inference
in ACM Transactions on Reconfigurable Technology and Systems
Wang E
(2023)
Logic Shrinkage: Learned Connectivity Sparsification for LUT-Based Neural Networks
in ACM Transactions on Reconfigurable Technology and Systems
Tarafdar N
(2021)
AIgean : An Open Framework for Deploying Machine Learning on Heterogeneous Clusters
in ACM Transactions on Reconfigurable Technology and Systems
Que Z
(2022)
Remarn: A Reconfigurable Multi-threaded Multi-core Accelerator for Recurrent Neural Networks
in ACM Transactions on Reconfigurable Technology and Systems
Joksas D
(2022)
Nonideality-Aware Training for Accurate and Robust Low-Power Memristive Neural Networks.
in Advanced science (Weinheim, Baden-Wurttemberg, Germany)
Bonato V
(2021)
Class-specific early exit design methodology for convolutional neural networks
in Applied Soft Computing
Description | Please see https://twitter.com/spatialmlnet for full details |
Exploitation Route | Being explored with industrial advisory board |
Sectors | Digital/Communication/Information Technologies (including Software) Electronics |
URL | https://twitter.com/spatialmlnet |
Description | Used inside consortium academically and also this patent with Intel https://patents.justia.com/patent/20220366112 |
First Year Of Impact | 2022 |
Sector | Digital/Communication/Information Technologies (including Software),Electronics |
Impact Types | Economic |
Description | Two research students -- bespoke |
Amount | £955,605 (GBP) |
Organisation | Advanced Micro Devices (AMD) |
Sector | Private |
Country | United States |
Start | 12/2022 |
End | 04/2028 |
Title | Data for Similarity-aware CNN for Efficient Video Recognition at the Edge |
Description | This data is associated with the the "Similarity-aware CNN for Efficient Video Recognition at the Edge" article published in IEEE transaction on Computer-Aided Design of Integrated Circuits and Systems. |
Type Of Material | Database/Collection of data |
Year Produced | 2021 |
Provided To Others? | Yes |
URL | https://eprints.soton.ac.uk/452703/ |
Title | Dataset for "Dynamic Transformer for Efficient Machine Translation on Embedded Devices" |
Description | This dataset supports the publication: 'Dynamic Transformer for Efficient Machine Translation on Embedded Devices' in '3rd ACM/IEEE Workshop on Machine Learning for CAD (MLCAD'21)'. |
Type Of Material | Database/Collection of data |
Year Produced | 2021 |
Provided To Others? | Yes |
URL | https://eprints.soton.ac.uk/450454/ |
Title | Dataset in support of the article 'Enabling Image Net-scale deep learning on MCUs for accurate and efficient inference' |
Description | Data obtained in Enabling Image Net-Scale Deep Learning on MCUs for Accurate and Efficient Inference research. To support an article published in IEEE Internet of Things Journal |
Type Of Material | Database/Collection of data |
Year Produced | 2023 |
Provided To Others? | Yes |
URL | https://eprints.soton.ac.uk/id/eprint/483991 |
Description | Cornell (& Cornell Tech) |
Organisation | Cornell University |
Country | United States |
Sector | Academic/University |
PI Contribution | Research ideas |
Collaborator Contribution | Studentships |
Impact | https://twitter.com/spatialmlnet |
Start Year | 2022 |
Description | Kings |
Organisation | King's College London |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Working with Prof Bashir Al-Hashimi |
Collaborator Contribution | Working on project as described |
Impact | See publications |
Start Year | 2020 |
Description | Sydney |
Organisation | University of Sydney |
Country | Australia |
Sector | Academic/University |
PI Contribution | Research ideas |
Collaborator Contribution | Studentship |
Impact | https://twitter.com/spatialmlnet |
Start Year | 2022 |
Description | Toronto |
Organisation | University of Toronto |
Country | Canada |
Sector | Academic/University |
PI Contribution | Research ideas |
Collaborator Contribution | Studentships |
Impact | https://twitter.com/spatialmlnet |
Start Year | 2020 |
Description | UCLA |
Organisation | University of California, Los Angeles (UCLA) |
Country | United States |
Sector | Academic/University |
PI Contribution | Research ideas |
Collaborator Contribution | Studentships |
Impact | https://twitter.com/spatialmlnet |
Start Year | 2020 |
Title | Apparatus, Device, Method, and Computer Program for Generating a Register Transfer Level Representation of a Circuit |
Description | Work with industrial partner Intel and centre student Coward |
IP Reference | 17/649,937 |
Protection | Patent / Patent application |
Year Protection Granted | 2022 |
Licensed | Commercial In Confidence |
Impact | Improved Intel tools for circuit design |
Title | PolyLUT |
Description | Neural network training for FPGA implementation |
Type Of Technology | Software |
Year Produced | 2023 |
Open Source License? | Yes |
Impact | Becoming widely cited. 25 stars, 2 forks on GitHub. |
URL | https://github.com/MartaAndronic/PolyLUT |
Title | fpgaConvNet |
Description | Tool for automatically mapping neural networks to FPGA hardware |
Type Of Technology | Software |
Year Produced | 2024 |
Impact | Widely cited. Note this is constantly refined software, so date is latest refinement. |
URL | https://cas.ee.ic.ac.uk/people/sv1310/fpgaConvNet.html |
Description | Online public seminars (and YouTube channel) |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Industry/Business |
Results and Impact | Raise awareness of work of the centre. Widely shared on professional networks thousands of times. Youtube figures can be viewed via link below. |
Year(s) Of Engagement Activity | 2023,2024 |
URL | https://www.youtube.com/@spatialmlseminar/videos |