From Stochasticity to Functionality: Probabilistic Computation with Magnetic Nanowires

Lead Research Organisation: University of Sheffield
Department Name: Materials Science and Engineering

Abstract

Machine learning, or the ability of computers to intelligently analyse complex data sets, may be the defining technology of our age. Society produces huge amounts of data in areas as diverse as economics, weather, medicine, science and social media, analysis of which can greatly enhance our decision making. However, current computers are poorly suited to analysing large and complex datasets, particularly when compared to animal and human brains.

For example, while a typical contemporary computer can perform simple numerical calculations much more quickly than the brain, its efficiency is almost one million times lower when performing more complex data analysis such as recognising human faces. This inefficiency results from attempts to emulate "neuromorphic", or brain-like, computational processes by brute force on hardware which is ill-suited to the task. For example, in a conventional computer memory and processing are inherently separated, whereas they share the same medium in the brain. To overcome these limitations, bespoke computers that are specifically designed for neuromorphic computation are required.

Away from pure computational power, the "big data" revolution has been driven by our ability to store information. This is the result of nano-scale magnetic technology in the form of hard-disk drives. However, nanomagnetic devices suffer from an encroaching limitation, in that their behaviour becomes increasingly unreliable, or stochastic, as they are further miniaturised to increase performance. While this is devastating for conventional digital computers, there is strong evidence that stochastic behaviour can be tolerated in, or even enhance, the performance of neuromorphic technologies. This raises the tantalising prospect of nanomagnetic technology being ideally suited for developing new computer forms of computer hardware for data analysis.

In this project, a collaboration between experts in nanomagnetic technology and computer science at the University of Sheffield, we will perform a pilot study to investigate whether stochastic behaviour in "magnetic domain wall devices" a promising form of magnetic nanotechnology where magnetic information is flowed through nanowire conduits, can be used to realise new, neuromorphic computer architectures. Working with an advisory board of academic and industry experts in machine learning technology we will demonstrate new data analysis devices, before creating a roadmap to develop the devices into real technology. Eventual success in developing these technologies could result in powerful hardware platforms that can provide even the smallest device the ability to intelligently analyse its environment to make informed decisions.

Planned Impact

The aim of this project is to lay the foundation for new forms of neuromorphic computer hardware platforms that are specifically designed to natively implement machine learning approaches to data analysis. At this conceptual stage our aim is to demonstrate the feasibility of these devices, and to gain an understanding of how their ultimate performance might compare to other technologies that are currently under development. The primary impact of the work during the project's lifetime will therefore be the generation of new knowledge, and the major outputs of the project will be papers in academic journals and presentations at international conferences. Thus, in the short term (1-5 years) our research will impact the academic community through our research-enabling results, which we believe will inspire entirely new lines of enquiry.

By the end of our project we aim to have created a roadmap indicating how the most promising of our results can be moved forward towards commercial devices. Such developments will require a consortium approach and require us to engage a range of industry experts in areas such as machine learning and computational hardware architecture. Thus, beyond the end of our project (5-10 years) we aim to begin to generate industrial impact through collaborative research. To assist us in achieving this we have assembled a project advisory board that includes researchers from a major semiconductor and software company (ARM) and a successful spin out company working to develop neuromorphic computing technology (aiCTX). The expertise of our advisory board will be invaluable in steering our project towards directions with genuine technological promise and in developing industry contacts.

Developing technologies to intelligently analyse data is one of the defining challenges of our era. These technologies may take the form of small scale systems targeted at ubiquitous computing applications, or larger scale systems aimed at tackling "big data" problems in a fast and efficient manner, and will have a transformational impact on areas as diverse as economics, climate monitoring, security, safety, healthcare, science and social media. Therefore, looking further into the future (10+ years), our research has the potential to create substantial societal and economic impact through the enabling of new powerful technologies.

Success in our programme of research will require detailed knowledge of contemporary nanoscale fabrication and characterisation techniques, machine learning algorithms, and mathematical modelling techniques. The two post-doctoral researchers employed on this project will gain training in each of these, thus providing them with an extensive portfolio of skills that will leave well equipped for their future careers in academia or in the private sector. A good supply of scientists and engineers with such skills is key to the future success of the UK economy. The impact here will be realised over the 3-year timescale of the project.

The project team have strong track records of public engagement activities and frequently deliver outreach activities to primary/secondary school and A-level students. We will use our strong links with schools and colleges to deliver talks to students, and will seek to engage with the general public through creating short videos disseminated through online platforms, and by presenting at exhibitions the University of Sheffield's annual Festival of the Mind, the Big Bang Fair and the Cheltenham Science Fair. These outreach activities will benefit the general public over the lifespan of the project (1-3 years).

Publications

10 25 50
publication icon
Ababei RV (2021) Neuromorphic computation with a single magnetic domain wall. in Scientific reports

publication icon
Dawidek R (2021) Dynamically Driven Emergence in a Nanomagnetic System in Advanced Functional Materials

publication icon
Ellis M (2023) Machine learning using magnetic stochastic synapses in Neuromorphic Computing and Engineering

publication icon
Manneschi L (2021) Exploiting Multiple Timescales in Hierarchical Echo State Networks in Frontiers in Applied Mathematics and Statistics

publication icon
Manneschi L (2023) SpaRCe: Improved Learning of Reservoir Computing Systems Through Sparse Representations. in IEEE transactions on neural networks and learning systems

publication icon
Ozdemir A (2022) EchoVPR: Echo State Networks for Visual Place Recognition in IEEE Robotics and Automation Letters

publication icon
Venkat G (2023) Magnetic domain walls: types, processes and applications in Journal of Physics D: Applied Physics

publication icon
Vidamour I (2023) Reconfigurable reservoir computing in a magnetic metamaterial in Communications Physics

 
Description Thus far we have achieved the following:

(1) We have produced preliminary results demonstrating how the stochastic behaviours of magnetic nanowire devices can be used to realise the paradigm of stochastic computing, where floating point numbers are encoded within random bitstreams and simple logic gates perform complex logical operations. For example, we have shown how an external stimulus applied to a defect site can allow the tuning of domain wall pinning probabilities. This can be used for the generation of random bitstreams with tuneable average values. We have also shown how magnetic AND gates can be used to multiply the values encoded in random bitstreams. Progress is being made towards the realisation of further types of logic gates within this framework.

(2) We have produced numerical models, showing that nanowire-based stochastic synapses can be used to realise feed-forward neural networks (e.g. for written digit recognition). Our results indicate that classification accuracy is reduced when compared to conventional artificial neural networks, but that relative performance can be improved by (a) adding duplicate synapses, (b) running networks multiple times or (c) creating multilayer networks. The models do however typically perform better than simple (non-stochastic) binary neural networks due to the effective "analogue" response provided by the tunability of the stochastic behaviour. Most recently we have created and benchmarked a new learning rule designed specifically for the stochastic networks. This rule allows the weights of the network to be tuned for maximum performance depending on how many time each synaptic response is to be averaged. Further modelling has shown that networks with stochastic synapses may have advantages over conventional ANN when applied to reinforcement learning problems, where "noise" must be added to prevent networks becoming trapped in local minima. We have also shown that similar devices may be used to realise restricted Boltzman machines as used for e.g. integer factorisation problems.

(3) We have experimentally performed machine learning tasks (written digit recognition) using nanowire-based synapses. These have measured a single synapse in series to emulate the behaviour of an entire network. The performance in tasks is in-line with that of the modelling described in (2).

(4) We have used newly developed mathematical model to show how emergent, stochastic domain wall dynamics within magnetic ensembles can be used to perform a range machine learning tasks, including speech recognition via the reservoir computing paradigm. Characterisation of these models has allowed us to develop understanding of how capabilities of the reservoirs to tackle different tasks can be tuned by manipulation speed and scaling of the data input. We have also measured the computational capability of these ensembles using task independent metrics, which provide a measure of the ability of reservoirs to map different inputs to different outputs and to generalise similar outputs to the same output. We then went on to show how characterisation of these metrics is a powerful optimisation tool for working out how to set up the system optimally for performing real work tasks.

(5) We have performed experimental measurements of emergent, stochastic domain wall dynamics within magnetic ensembles and shown that these can be used to perform reservoir computing. Here, the magnetic states of the arrays were probed by measuring their electrical resistance, while data was inputted into the ensemble using applied magnetic fields. We have performed a number of challenging computational tasks using this approach including signal transformation, speech recognition and time series prediction.

(6) We have conducted a modelling study exploring how ensembles of stochastic, super-paramagnetic (i.e. thermally active) magnetic nanostructures can be used to create a low power, voltage controlled platform for reservoir computing. We believe devices based on this technology could find applications in edge computing applications where low power consumption and latency are essential. A PhD student has now been recruited to work on an experimental realisation of these devices.
Exploitation Route As yet our findings show how nanomagnetic devices can be used to realise a variety different computational paradigms using magnetic devices. These are not yet ready for industrial development, but we aim to have a roadmap of how this will occur by the end of the project. Machine learning is now ubiquitous in society, and new devices that can realise it in an efficient manner would likely have an influence on a huge range of areas including smart technolgies, prediction of complex systems such as the weather and financial markets, and security.

In summer 2020 we held our first advisory board meeting, which allowed us to present our results to researchers from ARM, CNRS Thales and ETH Zurich.
Sectors Aerospace, Defence and Marine,Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Electronics,Financial Services, and Management Consultancy,Manufacturing, including Industrial Biotechology,Security and Diplomacy

 
Description MARCH: Magnetic Architectures for Reservoir Computing Hardware
Amount £1,162,094 (GBP)
Funding ID EP/V006029/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2021 
End 06/2024
 
Description Royce Materials Challenge Accelerator Programme - Magnets that Think and Feel
Amount £56,023 (GBP)
Funding ID MCAP016 
Organisation Henry Royce Institute 
Sector Academic/University
Country United Kingdom
Start 11/2022 
End 04/2023
 
Title Quantifying the Computational Capability of a Nanomagnetic Reservoir Computing Platform with Emergent Magnetisation Dynamics 
Description Data used in the production of the publication 'Quantifying the Computational Capability of a Nanomagnetic Reservoir Computing Platform with Emergent Magnetisation Dynamics'. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
URL https://figshare.shef.ac.uk/articles/dataset/Quantifying_the_Computational_Capability_of_a_Nanomagne...
 
Description Collaboration with Dr Joe Friedman (University of Texas, Dallas) 
Organisation University of Texas
Country United States 
Sector Academic/University 
PI Contribution We have been investigating how magnetic nanowire devices can be natively used to realise data operations required to perform inference tasks.
Collaborator Contribution Dr Friedman has provided us support in understanding how nanomagnetic hardware could be used for creating devices that can perform inference tasks in hardware.
Impact So far the collaboration has not resulted in any outcomes, although we anticipate at least one paper and a grant application will result from it.
Start Year 2019
 
Description Collaboration with Imperial College London on Reservoir Computing with Magnetic Nanostructures. 
Organisation Imperial College London
Department Imperial College Trust
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution We have been working with the group of Dr Will Branford and Dr Jack Gartside on neuromorphic computing with magnetic nanostructures. Our collaborations have involved the Sheffield Functional Magnetic Materials Group performing focused magneto-optic Kerr measurements on spin ice arrays made at ICL. There has also been collaboration between the Sheffield Machine Learning Group and ICL on emulating devices where multiple reservoirs are connected in series.
Collaborator Contribution The ICL group have performed FMR measurements on magnetic ring arrays made fabricated at the University of Sheffield.
Impact None so far.
Start Year 2021
 
Description Collaboration with University of Exeter to perform Kerr microscopy 
Organisation University of Exeter
Country United Kingdom 
Sector Academic/University 
PI Contribution We will be visiting the new Extremag facility at the University of Exeter to perform full-field Kerr microscopy measurements of domain wall logic gates. The current work aims to explore how effectively the Evico microscope system can measure these nanostructures and pump-prime future collaboration through the facility.
Collaborator Contribution Exeter will be providing access to their suite of experimental facilities and providing technical support during our measurements.
Impact No outputs yet - collaboration has only just begun.
Start Year 2022
 
Description Collaboration with the University of Sherbrooke 
Organisation University of Sherbrooke
Country Canada 
Sector Academic/University 
PI Contribution We have begun to collaborate with the group of Prof. Julien Sylvestre at the University of Sherbrooke. Thus, far this has resulted in discussion of how our group can collaborate to use networks of reservoir computers to control soft robotic machines.
Collaborator Contribution See above.
Impact None so far
Start Year 2022
 
Description Outreach Presentation by Prof. Dan Allwood at Magnetism 2020 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Prof. Dan Allwood gave a public outreach lecture entitled "How Magnets Work" as part of the Magnetism 2020 conference. This talk gave a general introduction to the properties of magnetic materials, before describing how we were trying to harness the emergent behavior of nanomagnet ensembles to perform reservoir computing. The talk was well received, with a large number of audience questions being asked at the end of the session.
Year(s) Of Engagement Activity 2020