Neuromorphic memristive circuits to simulate inhibitory and excitatory dynamics of neuron networks: from physiological similarities to deep learning

Why does the human brain not operate as a computer? Both use logic and numbers when operating. Nevertheless, the human memory is much more distributed (pattern-like) in contrast to localised computer bit-memory, has time decay (clogging), changes "wiring" when trained and is designed to process information signals (excitation waves) rather than just store bits in different memory locations as a computer does. Although modern computers significantly numerically outperform the human brain, they still cannot handle tasks requiring guessing and fuzzy logic.

This explains a booming interest in AI systems trained to perform certain tasks infeasible for numerical simulations. Currently AI technology is driven by two distinct goals: (i) technological demand (autonomous vehicles, online trading, AI for medical imaging analysis etc.) and (ii) an attempt to create an electronic brain with the ability to think, feel and interact with humans. Using different learning algorithms, AI has demonstrated abilities comparable to, or even outperforming, the human brain in several strategic and decision-making tasks (e.g., the game of Go). However, it is unclear how/if AI algorithms relate to information processing and the corresponding psycho-physiological processes in the brain. Answering this big question would help in not only making machines with human abilities but also elucidating whether a brain can be reduced to a biologically-wired electric circuit only or it has something beyond simple electric/chemical functionalities.

To truly emulate information processing in the brain (neuromorphic computing), a new generation of computer architecture should be developed. One of the most promising technologies for neuromorphic computing and signal processing is based on memristors, where resistance is tuned (e.g., switching between two states) by total charge passed through the system (e.g., resistance depends on applied electric pulse sequence/history which can encode an information signal propagating through the brain). Using Loughborough's expertise in solid state physics, functional materials, thin films, modelling, and AI, in synergy with a world-leading centre of neuromorphic research (the University of Massachusetts, Amherst) and neuroscience/physiological expertise (Salk Institute for Biological studies), and driven by the demand of UK industrial partners, we intend to develop a prototype of a memristive neuromorphic chipset able to analyse image-streams and to make decisions and choices by mimicking neural process in a brain cortex.

Inspired by biological neuron operation, and the deep learning AI paradigm, we propose to develop an electric circuit operating via two competing processes:
(1) Intermixing or interfering electric signals generated by visual stimuli at different time moments (e.g., subsequent video frames) and;
(2) Transmitting signals from one circuit layer to another in order to extract the main visual features/concepts.
A combination of these processes for image-stream analysis has never been considered before for neuromorphic systems and is the main novelty of the proposed research.

This allows us to compare image frames in a video and to reduce the complexity of the information towards a binary decision (choice). This neuromorphic two-process concept has a clear brain-functioning analogy: sensory stimuli excite a myriad of receptors generating superimposing signals, an end effect of which can be expressed by a short statement of recognition ("It's my Mom") or discrimination ("It was a car not a bike"). Cycles of interfering and convolving information followed by binary choice seems particularly well fit to memristor layered architecture where initial complex voltage-pattern encoding image stream reduces to switching or not of a certain memristor (signalling which decision/choice is made) in a deeper layer of the structure. The developed prototype will be the first memristive realisation of a visual cortex.

Various independent reports from the year 2017 estimate the effect of the future developments in artificial intelligence (AI) as potentially increasing the UK GDP by 10.3% by 2030, as well as in higher annual economic growth rate by 2035 - 3.9% instead of 2.5%. These ambitious plans can be achieved only if the AI progress is supported by growing AI researcher workforce as well as a capability and capacity in computational power, availability of new hardware and software, and novel/unconventional AI applications.

Capability and Capacity: The UK lags behind the US and China in investments into AI start-ups as well as behind the US and Europe in governmental support for research in AI. Recent hardware advances, such as GPUs, cloud computing and the multi-core CPU, have pushed forward AI and AI applications during the last decade. However, the future AI advancements in image processing, autonomous systems, object tracking, artificial decision-making etc., require a fundamental breakthrough in computational principles with no separation between the processing unit and memory. One of the most challenging directions of AI research is the development of electronic chips that use the architecture and elements mimicking neuron functionality. This offers not only new computational paradigms of information signal processing, but also a tool to study the brain at the level never achieved before. Indeed, due to the complexity of neural tissues, mathematical modelling of neural processes has a rather limited predictive power, while biological experiments face many ethical problems slowing down neuroscience research. Using neuromorphic hardware will enable the technology to control and enhance brain ability as well as a further progress in the understanding of brain disorders (testing, e.g., a hypothesis of excitatory-inhibitory imbalance nature of neurodevelopmental disorders by tuning inhibition balance in memristive circuits), enormously impacting mental health. One of the most promising technologies for brain emulators is based on memristors, thus, transferring this technology to the UK can create a nationwide impact and can enhance the capability for future AI advances in the UK.

Workforce: The project will allow training a new generation of AI-neuromorphic scientists and engineers contributing to future AI researcher labour force. The researchers and students involved in the project will be trained to fully understand AI tasks and market, the postdocs will be able to design their own neuromorphic hardware or to improve existing chips for specific tasks in mind and to programme this hardware in the most efficient way. Thus, the project will fill a gap in the workforce with both AI hardware and software expertise.

UK industry focus: To make our research industry-driven we plan to use the neuromorphic layered (deep learning) memristive chip for:
1. Large size image and video analysis (main industrial partner: ARM)
2. Pattern and motion recognition and object tracking (main industrial partner: ARM)
3. Decision-making for robotics, unmanned aerial vehicles and driverless vehicles (main industrial partner: HP)

Society: The general public will benefit from enhanced awareness about future AI opportunities and challenges. We will disseminate the project results to the end-users. Future AI technology is recognised by the public as a great opportunity to push further the intellectual ability of the whole civilisation and each individual by combining the human brain ability with modern computational electronics. We plan to inform the general public about the progress of the project via maintaining a project website, developing an outreach programme "Computers watching movies" with two public lectures per year, and an online neuromorphic AI contest for A-level and UG students where students will be invited to program neuromorphic chipsets remotely (online coding). This will also help in recruiting the best PhD applicants worldwide.