Engineering Fellowship for Growth - Neuromorphic Printed Tactile Skin (NeuPRINTSKIN) (Ext)

Lead Research Organisation: University of Glasgow
Department Name: School of Engineering

Abstract

This project is an extension of the Engineering Fellowship for Growth: Printable Tactile Skin (PRINTSKIN). PRINTSKIN focused on developing a robust ultra-flexible tactile skin, endowing state-of-the-art robotic hand with the tactile skin and validating it by using tactile information from large areas of robot hands to handle daily object with different curvatures. The tactile skin is critical for autonomy of robots and for the safe human-robot interaction need to meet societal needs such as helping elderly. The tactile feedback is critical in such tasks as the robots often use incomplete environmental model which are insufficient to deal with external changes. The touch sensing is also needed to augment other sensory modalities (e.g. vision) in robotics.

Inspired by nature, numerous works including PRINTSKIN project, have harnessed the technological advances to develop e-skin with some features mimicking human skin - particularly the contact parameters and morphological features. However, just morphology of skin or capturing few parameters that we experience as touch is not enough. To develop an effective tactile skin, there is also a need to understand the perceptual mechanism and to find the ways to extract the information from large tactile data (especially in the case of large area tactile skin). Research suggests that distributed computing takes place in the biological tactile sensory system. For example, the ensemble of tactile data from peripheral neurons is considered to indicate both the contact force and its direction. This means raw tactile data is not sent to brain and that some distributed computing takes place in our skin. This is in sharp contrast with current e-skin approaches which transmit the as acquired tactile data to higher perceptual levels. The research proposed here will break this trend by introducing neuron like processing and bring a step change in the tactile sensing research by developing the first neuromorphic tactile skin or the brainy skin.

A new neural layer, developed using the printed silicon nanowire methodology developed in PRINSKIN, will be integrated under the e-skin to enable fast, energy efficient and distributed tactile data processing. This groundbreaking research will lead to the first hardware implementation of neuromorphic tactile skin. Innovative schematic, with novel neural nanowire field effect transistors and memory devices as building blocks, will be used to develop the neurons which will eventually lead to the neural layer. The advanced tactile skin will be benchmarked against available semi-rigid skins and the skin developed through PRINTSKIN. The skin will be validated on at least three different robotic hands (Shadow Hand, i-Limb, and custom 3D printed hand) used for dexterous manipulation and prosthetics. By adding neural layer underneath the current tactile skin, this extension project will add significant new perspective to the fellowship achievements and trigger transformations in strategic areas such as robotics, prosthetics, neurotechnology, wearable systems, next-generation computing and flexible and printable electronics.

Planned Impact

neuPRINTSKIN will develop critical neuromorphic tactile skin with neural data process capability to enable future robots to engage in highly interactive tasks such as caring for the elderly. The feedback from tactile skin will shape up the way society interacts with robotic devices as robots will be able to gather rich contact information and think, act, and react humanly. The near to long-term impact of this multidisciplinary research will be seen in academia, public sector, industry, social enterprises, public, schools and more.

In near-term the tactile skin will result in new research foci in robotics allowing multiple contact points to be exploited to carry out manipulation tasks in cluttered environments. With neuromorphic distributed processing capability of skin, it will be possible to investigate how tactile data is processed by humans. Tactile skin will enable neuroscientists to understand the functioning of receptors in human skin. The impact of proposed research will also be seen in bioelectronics, for example, together with ultra-thin bendable electronics implanted in the brain, the tactile skin will enable in long-term the neural control of artificial limbs and new biomimetic technologies. Equipped with tactile skin the robots will be safe interactive learning tools to stimulate the imagination of children. For example, due to lack of emotions in robots, autistic children find them less threatening than their teachers and easier to engage with. Other potential near to mid-term application of could be industrial task such as fruit or object sorting, where the bio-mimicking neural networks in the skin of robotic hands could classify and held objects based on the physical and optical parameters from special optical sensors from tactile e-skin. I believe in the long-term, the ultra-flexible tactile skin will bestow sensory feeling to prosthetic limbs and improve lives of amputees. We will see quick benefits of tactile skin in areas such as teleoperation where tactile feedback enables extension of feelings. As per latest Gartner report, the neuromorphic hardware will have major impact on future computing in mid-to-long term.

neuPRINTSKIN will bring about profound impact through new printing methodology for neural and electronic layers and the 3D integration of flexible electronics. The paradigm of electronics manufacturing and prototyping will change with small-scale manufacturing, capable of printing high-performance circuits straight from the computer, replacing the large manufacturing centres based overseas. The cost reduction because of printing will have enormous impact on schemes such as using barcodes to track objects in supermarkets or disposable sensors for health and environment monitoring. As an example, the possibility to develop identity tags with performance at par with Si and cost at par with existing barcodes will drive replacement of some 15 trillion barcodes in existence today and boost new areas such as internet of things. The technology affordability could lead to changes in policies e.g. taxation system of UK could be based on real-time data collected from consumed products. With reduced electronic waste, printing methods are also environment friendly.

There is significant gain in printing of high-performance devices, as many new unforeseen applications which will almost inevitably - by ever-demanding end-user - supersede the solid-state IC, just as the IC replaced the discrete circuit board electronics. These include intelligent packaging, conformal electronics (e.g. in automotive applications), wearable electronics, smart windows including antenna for wireless communications, medical sensors and inorganic thin film solar cells. There is great potential in the UK to establish companies based on such end products. Summarizing, neuPRINTSKIN has the potential to emulate yet another revolution in the microelectronics and will trigger transformations robotics and similar areas.

Publications

10 25 50
publication icon
Bhattacharjee M (2021) Printed Chipless Antenna as Flexible Temperature Sensor in IEEE Internet of Things Journal

publication icon
Bhattacharjee M (2020) PEDOT:PSS Microchannel-Based Highly Sensitive Stretchable Strain Sensor in Advanced Electronic Materials

publication icon
Chakraborty M (2022) Electronic Waste Reduction Through Devices and Printed Circuit Boards Designed for Circularity in IEEE Journal on Flexible Electronics

publication icon
Christou A (2020) 3D Touch Surface for Interactive Pseudo-Holographic Displays in Advanced Intelligent Systems

publication icon
Christou A (2021) Pseudo-Hologram with Aerohaptic Feedback for Interactive Volumetric Displays in Advanced Intelligent Systems

publication icon
Christou A (2020) GlasVent-The Rapidly Deployable Emergency Ventilator. in Global challenges (Hoboken, NJ)

 
Description I developed a new large area electronics skin using printed electronics (based on Is). I have also submitted a patent application for internal review by the University IP office. I also reported first energy autonomous e-skin and patent application for this work has been submitted.

Novel method for 3D printing of multi-materials for affordable prosthetics with embedded touch sensing.

Novel flexible supercapacitors for energy autonomous e-skin and neuromorphic methods of computation.

Novel manufacturing method for development of high-performance electronics by printing. This will lead to first demonstration of printed CMOS processors. A patent application is being finalised.

Printed Synaptic transistors based e-skin realised.
Exploitation Route currently the e-skin development work is in progress. Once developed others in robotics and prosthetics will be able to use the e-skin to develop new human-robot interaction capabilities and in areas such as haptics and tactile internet. Neuromorphic e-skin is another direction. Further, the sustainable manufacturing of high-performance printed CMOS circuits is being explored. There should be several commercial opportunities.
Sectors Aerospace, Defence and Marine,Agriculture, Food and Drink,Education,Electronics,Energy,Healthcare,Manufacturing, including Industrial Biotechology,Culture, Heritage, Museums and Collections,Retail

 
Description Findings from this project have been used in many public engagement and knowledge exchange activities. The research has led to more projects including programme grant, standard grant and funded projects by industry. Researcher working on this project also participated in InnovateUK (e.g. iCure) projects promoting the commercialisation. Recently ARM has supported iCase phd studentship and currently there are discussions going on in terms of spin off.
First Year Of Impact 2020
Sector Aerospace, Defence and Marine,Agriculture, Food and Drink,Digital/Communication/Information Technologies (including Software),Education,Electronics,Energy,Environment,Healthcare,Manufacturing, including Industrial Biotechology
Impact Types Societal,Economic

 
Description 'Hetero-print': A holistic approach to transfer-printing for heterogeneous integration in manufacturing
Amount £5,541,651 (GBP)
Funding ID EP/R03480X/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 04/2018 
End 05/2023
 
Description (AQUASENSE) - Innovative Network for Training in wAter and Food QUality monitoring using Autonomous SENSors and IntelligEnt Data Gathering and Analysis
Amount € 4,064,539 (EUR)
Funding ID 813680 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 09/2018 
End 09/2022
 
Description (INTUITIVE) - INnovative Network for Training in ToUch InteracTIVE Interfaces
Amount € 4,156,300 (EUR)
Funding ID 861166 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 09/2019 
End 09/2023
 
Description (NeuTouch) - Understanding neural coding of touch as enabling technology for prosthetics and robotics
Amount € 4,108,695 (EUR)
Funding ID 813713 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 03/2019 
End 02/2023
 
Description Green Energy-Optimised Printed ICs
Amount £1,201,481 (GBP)
Funding ID EP/W019248/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2022 
End 12/2024
 
Description Next Generation Energy Autonomous Textile Fabrics based on Triboelectric Nanogenerators
Amount £1,500,000 (GBP)
Funding ID EP/V003380/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 03/2021 
End 03/2025
 
Description Predictive Haptic COding Devices In Next Generation interfaces
Amount € 4,000,000 (EUR)
Funding ID H2020-FETOPEN-2018- 829186 
Organisation European Commission H2020 
Sector Public
Country Belgium
Start 01/2019 
End 09/2022
 
Description UK Robotics and Artificial Intelligence Hub for Offshore Energy Asset Integrity Management
Amount £14,635,591 (GBP)
Funding ID EP/R026173/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2017 
End 04/2021
 
Description UKRI Centre for Doctoral Training in Socially Intelligent Artificial Agents
Amount £5,000,000 (GBP)
Funding ID EP/S02266X/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2020 
End 09/2024
 
Description ARM Collaboration 
Organisation Arm Limited
Country United Kingdom 
Sector Private 
PI Contribution Leading the fellowship research.
Collaborator Contribution Regular scientific exchanges about new directions for fellowship research and hosting of researchers.
Impact Several papers published as part of fellowship. ARM has been supporting the annual IEEE international conference (Flexible Printable Sensors and Systems) which founded in 2019.
Start Year 2018
 
Description Collaboration with McGill University on disposable sensors for digital agriculture 
Organisation McGill University
Country Canada 
Sector Academic/University 
PI Contribution participated in proposal writing and fabrication of sensors.
Collaborator Contribution Led the proposal for visiting as fellow.
Impact none yet
Start Year 2021
 
Description I and J Media Ltd T/A The Basement 
Organisation I&J Media Limited
Country Ireland 
Sector Private 
PI Contribution We made proof-of concept transparent touch screen for the company.
Collaborator Contribution Company funded the proof of concept study.
Impact proof-of-concept prototype provided to the company
Start Year 2019
 
Description Industry Collaboration - Nuprint 
Organisation Nuprint Technologies Ltd
Country United Kingdom 
Sector Private 
PI Contribution I am supervising the team of researchers (2 postdocs + 2 PhDs) who are working on projects finalised after discussions with Nuprint.
Collaborator Contribution Access to the manufacturing unit or researchers to have first hand knowledge about printing of labels and supply of some inks.
Impact We have published 5 journal articles based on the work with Nuprint. Currently an IP application is being discussed.
Start Year 2018
 
Description PhD Studentship - BMW 
Organisation Bayerische Motoren Werke (BMW)
Country Germany 
Sector Academic/University 
PI Contribution Supervising an international PhD student fully funded by BMW.
Collaborator Contribution Funding for the PhD student and access to labs in Munich.
Impact no output as yet.
Start Year 2020
 
Description PhD Studentship - Tata 
Organisation Tata Consultancy Services
Country India 
Sector Private 
PI Contribution Supervising a PhD student, whose tuition fee is partly covered by Tata consultancy services.
Collaborator Contribution Funding for student and access to their lab and computing facilities during secondments of student.
Impact no outcome yet.
Start Year 2020
 
Description PhD supportv- Shadow 
Organisation Shadow Robot Company
Country United Kingdom 
Sector Private 
PI Contribution We are developing new robotic hand with embedded touch sensing.
Collaborator Contribution Shadow Robot Company is sharing the design of Shadow Hand and we are updating it with embedded touch sensors.
Impact 1 International Conference Paper
Start Year 2018
 
Title DISPLAY APPARATUS, CONTROLLER THEREFOR AND METHOD OF CONTROLLING THE SAME 
Description This invention relates to a display apparatus, a controller for a display apparatus and a method of controlling a display apparatus. The display apparatus included a display arranged to display a source image; at least one screen arranged in front of said display such that the projection of the image and its reflection by the screen causes a holographic image of a three-dimensional object to be displayed to a user viewing said screen from a position that is neither perpendicular to nor parallel to the plane of said screen; a position detection apparatus for detecting the position of a part of a user's body in proximity to said holographic image or to said screen and to generate position information relating to said detected position; an air source arranged to direct air to one or more locations in proximity to said screen; and a controller arranged to receive position information from the position detection apparatus and to control the air source so as to direct air in accordance with said received position information so as to provide sensory feedback to the detected part of the user's body. In this manner a user can be enabled to manipulate a holographic image using their hands or other body parts and to receive sensory feedback, particularly touch, hardness and/or temperature feedback from their interaction. 
IP Reference WO2019207008 
Protection Patent application published
Year Protection Granted 2019
Licensed No
Impact currently licensing options are being worked out, One innovateUK grant was funded (under icure programme) to explore market for this work.
 
Title Electrochemical device and method for its manufacture 
Description Electrochemical device and method for its manufacture 
IP Reference GB1901078.4 
Protection Patent application published
Year Protection Granted 2019
Licensed No
Impact Development of high-performance electrochemical device.
 
Title SENSOR AND DEVICES INCORPORATING SENSORS 
Description This invention relates to a touch sensor having a layered structure, the layers including: a substrate; and a touch-sensitive layer formed of single-layer graphene and having a plurality of coplanar electrodes formed therein. Embodiments of the touch sensor are flexible and stretchable, making them suitable for use as an artificial skin. Further embodiments of the touch sensor are also capable of sensing pressure as well as touch. Further embodiments are substantially transparent and can therefore include a photovoltaic layer under the touch-sensitive layer which can provide a degree of energy autonomy. Further aspects of the invention provide prosthetic devices having such touch sensors forming a sensitive skin, and a method of manufacturing a touch sensor wherein interdigitated electrodes are cut in single-layer graphene by a blade-cutting process. 
IP Reference WO2018150018 
Protection Patent application published
Year Protection Granted 2018
Licensed No
Impact some companies have already contacted us and discussion are underway for licensing.
 
Title THREE DIMENSIONAL STRUCTURE WITH SENSOR CAPABILITY 
Description This invention relates to a device for sensing interaction with its surrounding environment, the device including: a plurality of sensing points (11); a plurality of detectors (12), each associated with one of said sensing points (11) and located remotely therefrom; a plurality of channels (14) which connect said sensing points (11) to said detectors (12) and provide a communication pathway therebetween; and communication media filling the channels, wherein each detector (12) is in communication with the associated sensing point or points (11) through one of said channels (14) and the medium in said channel (14) is arranged to transmit, transfer or transduce an interaction of the sensing point (11) with its surrounding environment to the detector (12) through the channel (14). The invention also relates to prosthetics which incorporate such devices. The devices according to the present aspect integrate sensing points and sensors within the structure of the device rather than adding an extra sensing layer to the structure and can overcome the traditional problems associated with the wiring when providing sensors on a robot arm or prosthetic. 
IP Reference WO2018210731 
Protection Patent application published
Year Protection Granted 2018
Licensed Commercial In Confidence
Impact This work is being used by Shadow Robot Company to advance the design of robotic hands. Shadow robotics has provided some research support for this purpose. Currently we are discussing the ways to transfer the technology.
 
Description ITU seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Online panel discussion on tactile sensing in robotics - organised by ITU (United Nations). The panel had 5 international experts.
Year(s) Of Engagement Activity 2022
 
Description Public Lecture 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Public Lecture invited by the Royal Philosophical Society of Glasgow
Year(s) Of Engagement Activity 2020
URL https://rsdahiya.com/media/keynoteinvited-talks/
 
Description Queen's Birthday Event 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Queen's Birthday Event, UK Embassy Lisbon, Portugal
Year(s) Of Engagement Activity 2020