Robotics to enhance independence & safety for dementia patients in the home

Lead Research Organisation: Imperial College London

Abstract

How can interactive ‘social’ robots help improve the lives of people with dementia? We aim to exploit the exciting potential of artificial intelligence (AI) to improve dementia care. We are aiming to develop a family of robotic devices that can engage people living with dementia, helping improve safety in the home and enhancing quality of life.
We will begin by designing social robotic systems that can respond to safety alerts within smart homes. Once triggered, the robot will engage with the individual and act to reduce risks. This may involve directing them to address the hazard – or deploying automated tools that can do so and deciding to call carers or medical support.
We plan to exploit existing commercial technologies (e.g. Amazon Alexa) and grow into development of bespoke devices. We will closely involve people with dementia, family and carers during the design process to define the characteristics of prototype devices and test these to ensure these robots are safe, accessible and enjoyable for people to engage.

Technical Summary

The UK Dementia Research Institute (UK DRI) is an initiative funded by the Medical Research Council, Alzheimer's Society and Alzheimer's Research UK. Funding details for UK DRI programmes will be added from 2020.

Interactive ‘social’ robots have laboratory established capacity used to engage and stimulate people living with dementia. They can: (a) undertake diverse behaviours; (b) collect rich data; (c) change the environment and (d) respond appropriately in a flexible, task-dependent manner. They provide a way to continuously monitor the home and engage with people to support behaviour or to provide companionship. However, their clinical potential is untapped and development of robotics for dementia care is at an early stage.
We will develop an automated ‘family’ of robotic devices focusing on improving safety in the home and enhancing the quality of life for a person living with dementia. The robotic devices will be designed to be safe, accessible and enjoyable to engage with. A design sprint will be used to define key features and functionality. The team will develop a small number of exemplars providing proof-of-principle. For example, they will develop robotic systems to respond to (a) environmental alerts produced by the Healthy Home (e.g. noting a kitchen spillage or cooker left on); and (b) alerts related to the person with dementia (e.g. responding to signs of agitation or injury). Following an alert, the robot device will engage with the individuals and, guided by information from a smart environment, will act to reduce risks. The simplest solution may be to direct the owner (help to clean a spillage), but it may be possible to deploy automated tools to achieve similar ends. In the case of agitation, for example,, if a cause can be identified such as confusion simple interventions may be effective and interactions through a conversational agent or social robot may be directly beneficial.
A related focus for the programme will be to develop effective methods of communication between people with dementia and robotic devices. The design process will evaluate the impact of variations in features such as configurable visual appearance, voice characteristics, movement, and tactile features that are optimal for the individuals involved, and the team will produce ‘lightweight’ prototypes for testing purposes.).
Main objectives include
1. To produce robotic devices with elementary AI that are capable of interacting with people living with dementia.
2. To integrate robotic devices within smart living environments to monitor and manage the environment for improved safety and quality of life.
3. Draw from user-centred design to define the characteristics of robotic devices suitable for these tasks

Publications

10 25 50
publication icon
Castillo C (2021) Wearable MMG-Plus-One Armband: Evaluation of Normal Force on Mechanomyography (MMG) to Enhance Human-Machine Interfacing in IEEE Transactions on Neural Systems and Rehabilitation Engineering

publication icon
Castillo CSM (2022) Synergistic Upper-Limb Functional Muscle Connectivity Using Acoustic Mechanomyography. in IEEE transactions on bio-medical engineering

publication icon
Caulcrick C (2021) Model Predictive Control for Human-Centred Lower Limb Robotic Assistance in IEEE Transactions on Medical Robotics and Bionics

publication icon
Formstone L (2021) Quantification of Motor Function Post-Stroke Using Novel Combination of Wearable Inertial and Mechanomyographic Sensors. in IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society

publication icon
Huo W (2020) A Heterogeneous Sensing Suite for Multisymptom Quantification of Parkinson's Disease. in IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society

publication icon
Lima M (2021) Conversational Affective Social Robots for Ageing and Dementia Support in IEEE Transactions on Cognitive and Developmental Systems

 
Description A Telemedicine Parkinson's Disease Assessment System for Home-based Monitoring
Amount £75,000 (GBP)
Funding ID 58830 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 05/2020 
End 05/2021
 
Description Closed-Loop Electronic Stimulation (ES)- Mechanomyogram Sensor (MMG) System for Passive Tremor Suppression Treatment
Amount £1,000,000 (GBP)
Funding ID NIHR202133 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 03/2021 
End 06/2023
 
Description Natural User Interface (NUI) for Assistive Technology (EPSRC Impact Acceleration Account)
Amount £153,000 (GBP)
Funding ID EP/R511547/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 03/2019 
End 03/2022
 
Description NuRO.sense - An Intelligent wearable to identify, assess and monitor the motor symptoms of neurodegeneration
Amount £540,000 (GBP)
Funding ID 10022189 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 03/2022 
End 04/2024
 
Description Play-Back: motivating core-muscle exercises with wearable sensors, haptics and interactive gaming (MRC Confidence in Concept (CiC) Scheme)
Amount £78,627 (GBP)
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 08/2019 
End 10/2020
 
Description Psychosocial Robotic Engagement with Persons with Dementia (PwD)
Amount £140,000 (GBP)
Organisation Imperial College London 
Sector Academic/University
Country United Kingdom
Start 09/2020 
End 09/2024
 
Description Rapid Manufacture of Customized Lower Limb Robotic Wearables with Real-Time Biofeedback
Amount £250,000 (GBP)
Organisation United Kingdom Research and Innovation 
Sector Public
Country United Kingdom
Start 09/2020 
End 03/2024
 
Description Wearable Biofeedback Support for Pain Management
Amount £125,000 (GBP)
Organisation United Kingdom Research and Innovation 
Sector Public
Country United Kingdom
Start 09/2019 
End 03/2023
 
Description Parkinson's Monitoring in Low-Middle Income Countries (LMICs) 
Organisation United International University
Country Bangladesh 
Sector Academic/University 
PI Contribution Algorithm development, research context, hardware/sensor design
Collaborator Contribution Algorithm development, data analysis, application in LMICs
Impact MSR Sajal, MT Ehsan, R Vaidyanathan, S Wang, T Aziz, KA Al Mamun, "Telemonitoring Parkinson's disease using machine learning by combining tremor and voice analysis", Brain Informatics, 7, 12, doi: 10.1186/s40708-020-00113-1, Oct 2020 F S Rawnaque, K M Rahman, S F Anwar, R Vaidyanathan, T Chau, F Sarker, K Abdullah Al Mamun, "Technological advancements and opportunities in Neuromarketing: a systematic review", Brain Informatics, 7, 1, 10 pp, doi: 10.1186/s40708-020-00109-x, 2020
Start Year 2019
 
Description Robotics and Wearable Systems for Parkinson's Disease Monitoring 
Organisation Medical Research Council (MRC)
Department MRC Brain Network Dynamics Unit at the University of Oxford (BNDU)
Country United Kingdom 
Sector Public 
PI Contribution Parkinson's disease (PD) is the second most common neurodegenerative disease worldwide. Bespoke subject-specific treatment (medication or deep brain stimulation (DBS)) is critical for management, yet depends on a precise assessment cardinal PD symptoms - bradykinesia, rigidity and tremor. Clinician diagnosis is the basis of treatment, yet it allows only a cross-sectional assessment of symptoms which can vary on an hourly basis and is liable to inter- and intra-rater subjectivity across human examin- ers. Automated symptomatic assessment has attracted significant interest to optimise treatment regimens between clinician visits, however, no wearable has the capacity to simultaneously assess all three cardinal symptoms. Challenges in the measurement of rigidity, mapping muscle activity out-of-clinic and sensor fusion have inhibited translation. In this study, we address these issues through a novel wearable sensor system and learning algorithms. We further address major thrusts in brain machine interface (BMI) is to establish a robust, bi-directional direct link between the central nervous system (CNS) and artificial devices (e.g. medical implants, artificial organs, neural stimulators, robotic limbs, etc) through the basal ganglia with PD patients in possession of implants associated with deep brain stimulation. Our team has developed wearable sensors and algorithms to map neural activity for diagnosis of PD and human-robot interface.
Collaborator Contribution The University of Oxford and MRC Brain Network Dynamics Group hosts neurological and clinical research aspects of this project. They have directed medical experiments of the hardware on PD patients and deep brain recording experiments as correlated to motor control.
Impact W Huo, P Angeles, S Wilson, Y F Tai, N Pavese, M Hu, S Wilson, R Vaidyanathan, "A Heterogeneous Sensing Suite for Multisymptom Quantification of Parkinson's Disease," IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28, 6, pp 1397-1406, doi: 10.1109/TNSRE.2020.2978197, 2020 MSR Sajal, MT Ehsan, R Vaidyanathan, S Wang, T Aziz, KA Al Mamun, "Telemonitoring Parkinson's disease using machine learning by combining tremor and voice analysis", Brain Informatics, 7, 12, doi: 10.1186/s40708-020-00113-1, Oct 2020 T Martineau, S He, R Vaidyanathan, P Brown, H Tan, "Optimizing Time-Frequency Feature Extraction and Channel Selection through Gradient Backpropagation to Improve Action Decoding Based on Subthalamic Local Field Potentials", 42nd IEEE International Conference on Engineering in Medicine and Biology (EMBC), 4 pp, Montreal, CA (virtual), July 2020
Start Year 2018
 
Description Telemonitoring Parkinson's Disease in Developing Nations 
Organisation United International University
Country Bangladesh 
Sector Academic/University 
PI Contribution With the growing number of the aged population, the number of Parkinson's disease (PD) affected people is also mounting. Unfortunately, due to insufficient resources and awareness in underdeveloped countries, proper and timely PD detection is highly challenged. Besides, all PD patients' symptoms are neither the same nor they all become pronounced at the same stage of the illness. Therefore, this work aims to combine more than one symptom (rest tremor and voice degradation) by collecting data remotely using smartphones and detect PD with the help of a cloud-based machine learning system for telemonitoring the PD patients in the developing countries. Our laboratories have developed sensors and machine learning algorithms for PD that are supporting translation in LMICs through our partners in Bangladesh.
Collaborator Contribution UIU Bangladesh has developed a system that receives rest tremor and vowel phonation data acquired by smartphones with built-in accelerometer and voice recorder sensors. The data are primarily collected from diagnosed PD patients and healthy people for building and optimizing machine learning models that exhibit higher performance. After that, data from newly suspected PD patients are collected, and the trained algorithms are evaluated to detect PD. Based on the majority-vote from those algorithms, PD-detected patients are connected with a nearby neurologist for consultation. Upon receiving patients' feedback after being diagnosed by the neurologist, the system may update the model by retraining using the latest data. Also, the system requests the detected patients periodically to upload new data to track their disease progress.
Impact JOURNAL PAPER: A Dev, N Roy, M K Islam, C Biswas, H U Ahmed, A Amin, F Sarker, R Vaidyanathan, K A. Mamun, "Exploration of EEG-based Depression Biomarkers Identification Techniques and their Applications: A Systematic Review", IEEE Access, r 10.1109/ACCESS.2022.3146711, 10, pp 167569-781, 2022 JOURNAL PAPER: MSR Sajal, MT Ehsan, R Vaidyanathan, S Wang, T Aziz, KA Al Mamun, "Telemonitoring Parkinson's disease using machine learning by combining tremor and voice analysis", Brain Informatics, 7, 12, doi: 10.1186/s40708-020-00113-1, Oct 2020 JOURNAL PAPER: F S Rawnaque, K M Rahman, S F Anwar, R Vaidyanathan, T Chau, F Sarker, K Abdullah Al Mamun, "Technological advancements and opportunities in Neuromarketing: a systematic review", Brain Informatics, 7, 1, 10 pp, doi: 10.1186/s40708-020-00109-x, Sept 2020 JOURNAL PAPER: K A Mamun, M Mace, M Lutman, J Stein, X Liu, T Aziz, R Vaidyanathan, S Wang, "Movement decoding using neural synchronisation and inter-hemispheric connectivity from deep brain local field potentials", Journal of Neural Engineering, 12, 5, pp 1-18, 2015 CONFERENCE PAPER: W Farzana, F Sarker, R Vaidyanathan, T Chau, KA Mamun, "Communication Support Utilizing AAC for Verbally Challenged Children in Developing Countries During COVID-19 Pandemic", International Conference on Human-Computer Interaction, pp 39-50, Copenhagen, Denmark, July 2020 CONFERENCE PAPER: Sajal M.S.R., Ehsan M.T., Vaidyanathan R., Wang S., Aziz T., Mamun K.A, "UPDRS Label Assignment by Analyzing Accelerometer Sensor Data Collected from Conventional Smartphones". In: Mahmud M., Vassanelli S., Kaiser M.S., Zhong N. (eds) Brain Informatics. BI 2020. Lecture Notes in Computer Science, vol 12241. Springer, Cham. doi.org/10.1007/978-3-030-59277-6_16, 6pp, 2020
Start Year 2019
 
Description Wearable Systems for Neurological Disorders 
Organisation New York University
Country United States 
Sector Academic/University 
PI Contribution Wearable technology has significant potential to support clinical and home diagnosis of neurological disorders. This collaboration aims to advance novel wearable technology for physiological sensing for home support for neurological disorders such as Parkinson's Disease and Stroke, as well as for use of assistive technology such as artificial limbs.
Collaborator Contribution In collaboration we have developed a novel muscle recording sensor suite and fused it with a machine learning infrastructure to diagnose neurological disorders and control artificial limbs.
Impact JOURNAL PAPER: M Gardner, S Mancello, S Wilson, B C Khoo, D Farina, E Burdet, S F Atheshzar, R Vaidyanathan, "A Multimodal Intention Detection Sensor Suite for Shared Autonomy of Upper-Limb Robotic Prostheses", Sensors, 20, 21, 6097, doi: 10.3390/s20216097, 2020
Start Year 2018
 
Title BIOMECHANICAL ACTIVITY MONITORING 
Description A wearable sensor apparatus comprises a motion sensor configured to sense two or three dimensional movement and orientation of the sensor and a vibration sensor configured to sense acoustic vibrations. The apparatus includes means for attaching the motion sensor and the vibration sensor to a body. The sensor apparatus enables long term monitoring of mechanomyographic muscle activity in combination with body motion for a number of applications. 
IP Reference US2019320944 
Protection Patent granted
Year Protection Granted 2019
Licensed Yes
Impact This patent (issued July 2019) was licenced to Serg Technologies in Nov 2019. Serg has completed two rounds of capital fundraising at a valuation of several million GBP. They have prototyped a first-generation Parkinson's wearable system based on the sensing system in the patent. A product prototype has been created with testing underway (2021 onwards) as a home telemedicine tool.
 
Title Neuro - remote monitoring of Parkinson's Disease 
Description Research from the Biomechatronics Laboratory, Imperial college London has led to NuRO - a wearable and artificial intelligence system for remote diagnosis of symptoms of neuromuscular diseases (NDMDs). The revolutionary platform provides early diagnosis, continuous and passive monitoring, and non-invasive treatment of Parkinsonian symptoms, in clinics, at home and remotely. It is now (Nov 2021) being tested in the homes of approximately 50 Parkinson's Patients. 
Type Diagnostic Tool - Non-Imaging
Current Stage Of Development Refinement. Clinical
Year Development Stage Completed 2021
Development Status Under active development/distribution
Impact The remote wearable system for Parkinson's disease has triggered approximately £1.5 mil in commercial/clinical development funding. 
URL https://sergtechnologies.com/
 
Company Name SERG TECHNOLOGIES LIMITED 
Description SERG Technologies is an Imperial College deep tech spin-out company, based in London, England. It was incorporated in February 2019 with a mission to revolutionise the way humans interact with technology by re-inventing human to machine interface and develop anthropocentric solutions in the healthcare field that will improve the quality of life for millions of people. The company's shareholders are its Founders: Ravi Vaidyanathan, Samuel Wilson, Alex Lewis and Christos Kapatos, with the venture firm Imperial Innovations maintaining a stake on behalf of Imperial College, and Velocity-Partners VC being the lead-investor of the pre-seed round that was closed in Jan 2020. Two independent Business-Angels are also shareholders. SERG is working closely with NHS hospitals and leading UK private clinics as well as with market-leading companies in the prosthetics and orthotics, medical and homecare fields, creating partnerships, raising awareness and promoting its technology. Our technology was awarded both the "Most Promising Innovation in Robotics" award by the UK Institution of Engineering and Technology (IET) in 2016, as well as the "National Health Service (NHS) Innovation Challenge Award". It has been featured for innovation by BBC and ITV, and the Team has been invited for presentation at 10 Downing Street. 
Year Established 2019 
Impact The company cleared pre-seed commercial investment funding in Jan 2020. In Oct 2020 they prototyped a Parkinson's telemedicine system to be used in the homes of patients as a precursor to a large trial. It has been deployed in several homes for telemedicine from Jan 2021- onwards.
Website https://sergtechnologies.com/index.php
 
Description International Press Feature: Research Highlighted in Times of India National Newspaper 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Dementia robot testing in India covered for innovation by India National Newspaper on National Alzheimer's Day, circulation several million
Year(s) Of Engagement Activity 2019
URL https://timesofindia.indiatimes.com/city/chennai/world-alzheimers-day-in-their-lonely-world-dolls-br...
 
Description Parkinson's, AI and me (BBC News Feature) 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact BBC lead technology correspondent wrote a feature of his own experience using our laboratories Parkinson's Disease telemedicine system. It is estimated to have reached an audience of 1 mil + people.
Year(s) Of Engagement Activity 2021
URL https://www.bbc.co.uk/news/technology-57342760
 
Description Public Talk and Roundtable Discussion (Royal Academy of Arts 2021 Biannale Research Lecture and Roundtable) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Royal Academy of Arts 2021 Biannale Research Lecture and Roundtable: How do you give form to sentimental machines? An Affective Futures Round: Dementia-robotics research featured by Royal Academy of Arts in 2021 Biannele Research Showcase, Feb 2021
Year(s) Of Engagement Activity 2021
URL https://research-biennale.rca.ac.uk/events/
 
Description Public lecture, workshop, roundtable on artificial intelligence in law 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact 100+ International law students attended a lecture and roundtable discussion by Dr Maitreyee Wairagkar (Researcher, UK Dementia Research Institute Care, Research and Technology Centre (DRI-CR&T)) on the impact of artificial intelligence in law, with context on automation and care for the elderly.
Year(s) Of Engagement Activity 2021
URL https://www.linkedin.com/in/ravi-vaidyanathan-96267016/detail/recent-activity/?msgControlName=reply_...
 
Description Research and Staff Highlighted: Royal Academy Engineering (RAEng) Engagement Feature 'This is Engineering ' 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Human-robotic interface of PhD student Chris Caulcrick selected to be featured by the RAEng as flagship This is Engineering campaign; nationally broadcast to inspire younger people to consider engineering as a career.
Year(s) Of Engagement Activity 2019
URL https://www.thisisengineering.org.uk/meet-the-engineers/