Robotics to enhance independence & safety for dementia patients in the home
Lead Research Organisation:
Imperial College London
Abstract
How can interactive ‘social’ robots help improve the lives of people with dementia? We aim to exploit the exciting potential of artificial intelligence (AI) to improve dementia care. We are aiming to develop a family of robotic devices that can engage people living with dementia, helping improve safety in the home and enhancing quality of life.
We will begin by designing social robotic systems that can respond to safety alerts within smart homes. Once triggered, the robot will engage with the individual and act to reduce risks. This may involve directing them to address the hazard – or deploying automated tools that can do so and deciding to call carers or medical support.
We plan to exploit existing commercial technologies (e.g. Amazon Alexa) and grow into development of bespoke devices. We will closely involve people with dementia, family and carers during the design process to define the characteristics of prototype devices and test these to ensure these robots are safe, accessible and enjoyable for people to engage.
We will begin by designing social robotic systems that can respond to safety alerts within smart homes. Once triggered, the robot will engage with the individual and act to reduce risks. This may involve directing them to address the hazard – or deploying automated tools that can do so and deciding to call carers or medical support.
We plan to exploit existing commercial technologies (e.g. Amazon Alexa) and grow into development of bespoke devices. We will closely involve people with dementia, family and carers during the design process to define the characteristics of prototype devices and test these to ensure these robots are safe, accessible and enjoyable for people to engage.
Technical Summary
The UK Dementia Research Institute (UK DRI) is an initiative funded by the Medical Research Council, Alzheimer's Society and Alzheimer's Research UK. Funding details for UK DRI programmes will be added from 2020.
Interactive ‘social’ robots have laboratory established capacity used to engage and stimulate people living with dementia. They can: (a) undertake diverse behaviours; (b) collect rich data; (c) change the environment and (d) respond appropriately in a flexible, task-dependent manner. They provide a way to continuously monitor the home and engage with people to support behaviour or to provide companionship. However, their clinical potential is untapped and development of robotics for dementia care is at an early stage.
We will develop an automated ‘family’ of robotic devices focusing on improving safety in the home and enhancing the quality of life for a person living with dementia. The robotic devices will be designed to be safe, accessible and enjoyable to engage with. A design sprint will be used to define key features and functionality. The team will develop a small number of exemplars providing proof-of-principle. For example, they will develop robotic systems to respond to (a) environmental alerts produced by the Healthy Home (e.g. noting a kitchen spillage or cooker left on); and (b) alerts related to the person with dementia (e.g. responding to signs of agitation or injury). Following an alert, the robot device will engage with the individuals and, guided by information from a smart environment, will act to reduce risks. The simplest solution may be to direct the owner (help to clean a spillage), but it may be possible to deploy automated tools to achieve similar ends. In the case of agitation, for example,, if a cause can be identified such as confusion simple interventions may be effective and interactions through a conversational agent or social robot may be directly beneficial.
A related focus for the programme will be to develop effective methods of communication between people with dementia and robotic devices. The design process will evaluate the impact of variations in features such as configurable visual appearance, voice characteristics, movement, and tactile features that are optimal for the individuals involved, and the team will produce ‘lightweight’ prototypes for testing purposes.).
Main objectives include
1. To produce robotic devices with elementary AI that are capable of interacting with people living with dementia.
2. To integrate robotic devices within smart living environments to monitor and manage the environment for improved safety and quality of life.
3. Draw from user-centred design to define the characteristics of robotic devices suitable for these tasks
Interactive ‘social’ robots have laboratory established capacity used to engage and stimulate people living with dementia. They can: (a) undertake diverse behaviours; (b) collect rich data; (c) change the environment and (d) respond appropriately in a flexible, task-dependent manner. They provide a way to continuously monitor the home and engage with people to support behaviour or to provide companionship. However, their clinical potential is untapped and development of robotics for dementia care is at an early stage.
We will develop an automated ‘family’ of robotic devices focusing on improving safety in the home and enhancing the quality of life for a person living with dementia. The robotic devices will be designed to be safe, accessible and enjoyable to engage with. A design sprint will be used to define key features and functionality. The team will develop a small number of exemplars providing proof-of-principle. For example, they will develop robotic systems to respond to (a) environmental alerts produced by the Healthy Home (e.g. noting a kitchen spillage or cooker left on); and (b) alerts related to the person with dementia (e.g. responding to signs of agitation or injury). Following an alert, the robot device will engage with the individuals and, guided by information from a smart environment, will act to reduce risks. The simplest solution may be to direct the owner (help to clean a spillage), but it may be possible to deploy automated tools to achieve similar ends. In the case of agitation, for example,, if a cause can be identified such as confusion simple interventions may be effective and interactions through a conversational agent or social robot may be directly beneficial.
A related focus for the programme will be to develop effective methods of communication between people with dementia and robotic devices. The design process will evaluate the impact of variations in features such as configurable visual appearance, voice characteristics, movement, and tactile features that are optimal for the individuals involved, and the team will produce ‘lightweight’ prototypes for testing purposes.).
Main objectives include
1. To produce robotic devices with elementary AI that are capable of interacting with people living with dementia.
2. To integrate robotic devices within smart living environments to monitor and manage the environment for improved safety and quality of life.
3. Draw from user-centred design to define the characteristics of robotic devices suitable for these tasks
Publications
Castillo CSM
(2021)
Wearable MMG-Plus-One Armband: Evaluation of Normal Force on Mechanomyography (MMG) to Enhance Human-Machine Interfacing.
in IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society
Castillo CSM
(2022)
Synergistic Upper-Limb Functional Muscle Connectivity Using Acoustic Mechanomyography.
in IEEE transactions on bio-medical engineering
Caulcrick C
(2021)
Model Predictive Control for Human-Centred Lower Limb Robotic Assistance
in IEEE Transactions on Medical Robotics and Bionics
Caulcrick C
(2021)
Human Joint Torque Modelling With MMG and EMG During Lower Limb Human-Exoskeleton Interaction
in IEEE Robotics and Automation Letters
Formstone L
(2021)
Quantification of Motor Function Post-Stroke Using Novel Combination of Wearable Inertial and Mechanomyographic Sensors.
in IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society
Gardner M
(2020)
A Multimodal Intention Detection Sensor Suite for Shared Autonomy of Upper-Limb Robotic Prostheses.
in Sensors (Basel, Switzerland)
Huo W
(2020)
A Heterogeneous Sensing Suite for Multisymptom Quantification of Parkinson's Disease.
in IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society
Huo W
(2022)
Impedance Modulation Control of a Lower-Limb Exoskeleton to Assist Sit-to-Stand Movements
in IEEE Transactions on Robotics
Lima M
(2022)
Conversational Affective Social Robots for Ageing and Dementia Support
in IEEE Transactions on Cognitive and Developmental Systems
| Description | A Telemedicine Parkinson's Disease Assessment System for Home-based Monitoring |
| Amount | £75,000 (GBP) |
| Funding ID | 58830 |
| Organisation | Innovate UK |
| Sector | Public |
| Country | United Kingdom |
| Start | 05/2020 |
| End | 05/2021 |
| Description | Closed-Loop Electronic Stimulation (ES)- Mechanomyogram Sensor (MMG) System for Passive Tremor Suppression Treatment |
| Amount | £1,000,000 (GBP) |
| Funding ID | NIHR202133 |
| Organisation | National Institute for Health Research |
| Sector | Public |
| Country | United Kingdom |
| Start | 03/2021 |
| End | 06/2023 |
| Description | Natural User Interface (NUI) for Assistive Technology (EPSRC Impact Acceleration Account) |
| Amount | £153,000 (GBP) |
| Funding ID | EP/R511547/1 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 03/2019 |
| End | 03/2022 |
| Description | NuRO.sense - An Intelligent wearable to identify, assess and monitor the motor symptoms of neurodegeneration |
| Amount | £540,000 (GBP) |
| Funding ID | 10022189 |
| Organisation | Innovate UK |
| Sector | Public |
| Country | United Kingdom |
| Start | 03/2022 |
| End | 04/2024 |
| Description | Play-Back: motivating core-muscle exercises with wearable sensors, haptics and interactive gaming (MRC Confidence in Concept (CiC) Scheme) |
| Amount | £78,627 (GBP) |
| Organisation | Medical Research Council (MRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 08/2019 |
| End | 10/2020 |
| Description | Psychosocial Robotic Engagement with Persons with Dementia (PwD) |
| Amount | £140,000 (GBP) |
| Organisation | Imperial College London |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 09/2020 |
| End | 09/2024 |
| Description | Rapid Manufacture of Customized Lower Limb Robotic Wearables with Real-Time Biofeedback |
| Amount | £250,000 (GBP) |
| Organisation | United Kingdom Research and Innovation |
| Sector | Public |
| Country | United Kingdom |
| Start | 09/2020 |
| End | 03/2024 |
| Description | Wearable Biofeedback Support for Pain Management |
| Amount | £125,000 (GBP) |
| Organisation | United Kingdom Research and Innovation |
| Sector | Public |
| Country | United Kingdom |
| Start | 09/2019 |
| End | 03/2023 |
| Description | Parkinson's Monitoring in Low-Middle Income Countries (LMICs) |
| Organisation | United International University |
| Country | Bangladesh |
| Sector | Academic/University |
| PI Contribution | Algorithm development, research context, hardware/sensor design |
| Collaborator Contribution | Algorithm development, data analysis, application in LMICs |
| Impact | MSR Sajal, MT Ehsan, R Vaidyanathan, S Wang, T Aziz, KA Al Mamun, "Telemonitoring Parkinson's disease using machine learning by combining tremor and voice analysis", Brain Informatics, 7, 12, doi: 10.1186/s40708-020-00113-1, Oct 2020 F S Rawnaque, K M Rahman, S F Anwar, R Vaidyanathan, T Chau, F Sarker, K Abdullah Al Mamun, "Technological advancements and opportunities in Neuromarketing: a systematic review", Brain Informatics, 7, 1, 10 pp, doi: 10.1186/s40708-020-00109-x, 2020 |
| Start Year | 2019 |
| Description | Robotics and Wearable Systems for Parkinson's Disease Monitoring |
| Organisation | Medical Research Council (MRC) |
| Department | MRC Brain Network Dynamics Unit at the University of Oxford (BNDU) |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | Parkinson's disease (PD) is the second most common neurodegenerative disease worldwide. Bespoke subject-specific treatment (medication or deep brain stimulation (DBS)) is critical for management, yet depends on a precise assessment cardinal PD symptoms - bradykinesia, rigidity and tremor. Clinician diagnosis is the basis of treatment, yet it allows only a cross-sectional assessment of symptoms which can vary on an hourly basis and is liable to inter- and intra-rater subjectivity across human examin- ers. Automated symptomatic assessment has attracted significant interest to optimise treatment regimens between clinician visits, however, no wearable has the capacity to simultaneously assess all three cardinal symptoms. Challenges in the measurement of rigidity, mapping muscle activity out-of-clinic and sensor fusion have inhibited translation. In this study, we address these issues through a novel wearable sensor system and learning algorithms. We further address major thrusts in brain machine interface (BMI) is to establish a robust, bi-directional direct link between the central nervous system (CNS) and artificial devices (e.g. medical implants, artificial organs, neural stimulators, robotic limbs, etc) through the basal ganglia with PD patients in possession of implants associated with deep brain stimulation. Our team has developed wearable sensors and algorithms to map neural activity for diagnosis of PD and human-robot interface. |
| Collaborator Contribution | The University of Oxford and MRC Brain Network Dynamics Group hosts neurological and clinical research aspects of this project. They have directed medical experiments of the hardware on PD patients and deep brain recording experiments as correlated to motor control. |
| Impact | W Huo, P Angeles, S Wilson, Y F Tai, N Pavese, M Hu, S Wilson, R Vaidyanathan, "A Heterogeneous Sensing Suite for Multisymptom Quantification of Parkinson's Disease," IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28, 6, pp 1397-1406, doi: 10.1109/TNSRE.2020.2978197, 2020 MSR Sajal, MT Ehsan, R Vaidyanathan, S Wang, T Aziz, KA Al Mamun, "Telemonitoring Parkinson's disease using machine learning by combining tremor and voice analysis", Brain Informatics, 7, 12, doi: 10.1186/s40708-020-00113-1, Oct 2020 T Martineau, S He, R Vaidyanathan, P Brown, H Tan, "Optimizing Time-Frequency Feature Extraction and Channel Selection through Gradient Backpropagation to Improve Action Decoding Based on Subthalamic Local Field Potentials", 42nd IEEE International Conference on Engineering in Medicine and Biology (EMBC), 4 pp, Montreal, CA (virtual), July 2020 |
| Start Year | 2018 |
| Description | Telemonitoring Parkinson's Disease in Developing Nations |
| Organisation | United International University |
| Country | Bangladesh |
| Sector | Academic/University |
| PI Contribution | With the growing number of the aged population, the number of Parkinson's disease (PD) affected people is also mounting. Unfortunately, due to insufficient resources and awareness in underdeveloped countries, proper and timely PD detection is highly challenged. Besides, all PD patients' symptoms are neither the same nor they all become pronounced at the same stage of the illness. Therefore, this work aims to combine more than one symptom (rest tremor and voice degradation) by collecting data remotely using smartphones and detect PD with the help of a cloud-based machine learning system for telemonitoring the PD patients in the developing countries. Our laboratories have developed sensors and machine learning algorithms for PD that are supporting translation in LMICs through our partners in Bangladesh. |
| Collaborator Contribution | UIU Bangladesh has developed a system that receives rest tremor and vowel phonation data acquired by smartphones with built-in accelerometer and voice recorder sensors. The data are primarily collected from diagnosed PD patients and healthy people for building and optimizing machine learning models that exhibit higher performance. After that, data from newly suspected PD patients are collected, and the trained algorithms are evaluated to detect PD. Based on the majority-vote from those algorithms, PD-detected patients are connected with a nearby neurologist for consultation. Upon receiving patients' feedback after being diagnosed by the neurologist, the system may update the model by retraining using the latest data. Also, the system requests the detected patients periodically to upload new data to track their disease progress. |
| Impact | JOURNAL PAPER: A Dev, N Roy, M K Islam, C Biswas, H U Ahmed, A Amin, F Sarker, R Vaidyanathan, K A. Mamun, "Exploration of EEG-based Depression Biomarkers Identification Techniques and their Applications: A Systematic Review", IEEE Access, r 10.1109/ACCESS.2022.3146711, 10, pp 167569-781, 2022 JOURNAL PAPER: MSR Sajal, MT Ehsan, R Vaidyanathan, S Wang, T Aziz, KA Al Mamun, "Telemonitoring Parkinson's disease using machine learning by combining tremor and voice analysis", Brain Informatics, 7, 12, doi: 10.1186/s40708-020-00113-1, Oct 2020 JOURNAL PAPER: F S Rawnaque, K M Rahman, S F Anwar, R Vaidyanathan, T Chau, F Sarker, K Abdullah Al Mamun, "Technological advancements and opportunities in Neuromarketing: a systematic review", Brain Informatics, 7, 1, 10 pp, doi: 10.1186/s40708-020-00109-x, Sept 2020 JOURNAL PAPER: K A Mamun, M Mace, M Lutman, J Stein, X Liu, T Aziz, R Vaidyanathan, S Wang, "Movement decoding using neural synchronisation and inter-hemispheric connectivity from deep brain local field potentials", Journal of Neural Engineering, 12, 5, pp 1-18, 2015 CONFERENCE PAPER: W Farzana, F Sarker, R Vaidyanathan, T Chau, KA Mamun, "Communication Support Utilizing AAC for Verbally Challenged Children in Developing Countries During COVID-19 Pandemic", International Conference on Human-Computer Interaction, pp 39-50, Copenhagen, Denmark, July 2020 CONFERENCE PAPER: Sajal M.S.R., Ehsan M.T., Vaidyanathan R., Wang S., Aziz T., Mamun K.A, "UPDRS Label Assignment by Analyzing Accelerometer Sensor Data Collected from Conventional Smartphones". In: Mahmud M., Vassanelli S., Kaiser M.S., Zhong N. (eds) Brain Informatics. BI 2020. Lecture Notes in Computer Science, vol 12241. Springer, Cham. doi.org/10.1007/978-3-030-59277-6_16, 6pp, 2020 |
| Start Year | 2019 |
| Description | Wearable Systems for Neurological Disorders |
| Organisation | New York University |
| Country | United States |
| Sector | Academic/University |
| PI Contribution | Wearable technology has significant potential to support clinical and home diagnosis of neurological disorders. This collaboration aims to advance novel wearable technology for physiological sensing for home support for neurological disorders such as Parkinson's Disease and Stroke, as well as for use of assistive technology such as artificial limbs. |
| Collaborator Contribution | In collaboration we have developed a novel muscle recording sensor suite and fused it with a machine learning infrastructure to diagnose neurological disorders and control artificial limbs. |
| Impact | JOURNAL PAPER: M Gardner, S Mancello, S Wilson, B C Khoo, D Farina, E Burdet, S F Atheshzar, R Vaidyanathan, "A Multimodal Intention Detection Sensor Suite for Shared Autonomy of Upper-Limb Robotic Prostheses", Sensors, 20, 21, 6097, doi: 10.3390/s20216097, 2020 |
| Start Year | 2018 |
| Title | BIOMECHANICAL ACTIVITY MONITORING |
| Description | A wearable sensor apparatus comprises a motion sensor configured to sense two or three dimensional movement and orientation of the sensor and a vibration sensor configured to sense acoustic vibrations. The apparatus includes means for attaching the motion sensor and the vibration sensor to a body. The sensor apparatus enables long term monitoring of mechanomyographic muscle activity in combination with body motion for a number of applications. |
| IP Reference | US2019320944 |
| Protection | Patent granted |
| Year Protection Granted | 2019 |
| Licensed | Yes |
| Impact | This patent (issued July 2019) was licenced to Serg Technologies in Nov 2019. Serg has completed two rounds of capital fundraising at a valuation of several million GBP. They have prototyped a first-generation Parkinson's wearable system based on the sensing system in the patent. A product prototype has been created with testing underway (2021 onwards) as a home telemedicine tool. |
| Title | Neuro - remote monitoring of Parkinson's Disease |
| Description | Research from the Biomechatronics Laboratory, Imperial college London has led to NuRO - a wearable and artificial intelligence system for remote diagnosis of symptoms of neuromuscular diseases (NDMDs). The revolutionary platform provides early diagnosis, continuous and passive monitoring, and non-invasive treatment of Parkinsonian symptoms, in clinics, at home and remotely. It is now (Nov 2021) being tested in the homes of approximately 50 Parkinson's Patients. |
| Type | Diagnostic Tool - Non-Imaging |
| Current Stage Of Development | Refinement. Clinical |
| Year Development Stage Completed | 2021 |
| Development Status | Under active development/distribution |
| Impact | The remote wearable system for Parkinson's disease has triggered approximately £1.5 mil in commercial/clinical development funding. |
| URL | https://sergtechnologies.com/ |
| Company Name | SERG TECHNOLOGIES LIMITED |
| Description | SERG Technologies is an Imperial College deep tech spin-out company, based in London, England. It was incorporated in February 2019 with a mission to revolutionise the way humans interact with technology by re-inventing human to machine interface and develop anthropocentric solutions in the healthcare field that will improve the quality of life for millions of people. The company's shareholders are its Founders: Ravi Vaidyanathan, Samuel Wilson, Alex Lewis and Christos Kapatos, with the venture firm Imperial Innovations maintaining a stake on behalf of Imperial College, and Velocity-Partners VC being the lead-investor of the pre-seed round that was closed in Jan 2020. Two independent Business-Angels are also shareholders. SERG is working closely with NHS hospitals and leading UK private clinics as well as with market-leading companies in the prosthetics and orthotics, medical and homecare fields, creating partnerships, raising awareness and promoting its technology. Our technology was awarded both the "Most Promising Innovation in Robotics" award by the UK Institution of Engineering and Technology (IET) in 2016, as well as the "National Health Service (NHS) Innovation Challenge Award". It has been featured for innovation by BBC and ITV, and the Team has been invited for presentation at 10 Downing Street. |
| Year Established | 2019 |
| Impact | The company cleared pre-seed commercial investment funding in Jan 2020. In Oct 2020 they prototyped a Parkinson's telemedicine system to be used in the homes of patients as a precursor to a large trial. It has been deployed in several homes for telemedicine from Jan 2021- onwards. |
| Website | https://sergtechnologies.com/index.php |
| Description | International Press Feature: Research Highlighted in Times of India National Newspaper |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | Dementia robot testing in India covered for innovation by India National Newspaper on National Alzheimer's Day, circulation several million |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://timesofindia.indiatimes.com/city/chennai/world-alzheimers-day-in-their-lonely-world-dolls-br... |
| Description | Parkinson's, AI and me (BBC News Feature) |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | BBC lead technology correspondent wrote a feature of his own experience using our laboratories Parkinson's Disease telemedicine system. It is estimated to have reached an audience of 1 mil + people. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.bbc.co.uk/news/technology-57342760 |
| Description | Public Talk and Roundtable Discussion (Royal Academy of Arts 2021 Biannale Research Lecture and Roundtable) |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | Royal Academy of Arts 2021 Biannale Research Lecture and Roundtable: How do you give form to sentimental machines? An Affective Futures Round: Dementia-robotics research featured by Royal Academy of Arts in 2021 Biannele Research Showcase, Feb 2021 |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://research-biennale.rca.ac.uk/events/ |
| Description | Public lecture, workshop, roundtable on artificial intelligence in law |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | 100+ International law students attended a lecture and roundtable discussion by Dr Maitreyee Wairagkar (Researcher, UK Dementia Research Institute Care, Research and Technology Centre (DRI-CR&T)) on the impact of artificial intelligence in law, with context on automation and care for the elderly. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.linkedin.com/in/ravi-vaidyanathan-96267016/detail/recent-activity/?msgControlName=reply_... |
| Description | Research and Staff Highlighted: Royal Academy Engineering (RAEng) Engagement Feature 'This is Engineering ' |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | Human-robotic interface of PhD student Chris Caulcrick selected to be featured by the RAEng as flagship This is Engineering campaign; nationally broadcast to inspire younger people to consider engineering as a career. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.thisisengineering.org.uk/meet-the-engineers/ |