Human-Robot Interaction for dementia prevention and research with a focus on cognitive well-being
Lead Research Organisation:
University of Edinburgh
Department Name: Sch of Molecular. Genetics & Pop Health
Abstract
There is an increasing trend in the incidence of dementia, and, particularly Alzheimer's disease. This, along with its current prevalence and the socioeconomic burden it poses, is making its treatment and, even more, its prevention, a major worldwide public health challenge.
This project will investigate the use of novel interactive computing technology, such as human-robot interaction to gather physiological and cognitive data, and new approaches for data analysis to answer a number of interrelated questions that will help dementia prevention.
In order to accomplish this, the first part of the project will consist in learning the state-of-the-art on assistive social robotics: what has been done (classically and recently), what is needed, how would I contribute given my background, what extra steps would I need to take from a theoretical perspective. At the same time, acknowledging and thoroughly understanding the methodological side of these procedures will be key for the project. A number of machine learning and speech processing techniques such as part-of-speech analysis, latent semantic analysis, principal component analysis, Bayesian probability approaches, etc., will be introduced and practised on existing databases using different programming languages. These methods are expected to point out key features of participants' behaviour that may tell us their probability to suffer future dementia, or help us distinguish patients from healthy controls earlier than we do now. Particularly, we might look at speech coherence indexes to test this, first using existing datasets like DementiaBank or Carolina Conversations Collection, and afterwards, using our own data.
The second part of the project will involve designing different experiments on human-robot interaction, gather relevant data and implement the methods mentioned above. For this, we will need to design the experiments according to ethical and clinical constrains, and then we will ask for appropriate advice and carefully decide which robotic platform would be suitable for the planned experiments.
In the third part, we will record large amounts of speech and gesture data, and employ the methods tested during part 1 to analyse these data. We will refine the robot's interaction model iteratively, taking into account the feedback gathered during the experiments in order to improve user engagement and the data collection methods as necessary.
This project will investigate the use of novel interactive computing technology, such as human-robot interaction to gather physiological and cognitive data, and new approaches for data analysis to answer a number of interrelated questions that will help dementia prevention.
In order to accomplish this, the first part of the project will consist in learning the state-of-the-art on assistive social robotics: what has been done (classically and recently), what is needed, how would I contribute given my background, what extra steps would I need to take from a theoretical perspective. At the same time, acknowledging and thoroughly understanding the methodological side of these procedures will be key for the project. A number of machine learning and speech processing techniques such as part-of-speech analysis, latent semantic analysis, principal component analysis, Bayesian probability approaches, etc., will be introduced and practised on existing databases using different programming languages. These methods are expected to point out key features of participants' behaviour that may tell us their probability to suffer future dementia, or help us distinguish patients from healthy controls earlier than we do now. Particularly, we might look at speech coherence indexes to test this, first using existing datasets like DementiaBank or Carolina Conversations Collection, and afterwards, using our own data.
The second part of the project will involve designing different experiments on human-robot interaction, gather relevant data and implement the methods mentioned above. For this, we will need to design the experiments according to ethical and clinical constrains, and then we will ask for appropriate advice and carefully decide which robotic platform would be suitable for the planned experiments.
In the third part, we will record large amounts of speech and gesture data, and employ the methods tested during part 1 to analyse these data. We will refine the robot's interaction model iteratively, taking into account the feedback gathered during the experiments in order to improve user engagement and the data collection methods as necessary.
Publications
De La Fuente S
(2018)
Evaluating cognition through linguistic features
De La Fuente S
(2018)
Detecting cognitive decline through dialogue processing
De La Fuente Garcia S
(2019)
Protocol for a conversation-based analysis study: PREVENT-ED investigates dialogue features that may help predict dementia onset in later life
in BMJ Open
De Frutos-Lucas J
(2019)
Enhancement of posterior brain functional networks in bilingual older adults
in Bilingualism: Language and Cognition
Haider F
(2020)
An Assessment of Paralinguistic Acoustic Features for Detection of Alzheimer's Dementia in Spontaneous Speech
in IEEE Journal of Selected Topics in Signal Processing
Studentship Projects
Project Reference | Relationship | Related To | Start | End | Student Name |
---|---|---|---|---|---|
MR/N013166/1 | 01/10/2016 | 30/09/2025 | |||
1805054 | Studentship | MR/N013166/1 | 01/09/2016 | 29/02/2020 | Sofia De La Fuente |
Title | Map-task for cognitive assessments |
Description | Three map-tasks were first hand-drawn and then digitised to be part of a cognitive assessments. The maps depict an imaginary land with some celtic-inspired landmarks (e.g. standing stones of Soor, or Ring of Soor). Also a storyboard was created to generate a narrative coherent with the maps. In order to see new version of the task, published in January 2020, please visit the following links: MAPS, version 2: https://blogs.bmj.com/bmjopen/map-2/ https://blogs.bmj.com/bmjopen/map_nr/ https://blogs.bmj.com/bmjopen/map_nl/ Instructions and storyline, version 2: https://blogs.bmj.com/bmjopen/instructionsprompts_v2/ https://blogs.bmj.com/bmjopen/storyboard_v2/ |
Type Of Art | Image |
Year Produced | 2018 |
Impact | The creativity behind the imaginary land and the storyline keeps participants engaged, hence facilitating a more naturalistic dialogue and providing the method with ecological validity. The research is in data collection stage at the moment. The artistic product is part and parcel of the experimental design: the participants have one map, the researcher has another one, and they navigate the land cooperatively so that we can collect data in the form of voice recordings. A third map is used to assess their spatial navigation skills. The whole experimental design has been accepted for publication in the BMJ Journal (DOI: http://dx.doi.org/10.1136/bmjopen-2018-026254). This publication will include images of the maps. |
URL | https://bmjopen.bmj.com/content/9/3/e026254 |
Title | Prevent-ED Map-Task |
Description | Cooperative navigation task created specifically for dementia context, in order to elicit spontaneous dialogues in search for linguistic markers of cognitive decline. A second version of the map has been recently published as a response to the initial methodology publication. The main changes aim to enhanced readability, comprehensibility, and generalisability; hence making the task more inclusive for vulnerable populations. Besides, the current version of the map is fully digitised, and therefore, fully modifiable. From a Global Health perspective, this means that landmark names could be translated to any language, or according to any mythology worldwide. Hence, the Prevent-ED map has the potential to serve as a standard task to elicit natural dialogues in this field. In order to see new version of the task, please visit the following links: MAPS, version 2: https://blogs.bmj.com/bmjopen/map-2/ https://blogs.bmj.com/bmjopen/map_nr/ https://blogs.bmj.com/bmjopen/map_nl/ Instructions and storyline, version 2: https://blogs.bmj.com/bmjopen/instructionsprompts_v2/ https://blogs.bmj.com/bmjopen/storyboard_v2/ |
Type Of Material | Improvements to research infrastructure |
Year Produced | 2019 |
Provided To Others? | Yes |
Impact | Researchers in Heriot-Watt University are implementing this method for their own research in collaboration with Alzheimer's Scotland (see their publication: https://arxiv.org/pdf/1909.06644.pdf) |
URL | https://bmjopen.bmj.com/content/9/3/e026254 |
Title | Prevent-ED: spoken dialogue database |
Description | Database consisting in voice recordings of participants from the Prevent Dementia project. These speech data is collected in a dementia context that ensures homogenised dialogue audio in a naturalistic setting, as well as healthy-but-at-risk rather than an already diagnosed cohort, with a comprehensive set of risk biomarkers for comparison and validation. Once quality-checked, these data will be available for experts on acoustic and speech analysis, as well as on natural language processing in order for researchers to identify dialogical features that may help detect dementia onset later in life. |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | No |
Impact | To the best of our knowledge, this is the first collection of speech data in a dementia context that ensures homogenised dialogue audio in a naturalistic setting, as well as healthy-but-at-risk rather than an already diagnosed cohort, with a comprehensive set of risk biomarkers for comparison and validation. One collection is finalised, we will make our high-quality data set available for wider use, including manual annotations and transcriptions of the dialogues. Accessibility will be through and at the discretion of the Prevent project platform. |
URL | https://preventdementia.co.uk/ |
Description | Prevent-ED as a multi-centre study |
Organisation | Imperial College London |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | Prevent-ED study (https://bmjopen.bmj.com/content/9/3/e026254) started in Edinburgh only. Now it has been approved to run in London, in collaboration with the Imperial College of London and the West London Mental Health NHS Trust. Contributions: ethics procedure, material adaptation, travelling to London for data collection, data analysis, results dissemination. |
Collaborator Contribution | Contribution: local coordination and support for participant recruitment, help with ethical procedures. |
Impact | Multi-centre: Edinburgh and London. Multi-disciplinarity: speech technology, neuroimaging, neuropsychology, neurobiology. Outputs so far: task adaptation, ethical approvals (i.e. IRAS, HRA, NRSPCC, Caldicott). Outputs soon-to-be: data collection in London sites. |
Start Year | 2019 |
Description | Prevent-ED as a multi-centre study |
Organisation | West London Mental Health NHS Trust |
Country | United Kingdom |
Sector | Public |
PI Contribution | Prevent-ED study (https://bmjopen.bmj.com/content/9/3/e026254) started in Edinburgh only. Now it has been approved to run in London, in collaboration with the Imperial College of London and the West London Mental Health NHS Trust. Contributions: ethics procedure, material adaptation, travelling to London for data collection, data analysis, results dissemination. |
Collaborator Contribution | Contribution: local coordination and support for participant recruitment, help with ethical procedures. |
Impact | Multi-centre: Edinburgh and London. Multi-disciplinarity: speech technology, neuroimaging, neuropsychology, neurobiology. Outputs so far: task adaptation, ethical approvals (i.e. IRAS, HRA, NRSPCC, Caldicott). Outputs soon-to-be: data collection in London sites. |
Start Year | 2019 |
Description | Centre for Dementia Prevention - Edinburgh Science Festival |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Public/other audiences |
Results and Impact | The Centre for Dementia Prevention run an even in the Edinburgh Science Festival 2017 in which general public went through different interactive stands. This stands explained several aspects of dementia (anatomical, social, psychological). A few volunteers, including me, ensured that all stands run smoothly and help with any arising questions. |
Year(s) Of Engagement Activity | 2017 |
Description | Centre for Dementia Prevention Annual Conference: Preventing Dementia: Advice and Answers |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Patients, carers and/or patient groups |
Results and Impact | Apart from the speakers and debates, there were some stands showing new technology for Dementia Prevention. We were there with Pepper the Robot (Aldeberan), giving a brief interactive presentation. |
Year(s) Of Engagement Activity | 2017 |
URL | http://centrefordementiaprevention.com/2017/10/11/annual-conference-preventing-dementia-advice-answe... |