MODEM - Monitoring Of Dementia using Eye Movements
Lead Research Organisation:
Lancaster University
Department Name: Computing & Communications
Abstract
There is mounting evidence that deficits in saccadic and smooth pursuit eye movements are characteristic of dementia. These deficits can be detected in a lab or clinical setting using specialised eye-tracking equipment but this is inconvenient for the patient, costly for the NHS and introduces the risk of sampling bias because clinic visits are inevitably intermittent. The aim of the Monitoring Of Dementia using Eye Movements (MODEM) project is to enable the longitudinal collection of data at low cost and with minimal inconvenience, to provide a novel platform for prognosis and diagnosis of dementia.
We propose to tackle monitoring of disease progression with in-home eye tracking and computational analysis of eye movement embedded with patients' everyday activity. This is an entirely novel approach, and hence high risk. However, it has the potential to lead to major breakthroughs, for three reasons: (i) Eye movement and cognitive health are closely linked, including initial evidence of markers for dementia diagnosis. (ii) Eye trackers are on the verge of a step change from lab instrument to widely deployed sensor, and their adoption for contact-less health monitoring is becoming a realistic proposition. (iii) People/patients use their eyes in daily routines that are visually engaging, and that present rich contexts for collection of information about how their eye movement changes over time, as a function of disease progression.
Our vision is that rather than patients having to attend a clinic or laboratory, eye movement data can be collected in settings where the technology is ambient and peoples' behaviour is relaxed and natural. The target settings are peoples' own homes and care homes. Eye trackers can be placed strategically to observe eye movement in the context of everyday tasks. For example they can be used to track hand-eye coordination in routine tasks such as tea-making for possible signs of change; these might signal cognitive decline long before routines become more obviously affected. Eye trackers can also be deployed interactively. People spend significant amounts of their daily lives as consumers of visual media, especially through TV, which affords interactive stimulation of eye movement. For example, content (e.g. TV programmes) can be designed to elicit behaviours of interest for diagnosis. People can also be provided with active gaze controls for interaction, for instance as alternative to remote control functions of a TV. Use of gaze for control stimulates specific eye movements which can be used for testing. Though beyond the scope project, this could also lead to therapeutic application of the technology. Moreover, as eye trackers are based on cameras and computer vision, this opens up avenues for integration with other vision-based approaches such as analysis of facial expressions, for multimodal cognitive health analysis.
We propose to tackle monitoring of disease progression with in-home eye tracking and computational analysis of eye movement embedded with patients' everyday activity. This is an entirely novel approach, and hence high risk. However, it has the potential to lead to major breakthroughs, for three reasons: (i) Eye movement and cognitive health are closely linked, including initial evidence of markers for dementia diagnosis. (ii) Eye trackers are on the verge of a step change from lab instrument to widely deployed sensor, and their adoption for contact-less health monitoring is becoming a realistic proposition. (iii) People/patients use their eyes in daily routines that are visually engaging, and that present rich contexts for collection of information about how their eye movement changes over time, as a function of disease progression.
Our vision is that rather than patients having to attend a clinic or laboratory, eye movement data can be collected in settings where the technology is ambient and peoples' behaviour is relaxed and natural. The target settings are peoples' own homes and care homes. Eye trackers can be placed strategically to observe eye movement in the context of everyday tasks. For example they can be used to track hand-eye coordination in routine tasks such as tea-making for possible signs of change; these might signal cognitive decline long before routines become more obviously affected. Eye trackers can also be deployed interactively. People spend significant amounts of their daily lives as consumers of visual media, especially through TV, which affords interactive stimulation of eye movement. For example, content (e.g. TV programmes) can be designed to elicit behaviours of interest for diagnosis. People can also be provided with active gaze controls for interaction, for instance as alternative to remote control functions of a TV. Use of gaze for control stimulates specific eye movements which can be used for testing. Though beyond the scope project, this could also lead to therapeutic application of the technology. Moreover, as eye trackers are based on cameras and computer vision, this opens up avenues for integration with other vision-based approaches such as analysis of facial expressions, for multimodal cognitive health analysis.
Planned Impact
Beneficiaries include
The NHS will benefit because
- the means for continuous tracking of the progression of dementia in domestic settings has the potential to yield cost savings associated with people periodically attending memory clinics.
- MODEM technology will yield continuous rather than fragmentary data, providing data sets of a size that should improve the fidelity of insights into the progression of dementia in an individual, and when aggregated across the population, provide significant insights into how the condition progresses.
Government and wider society will benefit because MODEM will contribute to the G8 Dementia Challenge's priorities and lead to cost savings and quality of life improvements for dementia sufferers and their carers, allowing (e.g.) mitigating interventions at thepoint where behavioural changes are probable.
Industry has the potential to benefit, particularly:
- The UK video games industry is already set to benefit from on-going advances in eye-tracking technology, but MODEM is likely to lead to unique developments in the integration of gaze and interaction. Using this as the means for intervention in eye-tracking for dementia will have potential analogues that are exploitable for games design.
- TV manufacturers may also benefit from MODEM's integration of camera and display technology, not only using eye-tracking to (e.g.) infer the locus of peoples' attention, but also more generally to e.g. infer who is watching and tailor the display to their favoured volume, brightness, etc. This generalises to other display manufacturers and content providers.
The NHS will benefit because
- the means for continuous tracking of the progression of dementia in domestic settings has the potential to yield cost savings associated with people periodically attending memory clinics.
- MODEM technology will yield continuous rather than fragmentary data, providing data sets of a size that should improve the fidelity of insights into the progression of dementia in an individual, and when aggregated across the population, provide significant insights into how the condition progresses.
Government and wider society will benefit because MODEM will contribute to the G8 Dementia Challenge's priorities and lead to cost savings and quality of life improvements for dementia sufferers and their carers, allowing (e.g.) mitigating interventions at thepoint where behavioural changes are probable.
Industry has the potential to benefit, particularly:
- The UK video games industry is already set to benefit from on-going advances in eye-tracking technology, but MODEM is likely to lead to unique developments in the integration of gaze and interaction. Using this as the means for intervention in eye-tracking for dementia will have potential analogues that are exploitable for games design.
- TV manufacturers may also benefit from MODEM's integration of camera and display technology, not only using eye-tracking to (e.g.) infer the locus of peoples' attention, but also more generally to e.g. infer who is watching and tailor the display to their favoured volume, brightness, etc. This generalises to other display manufacturers and content providers.
Publications
Wilcockson T
(2017)
[P3-439]: MONITORING OF DEMENTIA USING EYE MOVEMENTS: DETECTING COGNITIVE DECLINE USING EYE MOVEMENT SCANPATHS
in Alzheimer's & Dementia
Mardanbegi D
(2020)
A comparison of post-saccadic oscillations in European-Born and China-Born British University Undergraduates.
in PloS one
Wilcockson TDW
(2019)
Abnormalities of saccadic eye movements in dementia due to Alzheimer's disease and mild cognitive impairment.
in Aging
Polden M
(2021)
Active visual inhibition is preserved in the presence of a distracter: A cross-cultural, ageing and dementia study.
in Cortex; a journal devoted to the study of the nervous system and behavior
Crawford TJ
(2016)
Distinguishing between impairments of working memory and inhibitory control in cases of early dementia.
in Neuropsychologia
Mardanbegi D
(2018)
Effect of aging on post-saccadic oscillations.
in Vision research
Crawford TJ
(2017)
Eye Gaze and Aging: Selective and Combined Effects of Working Memory and Inhibitory Control.
in Frontiers in human neuroscience
Polden M
(2023)
Eye Movement Latency Coefficient of Variation as a Predictor of Cognitive Impairment: An Eye Tracking Study of Cognitive Impairment.
in Vision (Basel, Switzerland)
Mardanbegi D
(2019)
EyeMRTK
Mardanbegi D
(2019)
EyeSeeThrough: Unifying Tool Selection and Application in Virtual Environments
Description | Patients with Mild Cognitive Impairment (MCI) are at an increased likelihood of developing AD. This project was first to find that eye tracking can distinguish between patients with the amnesic and the non-amnesic variants of MCI. This provides further support for eye-tracking as a useful diagnostic tool in the assessment of dementia. The project developed novel eye movement analytics, in particular for analysis of the previously understudied oscillatory movement the eye performs at the end of saccade. This lead to two major findings: first, that post-saccadic oscillations (PSO) are effected by age; secondly, that there is a cross-cultural factor. However, in studying PSO with Dyslexia patients, we found that PSO was not impaired, showing that inhibition is impaired at a higher level. The project created three new software packages to support eye movement analysis, and the integration of novel analytics in eye-tracked applications. |
Exploitation Route | The results have two primary potential uses: - as a clinical diagnostic tool, perhaps as part of a battery of cognitive tests. - as a feature that might be built in to an eye-monitoring system which itself might form part of a health monitoring system for older adults living alone. |
Sectors | Digital/Communication/Information Technologies (including Software) Healthcare |
Description | Software used by other research labs. |
First Year Of Impact | 2019 |
Sector | Digital/Communication/Information Technologies (including Software),Healthcare |
Impact Types | Societal |
Description | Games for Life |
Organisation | Games for Life |
Country | United Kingdom |
Sector | Charity/Non Profit |
PI Contribution | Funding follow-on project on eye-tracking for attention training |
Collaborator Contribution | Direct funding |
Impact | Eye-tracking game demonstrator |
Start Year | 2017 |
Title | EyeMRTK |
Description | Eye tracking building blocks for mixed reality application development |
Type Of Technology | Software |
Year Produced | 2019 |
Open Source License? | Yes |
Impact | First eye tracking toolkit for developing gaze application in Unity. |
Title | PSOVIS: An interactive tool for extracting post-saccadic oscillations from eye movement data |
Description | Post-microsaccadic eye movements recorded by high frame-rate pupil-based eye trackers reflect movements of different ocular structures such as deformation of the iris and pupil- eyeball relative movement as well as the dynamic overshoot of the eye globe at the end of each saccade. These Post-Saccadic Oscillations (PSO) exhibit a high degree of reproducibility across saccades and within participants. Therefore in order to study the characteristics of the post-saccadic eye movements, it is often desirable to extract the post-saccadic parts of the recorded saccades and to look at the ending part of all saccades. In order to ease the study- ing of PSO eye movements, a simple tool for extracting PSO signals from the eye movement recordings has been developed. The software application implements functions for extracting, aligning, visualising and finally exporting the PSO signals from eye movement recordings, to be used for post-processing. The code which is written in Python can be download from https://github.com/dmardanbeigi/PSOVIS.git |
Type Of Technology | Software |
Year Produced | 2017 |
Impact | The tool has been taken up by researchers for PSO analysis with other data sets. |
Title | Saccade Machine |
Description | SaccadeMachine is an open source software for analyzing Pro-Saccade and Anti-Saccade tasks. The software is written in Python with a simple and user friendly user interface that can be run as a notebook in Jupyter. No programming knowledge is required for using the software! |
Type Of Technology | Software |
Year Produced | 2019 |
Open Source License? | Yes |
Impact | Uptake by other practitioners for development of gaze-based interfaces. |
Description | Alzheimer's Research UK Conference (Manchester, 2016) |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | UK and European leading and emerging leaders in Alzheimer's disease researcher will be attending this important annual conference. We anticipate that there will be new interest and questions arising from the MODEM study, as this has now been adopted on the National Institute for Health Research portfolio and is also featured on the Join Dementia Research NHS website portal. |
Year(s) Of Engagement Activity | 2016 |
Description | Dr Trevor Crawford & Dr Thom Wilcockson were Interviewed for the Joint Dementia Research website |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Study participants or study members |
Results and Impact | JDR is an NHS NIHR website designed to publicise dementia research and help recruit trial participants. Our interview led to a summary article about MoDEM on the JDR site. It has helped us recruit participants for our eye-tracking experiments which are generating large volumes of data that are beginning to translate in to new discoveries. |
Year(s) Of Engagement Activity | 2016 |
URL | http://news.joindementiaresearch.nihr.ac.uk/can-dementia-diagnosed-eye-movement/ |
Description | Eye movements and Neuropsychology - an Invited talk to H2020 SynaNET project given by Dr Trevor Crawford |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Other audiences |
Results and Impact | The workshop covered current research and topics in the use of behavioural approaches to study brain function, in health and in disease, in model organisms and in humans. The workshop consisted of both lectures and practical sessions, giving both theoretic and practical insight into the application of these approaches. These sessions were delivered by both SynaNET members and other international experts in the field. Participants developed a greater knowledge of the behavioural paradigms used in both model organisms and in humans. |
Year(s) Of Engagement Activity | 2016 |
URL | http://www.synanet2020.com/activities/workshops/ |
Description | Gordon research conference - poster |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Poster presented at Gordon research conference |
Year(s) Of Engagement Activity | 2017 |
Description | Introduction to human eye-tracking - Invited talk at SynaNET, Lancaster, given by Dr Thom Wilcockson |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | The workshop covered current research and topics in the use of behavioural approaches to study brain function, in health and in disease, in model organisms and in humans. The workshop consisted of both lectures and practical sessions, giving both theoretic and practical insight into the application of these approaches. These sessions were delivered by both SynaNET members and other international experts in the field. Participants developed a greater knowledge of the behavioural paradigms used in both model organisms and in humans. |
Year(s) Of Engagement Activity | 2016 |
URL | http://www.synanet2020.com/activities/workshops/ |
Description | Invited Participation at Dagstuhl Seminar UGSI |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Week-long workshop on future application of eye tracking and gaze. Two members of the MODEM project were invited, Diako Mardanbegi and myself, and we presented on advances in eye movement analysis and gaze interfaces. |
Year(s) Of Engagement Activity | 2018 |
Description | Invited participation RE@40 workshop at University of Zurich |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Participation in a workshop marking 40 years of Requirements Engineering as a recognised discipline. I was invited to give a talk on the Internet of Things and the challenges it poses for Requirements Engineering. Much of my talk was focused on IoT for medical and health monitoring and diagnostics. The talk fed into wider discussions about challenges facing RE and should eventually result in a publication. |
Year(s) Of Engagement Activity | 2017 |
URL | https://www.ifi.uzh.ch/en/rerg/RE-40.html |
Description | Keynote, The COGAIN Symposium |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Keynote to attendees of the 2017 Symposium by the COGAIN Association, held at European Conference on Eye Movement |
Year(s) Of Engagement Activity | 2017 |
URL | http://cogain2017.cogain.org/?index |
Description | Participation in EPSRC Healthy Nation video |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Media (as a channel to the public) |
Results and Impact | The video featured a brief section on MoDEM, it's aims and envisioned technologies. |
Year(s) Of Engagement Activity | 2016 |
URL | https://www.youtube.com/watch?v=EXPf9uau4Do |
Description | Poster at Alzheimer's Research UK (March 2016): MoDEM: Monitoring of dementia using eye movements |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | The poster presentation gave MoDEM a presence at the Alzheimer's Research UK's annual conference in 2016. Many attendees representing various stakeholder types expressed in terse in MoDEM. |
Year(s) Of Engagement Activity | 2016 |
URL | https://alzres.biomedcentral.com/articles/10.1186/s13195-016-0203-0 |
Description | Presentation at ECEM: EyeGrip as a tool for assessing dementia |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Presentation on EyeGrip at ECEM, the European conference on Eye movement |
Year(s) Of Engagement Activity | 2017 |
URL | https://bop.unibe.ch/index.php/JEMR/article/view/4182 |
Description | Presentation on PSOVIS at COGAIN symposium |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Presentation of a new tool developed, now available open source. |
Year(s) Of Engagement Activity | 2017 |
URL | http://cogain2017.cogain.org/?schedule |
Description | Royal College of Physicians mature healthcare IT principles workshop |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Professional Practitioners |
Results and Impact | The workshop was in part a response to Professor Robert Watcher's report "Making IT Work: Harnessing the Power of Health Information Technology to Improve Care in England", which recommended principles for the adoption of digital technology by the NHS. However, Watcher took a traditional approach: fixing the issues that were presented to it with the tools currently in use. It assumed the benefits we experience from digital in our daily lives could simply transfer to healthcare. Rather than critique Watcher, it was realized the report could be used as a platform to strengthen systemic and strategic thinking not just for the next few years, but to navigate forward: • What were the critical questions that Watcher did not ask? • Thinking about possible futures for the NHS, what are the biggest questions we should be asking? • How can we devise powerful and enduring guiding principles for the development and adoption of digital healthcare that would enable us to be more strategic and discerning in digitisation or in developing "IT strategies"? |
Year(s) Of Engagement Activity | 2017 |
Description | UBISS summer school |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | 13 Postgraduate students from around the world attended a week-long workshop on eye tracking and eye movement analysis, influencing postgraduate research internationally. |
Year(s) Of Engagement Activity | 2016 |