MODEM - Monitoring Of Dementia using Eye Movements

Lead Research Organisation: Lancaster University
Department Name: Computing & Communications

Abstract

There is mounting evidence that deficits in saccadic and smooth pursuit eye movements are characteristic of dementia. These deficits can be detected in a lab or clinical setting using specialised eye-tracking equipment but this is inconvenient for the patient, costly for the NHS and introduces the risk of sampling bias because clinic visits are inevitably intermittent. The aim of the Monitoring Of Dementia using Eye Movements (MODEM) project is to enable the longitudinal collection of data at low cost and with minimal inconvenience, to provide a novel platform for prognosis and diagnosis of dementia.

We propose to tackle monitoring of disease progression with in-home eye tracking and computational analysis of eye movement embedded with patients' everyday activity. This is an entirely novel approach, and hence high risk. However, it has the potential to lead to major breakthroughs, for three reasons: (i) Eye movement and cognitive health are closely linked, including initial evidence of markers for dementia diagnosis. (ii) Eye trackers are on the verge of a step change from lab instrument to widely deployed sensor, and their adoption for contact-less health monitoring is becoming a realistic proposition. (iii) People/patients use their eyes in daily routines that are visually engaging, and that present rich contexts for collection of information about how their eye movement changes over time, as a function of disease progression.

Our vision is that rather than patients having to attend a clinic or laboratory, eye movement data can be collected in settings where the technology is ambient and peoples' behaviour is relaxed and natural. The target settings are peoples' own homes and care homes. Eye trackers can be placed strategically to observe eye movement in the context of everyday tasks. For example they can be used to track hand-eye coordination in routine tasks such as tea-making for possible signs of change; these might signal cognitive decline long before routines become more obviously affected. Eye trackers can also be deployed interactively. People spend significant amounts of their daily lives as consumers of visual media, especially through TV, which affords interactive stimulation of eye movement. For example, content (e.g. TV programmes) can be designed to elicit behaviours of interest for diagnosis. People can also be provided with active gaze controls for interaction, for instance as alternative to remote control functions of a TV. Use of gaze for control stimulates specific eye movements which can be used for testing. Though beyond the scope project, this could also lead to therapeutic application of the technology. Moreover, as eye trackers are based on cameras and computer vision, this opens up avenues for integration with other vision-based approaches such as analysis of facial expressions, for multimodal cognitive health analysis.

Planned Impact

Beneficiaries include

The NHS will benefit because
- the means for continuous tracking of the progression of dementia in domestic settings has the potential to yield cost savings associated with people periodically attending memory clinics.
- MODEM technology will yield continuous rather than fragmentary data, providing data sets of a size that should improve the fidelity of insights into the progression of dementia in an individual, and when aggregated across the population, provide significant insights into how the condition progresses.

Government and wider society will benefit because MODEM will contribute to the G8 Dementia Challenge's priorities and lead to cost savings and quality of life improvements for dementia sufferers and their carers, allowing (e.g.) mitigating interventions at thepoint where behavioural changes are probable.

Industry has the potential to benefit, particularly:
- The UK video games industry is already set to benefit from on-going advances in eye-tracking technology, but MODEM is likely to lead to unique developments in the integration of gaze and interaction. Using this as the means for intervention in eye-tracking for dementia will have potential analogues that are exploitable for games design.
- TV manufacturers may also benefit from MODEM's integration of camera and display technology, not only using eye-tracking to (e.g.) infer the locus of peoples' attention, but also more generally to e.g. infer who is watching and tailor the display to their favoured volume, brightness, etc. This generalises to other display manufacturers and content providers.

Publications

10 25 50
 
Description Patients with Mild Cognitive Impairment (MCI) are at an increased likelihood of developing AD. This project was first to find that eye tracking can distinguish between patients with the amnesic and the non-amnesic variants of MCI. This provides further support for eye-tracking as a useful diagnostic tool in the assessment of dementia.

The project developed novel eye movement analytics, in particular for analysis of the previously understudied oscillatory movement the eye performs at the end of saccade. This lead to two major findings: first, that post-saccadic oscillations (PSO) are effected by age; secondly, that there is a cross-cultural factor. However, in studying PSO with Dyslexia patients, we found that PSO was not impaired, showing that inhibition is impaired at a higher
level.

The project created three new software packages to support eye movement analysis, and the integration of novel analytics in eye-tracked applications.
Exploitation Route The results have two primary potential uses:

- as a clinical diagnostic tool, perhaps as part of a battery of cognitive tests.

- as a feature that might be built in to an eye-monitoring system which itself might form part of a health monitoring system for older adults living alone.
Sectors Digital/Communication/Information Technologies (including Software),Healthcare

 
Description Software used by other research labs.
First Year Of Impact 2019
Sector Digital/Communication/Information Technologies (including Software),Healthcare
Impact Types Societal

 
Description Games for Life 
Organisation Games for Life
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Funding follow-on project on eye-tracking for attention training
Collaborator Contribution Direct funding
Impact Eye-tracking game demonstrator
Start Year 2017
 
Title EyeMRTK 
Description Eye tracking building blocks for mixed reality application development 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact First eye tracking toolkit for developing gaze application in Unity. 
 
Title PSOVIS: An interactive tool for extracting post-saccadic oscillations from eye movement data 
Description Post-microsaccadic eye movements recorded by high frame-rate pupil-based eye trackers reflect movements of different ocular structures such as deformation of the iris and pupil- eyeball relative movement as well as the dynamic overshoot of the eye globe at the end of each saccade. These Post-Saccadic Oscillations (PSO) exhibit a high degree of reproducibility across saccades and within participants. Therefore in order to study the characteristics of the post-saccadic eye movements, it is often desirable to extract the post-saccadic parts of the recorded saccades and to look at the ending part of all saccades. In order to ease the study- ing of PSO eye movements, a simple tool for extracting PSO signals from the eye movement recordings has been developed. The software application implements functions for extracting, aligning, visualising and finally exporting the PSO signals from eye movement recordings, to be used for post-processing. The code which is written in Python can be download from https://github.com/dmardanbeigi/PSOVIS.git 
Type Of Technology Software 
Year Produced 2017 
Impact The tool has been taken up by researchers for PSO analysis with other data sets. 
 
Title Saccade Machine 
Description SaccadeMachine is an open source software for analyzing Pro-Saccade and Anti-Saccade tasks. The software is written in Python with a simple and user friendly user interface that can be run as a notebook in Jupyter. No programming knowledge is required for using the software! 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact Uptake by other practitioners for development of gaze-based interfaces. 
 
Description Alzheimer's Research UK Conference (Manchester, 2016) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact UK and European leading and emerging leaders in Alzheimer's disease researcher will be attending this important annual conference. We anticipate that there will be new interest and questions arising from the MODEM study, as this has now been adopted on the National Institute for Health Research portfolio and is also featured on the Join Dementia Research NHS website portal.
Year(s) Of Engagement Activity 2016
 
Description Dr Trevor Crawford & Dr Thom Wilcockson were Interviewed for the Joint Dementia Research website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Study participants or study members
Results and Impact JDR is an NHS NIHR website designed to publicise dementia research and help recruit trial participants. Our interview led to a summary article about MoDEM on the JDR site. It has helped us recruit participants for our eye-tracking experiments which are generating large volumes of data that are beginning to translate in to new discoveries.
Year(s) Of Engagement Activity 2016
URL http://news.joindementiaresearch.nihr.ac.uk/can-dementia-diagnosed-eye-movement/
 
Description Eye movements and Neuropsychology - an Invited talk to H2020 SynaNET project given by Dr Trevor Crawford 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The workshop covered current research and topics in the use of behavioural approaches to study brain function, in health and in disease, in model organisms and in humans. The workshop consisted of both lectures and practical sessions, giving both theoretic and practical insight into the application of these approaches. These sessions were delivered by both SynaNET members and other international experts in the field. Participants developed a greater knowledge of the behavioural paradigms used in both model organisms and in humans.
Year(s) Of Engagement Activity 2016
URL http://www.synanet2020.com/activities/workshops/
 
Description Gordon research conference - poster 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Poster presented at Gordon research conference
Year(s) Of Engagement Activity 2017
 
Description Introduction to human eye-tracking - Invited talk at SynaNET, Lancaster, given by Dr Thom Wilcockson 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The workshop covered current research and topics in the use of behavioural approaches to study brain function, in health and in disease, in model organisms and in humans. The workshop consisted of both lectures and practical sessions, giving both theoretic and practical insight into the application of these approaches. These sessions were delivered by both SynaNET members and other international experts in the field. Participants developed a greater knowledge of the behavioural paradigms used in both model organisms and in humans.
Year(s) Of Engagement Activity 2016
URL http://www.synanet2020.com/activities/workshops/
 
Description Invited Participation at Dagstuhl Seminar UGSI 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Week-long workshop on future application of eye tracking and gaze. Two members of the MODEM project were invited, Diako Mardanbegi and myself, and we presented on advances in eye movement analysis and gaze interfaces.
Year(s) Of Engagement Activity 2018
 
Description Invited participation RE@40 workshop at University of Zurich 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Participation in a workshop marking 40 years of Requirements Engineering as a recognised discipline. I was invited to give a talk on the Internet of Things and the challenges it poses for Requirements Engineering. Much of my talk was focused on IoT for medical and health monitoring and diagnostics. The talk fed into wider discussions about challenges facing RE and should eventually result in a publication.
Year(s) Of Engagement Activity 2017
URL https://www.ifi.uzh.ch/en/rerg/RE-40.html
 
Description Keynote, The COGAIN Symposium 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Keynote to attendees of the 2017 Symposium by the COGAIN Association, held at European Conference on Eye Movement
Year(s) Of Engagement Activity 2017
URL http://cogain2017.cogain.org/?index
 
Description Participation in EPSRC Healthy Nation video 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact The video featured a brief section on MoDEM, it's aims and envisioned technologies.
Year(s) Of Engagement Activity 2016
URL https://www.youtube.com/watch?v=EXPf9uau4Do
 
Description Poster at Alzheimer's Research UK (March 2016): MoDEM: Monitoring of dementia using eye movements 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The poster presentation gave MoDEM a presence at the Alzheimer's Research UK's annual conference in 2016. Many attendees representing various stakeholder types expressed in terse in MoDEM.
Year(s) Of Engagement Activity 2016
URL https://alzres.biomedcentral.com/articles/10.1186/s13195-016-0203-0
 
Description Presentation at ECEM: EyeGrip as a tool for assessing dementia 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation on EyeGrip at ECEM, the European conference on Eye movement
Year(s) Of Engagement Activity 2017
URL https://bop.unibe.ch/index.php/JEMR/article/view/4182
 
Description Presentation on PSOVIS at COGAIN symposium 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation of a new tool developed, now available open source.
Year(s) Of Engagement Activity 2017
URL http://cogain2017.cogain.org/?schedule
 
Description Royal College of Physicians mature healthcare IT principles workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The workshop was in part a response to Professor Robert Watcher's report "Making IT Work: Harnessing the Power of Health Information Technology to Improve Care in England", which recommended principles for the adoption of digital technology by the NHS. However, Watcher took a traditional approach: fixing the issues that were presented to it with the tools currently in use. It assumed the benefits we experience from digital in our daily lives could simply transfer to healthcare.
Rather than critique Watcher, it was realized the report could be used as a platform to strengthen systemic and strategic thinking not just for the next few years, but to navigate forward:

• What were the critical questions that Watcher did not ask?
• Thinking about possible futures for the NHS, what are the biggest questions we should be asking?
• How can we devise powerful and enduring guiding principles for the development and adoption of digital healthcare that would enable us to be more strategic and discerning in digitisation or in developing "IT strategies"?
Year(s) Of Engagement Activity 2017
 
Description UBISS summer school 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact 13 Postgraduate students from around the world attended a week-long workshop on eye tracking and eye movement analysis, influencing postgraduate research internationally.
Year(s) Of Engagement Activity 2016