Multiscale Signal Processing for Next Generation Electroencephalography

Lead Research Organisation: Imperial College London
Department Name: Electrical and Electronic Engineering

Abstract

This proposal seeks to develop a fundamentally new multiscale framework for data-adaptive exploratory analysis of multivariate real-world processes. This will be achieved through a rigorous treatment of both within- and cross-channel intrinsic signal features, spanning time, space, frequency and entropy. Particular emphasis will be on approaches that are free of statistical assumptions and mathematical artefacts, and match the time-varying oscillatory modes inherent in multivariate data. This will help bypass the mathematical obstacles associated with currently used techniques (Fourier, wavelet), which rely on fixed basis functions and integral transforms, thus colouring the representation, limiting their accuracy, and restricting their applicability in problems involving real-world drifting and noisy information.

For multiscale data current statistical and information theoretic measures are inadequate, as they will indicate high correlation for two data channels that share common noises, but do not contain the same useful signal. The proposed data-adaptive analysis framework will resolve such issues, and will create natural "intrinsic" data association measures (intrinsic multi-correlation, intrinsic multi-information). While current univariate data-adaptive approaches have enormous potential, they are not suitable for direct application to multivariate or heterogeneous sources, as they are bound to create a different number of basis functions for every data channel.

Wearable systems, such as bodysensor networks, strive to find a balance between performance and user benefits (low cost, ease of use), and require next-generation signal processing tools to establish the extent to which they can produce valuable information. The thrust of this proposal is on developing rigorous, data-adaptive, compact, and physically meaningful signal processing solutions in order to provide an algorithmic support for progress in multi-sensor and wearable technologies. Our own initial multivariate data-adaptive solutions show great promise; they need to be further developed and comprehensively tested for data exhibiting rotation-dependent (noncircular) distributions, power imbalance, uncertainty, and noise. With the aid of nonlinear optimisation in the algorithmic design and insights from dynamical complexity science and multiresolution information theory, our approach promises a quantum step forward in multivariate data analysis, and a significant long-term impact.

The successful outcomes of this proposal will open radically new possibilities for advances in areas that depend on multi-sensor data, and a new front of research in applications dealing with uncertainty, noncircularity, complexity, and nonstationarity in multi-channel recordings. To maximise the short- to medium-term impact of this work and for cost effectiveness, we consider applications in emerging wearable technologies for brain monitoring, in collaboration with the Royal Brompton Sleep Clinic in London and Aarhus University in Denmark.

Planned Impact

Recorded data are the most important link that we have with the physical world, and the development of physically meaningful analysis methods is crucial to its understanding. Due to the intermittency of useful information, noise, and artefacts, recordings from real-world sources have an ever-changing frequency content, while nonlinear systems modulate the frequency not only among different oscillation periods, but also within one such period (intra-wave modulation). This causes temporal and spatial fluctuations in the intrinsic data scales and variations in within- and cross-channels couplings (synchrony, causality), hence calling for data-adaptive time-frequency and complexity analysis - a subject of this proposal.

This research proposes to introduce next generation solutions for multivariate exploratory data analysis. Due to its fundamental nature, immediate benefits will be to the academic research and education communities. In the longer term, this research is likely to offer quantum improvements in a number of emerging practical applications, particularly in providing a rigorous mathematical framework for establishing wearable (e.g. bodysensor) technologies in all aspects of life. Existing algorithms impose rigid assumptions and introduce mathematical artefacts for nonlinear and nonstationary signals, and are thus less than adequate for the analysis of intermittent and drifting multichannel information, common in a number of applications including wearable bodysensor technologies and robotics. Such areas are of strategic importance; they attract multi-billion pound investments, and have direct impact on quality of life.

We will also engage in impact and dissemination activities through, e.g. our entries to BCI competitions and exploratory data analysis contests. Such activities are already underway, and our Ear-EEG prototype has been nominated for the Annual BCI 2012 award. Impact in personalised healthcare is facilitated through our engagement with leading industries in this area (support letters from Widex and g.tec). Wearable technologies have enormous opportunities in user-friendly, self-administered, and 24/7 monitoring of the most vulnerable populations, such as the newborn and elderly. The possibility of monitoring patients in their natural environment promises huge economic savings (support letter from NHS). The microchips for data-adaptive time-frequency analysis are about to be launched, and will facilitate future commercial exploitation of the proposed research.

The PIs are on the boards of governing bodies in their respective communities, and are perfectly positioned to maximise the dissemination and impact of this research. By developing highly skilled researchers, through the work-plan of this project and related MEng, MSc, and group projects, we are likely to attract more interest and further research in this area, and to strengthen the position of the UK in statistical signal processing. These skills will be of considerable value to the industries working in this area; through our dissemination plan we have ensured that both the academic circles and relevant industries are aware of this work. We also have established close links with the Defence Sector (through the current University Defence Research Centre in Signal Processing and its legacy) whose strategic area is the networked battlefield - some key enabling technologies for accurate estimation in this context are a subject of this proposal.

Publications

10 25 50

publication icon
Goverdovsky V (2014) Multimodal physiological sensor for motion artefact rejection. in Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference

publication icon
Wang YT (2015) Developing an online steady-state visual evoked potential-based brain-computer interface system using EarEEG. in Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference

publication icon
Kappel SL (2014) A method for quantitative assessment of artifacts in EEG, and an empirical study of artifacts. in Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference

publication icon
Kidmose P (2013) Ear-EEG from generic earpieces: a feasibility study. in Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference

publication icon
Hansen ST (2019) Unmixing Oscillatory Brain Activity by EEG Source Localization and Empirical Mode Decomposition. in Computational intelligence and neuroscience

publication icon
Mikkelsen KB (2015) EEG Recorded from the Ear: Characterizing the Ear-EEG Method. in Frontiers in neuroscience

 
Description We have been able to identify sleep patterns from both standard Electroencephalogram (EEG) and our own in-ear EEG and to develop signal processing algorithms specially tailored for weak EEG signals in noise. This has been achieved in a data-driven manner and has lead to a number of new measures of signals association, such as intrinsic multivariate coherence, or intrinsic multivariate synchrony. This has also led to new "synchrosqueezing" transform for enhanced time-frequency analysis, and has enabled EEG estimation from the very convenient in-ear location. The findings have been supported by two patent applications.
Exploitation Route The findings may be commercialised, and we are talking with several big multinational companies in this direction (Bose, Huawei, Google, GSK). The theoretical findings are already picking up citations and will be helpful to any data analyst working on the detection and analysis of weak signals in noise.
Sectors Electronics,Healthcare

 
Description We have been able to attract attention of leading international companies, such as Bose, Huawei, Google, GSK, Sonion and they have all visited at least twice and have shown interest in commercial aspects of this technology.
First Year Of Impact 2016
Sector Healthcare
Impact Types Economic

 
Description Closed-loop multi-sensory Brain-Computer Interface for enhanced decision accuracy
Amount $6,000,000 (USD)
Funding ID EP/P008461/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 12/2016 
End 12/2019
 
Description Imperial College Dementia Research Centre
Amount £20,000,000 (GBP)
Funding ID Imperial College Dementia Research Centre 
Organisation Care UK 
Sector Private
Country United Kingdom
Start 10/2019 
End 09/2026
 
Description Low-cost high-tech concussion assessment and post-fall health monitoring, funded by The Racing Foundation
Amount £241,000 (GBP)
Organisation British Heart Foundation (BHF) 
Sector Charity/Non Profit
Country United Kingdom
Start 10/2018 
End 11/2020
 
Description Collaboration with Imperial Dementia Research Centre 
Organisation UK Dementia Research Institute
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution We are part of the new Imperial Dementia Research Centre, a 20 million GBP centre funded by DRI Care & Technology Programme, 2019-2015
Collaborator Contribution Our in-ear EEG system will be further developed and used to detect and predict the progress of dementia, in a 24/7 fashion
Impact No outputs yet, just started
Start Year 2019
 
Description Collaboration with Prof Mary Morrell, Royal Brompton Hospital on patient recruitement and data analysis 
Organisation National Heart and Lung Institute Foundation
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution We jointly collected data on sleep-related EEG
Collaborator Contribution Data analysis, protocol design, joint publications
Impact T. Nakamura, V. Goverdovsky, M. Morrell, and D. P. Mandic, ``Automatic sleep monitoring using ear-EEG'', IEEE Journal of Translational Engineering in Health and Medicine, minor revision, 2017. D. Looney, V. Goverdovsky, I. Rosenzweig, M. J. Morrell, and D. P. Mandic, ``A wearable in-ear encephalography sensor for monitoring sleep: Preliminary observations from nap studies'', Annals of the American Thoracic Society, vol. 13, no. 12, pp. 2229-2233, 2016. D. Looney, P. Kidmose, M. Morrell, and D. P. Mandic, ``Ear-EEG: Continuous Brain Monitoring'', in C. Guger et al. (editors), ``Brain-Computer Interface Research, pp. 63-71, Springer, 2014.
Start Year 2014
 
Description Collaboration with SONY company, Japan 
Organisation SONY
Country Japan 
Sector Private 
PI Contribution This is a funded work on the analysis of Ear-EEG data for entertainment systems
Collaborator Contribution SONY provided funding for research into artefact removal from ear-EEG
Impact N/A
Start Year 2020
 
Description Collaboration with Sonion, manufacturer of hearing aid components 
Organisation Sonion
Country Denmark 
Sector Private 
PI Contribution Sonion have provided us with their last generation MEMS microphones and in-ear receivers, which we will use in order to improve our current multimodal sensor
Collaborator Contribution Expert advice and parts, ca 600 GBP
Impact Improved multimodal sensor
Start Year 2017
 
Title BIOSENSING ELECTRODES 
Description A dual modality sensor comprises a tissue-contact electrode having a first surface configured for receiving an electrical signal from a user's tissue when attached thereto; and a mechanical sensor overlying the cutaneous electrode and configured to sense a mechanical displacement of the first surface through the electrode. The electrode and the mechanical sensor thereby provide electrical and mechanical signals which originate from precisely the same tissue location. 
IP Reference EP3094235 
Protection Patent granted
Year Protection Granted 2016
Licensed No
Impact A new collaboration with SONY corporation
 
Title BIOSENSING ELECTRODES 
Description A dual modality sensor comprises a tissue-contact electrode having a first surface configured for receiving an electrical signal from a user's tissue when attached thereto; and a mechanical sensor overlying the cutaneous electrode and configured to sense a mechanical displacement of the first surface through the electrode. The electrode and the mechanical sensor thereby provide electrical and mechanical signals which originate from precisely the same tissue location. 
IP Reference US2016331328 
Protection Patent granted
Year Protection Granted 2016
Licensed No
Impact We have established a new collaboration with Sony
 
Title ELECTROCARDIOGRAM APPARATUS AND METHOD 
Description The disclosure relates to a device and method of obtaining an electrocardiogram for a subject. The method comprises receiving electrical signals from at least two head- mounted sensors; and analysing said electrical signals to resolve shape and timing information for each of the P-, Q-, R-, S-, and T-waves available for the subject over a number of cardiac cycles, to derive a composite electrocardiogram, ECG, in which the composite electrocardiogram is derived using signals only from said head-mounted sensors. 
IP Reference WO2019077363 
Protection Patent application published
Year Protection Granted 2019
Licensed No
Impact n/a yet
 
Title FREQUENCY ESTIMATION 
Description A method and an apparatus for estimating a frequency are disclosed. In one method, the frequency of an electrical signal at a node in an electrical system having a plurality of nodes is estimated. The method comprises determining a current function indicative of the one or more properties of the electrical signal at the node at the current instant in time in accordance with at least one determined property of the electrical signal at the node at the current instant in time, a previous function indicative of the one or more properties of the electrical signal at the node at a previous instant in time, and a function of the error in the previous frequency estimation. A current function indicative of one or more properties of an electrical signal at the at least one other node at the current instant in time is received from at least one other node of the plurality of nodes. The current function indicative of one or more properties of the electrical signal at the node and the current function indicative of one or more properties of the electrical signal at the at least one other node are then combined to produce a current combined function indicative of one or more properties of the electrical signal at the node at the current instant in time. Finally, a current frequency of the electrical signal at the node is estimated in accordance with the current combined function. 
IP Reference WO2014053610 
Protection Patent application published
Year Protection Granted 2014
Licensed No
Impact n/a
 
Title This is an open source Python library for manipulation of big data 
Description HOTTBOX is a Python library for exploratory analysis and visualisation of multi-dimensional arrays of data, also known as tensors. It comprises methods ranging from standard multi-way operations through to multi-linear algebra based tensor decompositions and sophisticated algorithms for generalised multi-linear classification and data fusion such as Support Tensor Machine (STM) and Tensor Ensemble Learning (TEL). For user convenience, HOTTBOX offers a unifying API which establishes a self-sufficient ecosystem for various forms of efficient representation of multi-way data and corresponding decomposition and association algorithms. Particular emphasis is placed on scalability and interactive visualisation, to support multidisciplinary data analysis communities working on big data and tensors. HOTTBOX also provides means for integration with other popular data science libraries for visualisation and data manipulation. The source code, examples and documentation ca be found at https://github.com/hottbox/hottbox. 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact It has been downloaded and used more than 1000 times 
URL https://github.com/hottbox/hottbox
 
Description Tutorial at a foremost international conference : Multiscale Signal Processing for Wearable Health: Sleep, Stress, and Fatigue Applications 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Tutorial at ICASSP 2016 conference (2000 + attendenees in the conference)
Year(s) Of Engagement Activity 2016
URL http://www.icassp2016.org/Tutorials.asp#T15
 
Description Tutorial at an international conference 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This was a tutorial at the World COngress on Computational Intelligence, entitled "Computational intelligence for wearable physiological sensing"
Year(s) Of Engagement Activity 2015
URL http://www.ijcnn.org/assets/docs/ijcnn2015-program-v10.pdf