Multiscale Signal Processing for Next Generation Electroencephalography
Lead Research Organisation:
Imperial College London
Department Name: Electrical and Electronic Engineering
Abstract
This proposal seeks to develop a fundamentally new multiscale framework for data-adaptive exploratory analysis of multivariate real-world processes. This will be achieved through a rigorous treatment of both within- and cross-channel intrinsic signal features, spanning time, space, frequency and entropy. Particular emphasis will be on approaches that are free of statistical assumptions and mathematical artefacts, and match the time-varying oscillatory modes inherent in multivariate data. This will help bypass the mathematical obstacles associated with currently used techniques (Fourier, wavelet), which rely on fixed basis functions and integral transforms, thus colouring the representation, limiting their accuracy, and restricting their applicability in problems involving real-world drifting and noisy information.
For multiscale data current statistical and information theoretic measures are inadequate, as they will indicate high correlation for two data channels that share common noises, but do not contain the same useful signal. The proposed data-adaptive analysis framework will resolve such issues, and will create natural "intrinsic" data association measures (intrinsic multi-correlation, intrinsic multi-information). While current univariate data-adaptive approaches have enormous potential, they are not suitable for direct application to multivariate or heterogeneous sources, as they are bound to create a different number of basis functions for every data channel.
Wearable systems, such as bodysensor networks, strive to find a balance between performance and user benefits (low cost, ease of use), and require next-generation signal processing tools to establish the extent to which they can produce valuable information. The thrust of this proposal is on developing rigorous, data-adaptive, compact, and physically meaningful signal processing solutions in order to provide an algorithmic support for progress in multi-sensor and wearable technologies. Our own initial multivariate data-adaptive solutions show great promise; they need to be further developed and comprehensively tested for data exhibiting rotation-dependent (noncircular) distributions, power imbalance, uncertainty, and noise. With the aid of nonlinear optimisation in the algorithmic design and insights from dynamical complexity science and multiresolution information theory, our approach promises a quantum step forward in multivariate data analysis, and a significant long-term impact.
The successful outcomes of this proposal will open radically new possibilities for advances in areas that depend on multi-sensor data, and a new front of research in applications dealing with uncertainty, noncircularity, complexity, and nonstationarity in multi-channel recordings. To maximise the short- to medium-term impact of this work and for cost effectiveness, we consider applications in emerging wearable technologies for brain monitoring, in collaboration with the Royal Brompton Sleep Clinic in London and Aarhus University in Denmark.
For multiscale data current statistical and information theoretic measures are inadequate, as they will indicate high correlation for two data channels that share common noises, but do not contain the same useful signal. The proposed data-adaptive analysis framework will resolve such issues, and will create natural "intrinsic" data association measures (intrinsic multi-correlation, intrinsic multi-information). While current univariate data-adaptive approaches have enormous potential, they are not suitable for direct application to multivariate or heterogeneous sources, as they are bound to create a different number of basis functions for every data channel.
Wearable systems, such as bodysensor networks, strive to find a balance between performance and user benefits (low cost, ease of use), and require next-generation signal processing tools to establish the extent to which they can produce valuable information. The thrust of this proposal is on developing rigorous, data-adaptive, compact, and physically meaningful signal processing solutions in order to provide an algorithmic support for progress in multi-sensor and wearable technologies. Our own initial multivariate data-adaptive solutions show great promise; they need to be further developed and comprehensively tested for data exhibiting rotation-dependent (noncircular) distributions, power imbalance, uncertainty, and noise. With the aid of nonlinear optimisation in the algorithmic design and insights from dynamical complexity science and multiresolution information theory, our approach promises a quantum step forward in multivariate data analysis, and a significant long-term impact.
The successful outcomes of this proposal will open radically new possibilities for advances in areas that depend on multi-sensor data, and a new front of research in applications dealing with uncertainty, noncircularity, complexity, and nonstationarity in multi-channel recordings. To maximise the short- to medium-term impact of this work and for cost effectiveness, we consider applications in emerging wearable technologies for brain monitoring, in collaboration with the Royal Brompton Sleep Clinic in London and Aarhus University in Denmark.
Planned Impact
Recorded data are the most important link that we have with the physical world, and the development of physically meaningful analysis methods is crucial to its understanding. Due to the intermittency of useful information, noise, and artefacts, recordings from real-world sources have an ever-changing frequency content, while nonlinear systems modulate the frequency not only among different oscillation periods, but also within one such period (intra-wave modulation). This causes temporal and spatial fluctuations in the intrinsic data scales and variations in within- and cross-channels couplings (synchrony, causality), hence calling for data-adaptive time-frequency and complexity analysis - a subject of this proposal.
This research proposes to introduce next generation solutions for multivariate exploratory data analysis. Due to its fundamental nature, immediate benefits will be to the academic research and education communities. In the longer term, this research is likely to offer quantum improvements in a number of emerging practical applications, particularly in providing a rigorous mathematical framework for establishing wearable (e.g. bodysensor) technologies in all aspects of life. Existing algorithms impose rigid assumptions and introduce mathematical artefacts for nonlinear and nonstationary signals, and are thus less than adequate for the analysis of intermittent and drifting multichannel information, common in a number of applications including wearable bodysensor technologies and robotics. Such areas are of strategic importance; they attract multi-billion pound investments, and have direct impact on quality of life.
We will also engage in impact and dissemination activities through, e.g. our entries to BCI competitions and exploratory data analysis contests. Such activities are already underway, and our Ear-EEG prototype has been nominated for the Annual BCI 2012 award. Impact in personalised healthcare is facilitated through our engagement with leading industries in this area (support letters from Widex and g.tec). Wearable technologies have enormous opportunities in user-friendly, self-administered, and 24/7 monitoring of the most vulnerable populations, such as the newborn and elderly. The possibility of monitoring patients in their natural environment promises huge economic savings (support letter from NHS). The microchips for data-adaptive time-frequency analysis are about to be launched, and will facilitate future commercial exploitation of the proposed research.
The PIs are on the boards of governing bodies in their respective communities, and are perfectly positioned to maximise the dissemination and impact of this research. By developing highly skilled researchers, through the work-plan of this project and related MEng, MSc, and group projects, we are likely to attract more interest and further research in this area, and to strengthen the position of the UK in statistical signal processing. These skills will be of considerable value to the industries working in this area; through our dissemination plan we have ensured that both the academic circles and relevant industries are aware of this work. We also have established close links with the Defence Sector (through the current University Defence Research Centre in Signal Processing and its legacy) whose strategic area is the networked battlefield - some key enabling technologies for accurate estimation in this context are a subject of this proposal.
This research proposes to introduce next generation solutions for multivariate exploratory data analysis. Due to its fundamental nature, immediate benefits will be to the academic research and education communities. In the longer term, this research is likely to offer quantum improvements in a number of emerging practical applications, particularly in providing a rigorous mathematical framework for establishing wearable (e.g. bodysensor) technologies in all aspects of life. Existing algorithms impose rigid assumptions and introduce mathematical artefacts for nonlinear and nonstationary signals, and are thus less than adequate for the analysis of intermittent and drifting multichannel information, common in a number of applications including wearable bodysensor technologies and robotics. Such areas are of strategic importance; they attract multi-billion pound investments, and have direct impact on quality of life.
We will also engage in impact and dissemination activities through, e.g. our entries to BCI competitions and exploratory data analysis contests. Such activities are already underway, and our Ear-EEG prototype has been nominated for the Annual BCI 2012 award. Impact in personalised healthcare is facilitated through our engagement with leading industries in this area (support letters from Widex and g.tec). Wearable technologies have enormous opportunities in user-friendly, self-administered, and 24/7 monitoring of the most vulnerable populations, such as the newborn and elderly. The possibility of monitoring patients in their natural environment promises huge economic savings (support letter from NHS). The microchips for data-adaptive time-frequency analysis are about to be launched, and will facilitate future commercial exploitation of the proposed research.
The PIs are on the boards of governing bodies in their respective communities, and are perfectly positioned to maximise the dissemination and impact of this research. By developing highly skilled researchers, through the work-plan of this project and related MEng, MSc, and group projects, we are likely to attract more interest and further research in this area, and to strengthen the position of the UK in statistical signal processing. These skills will be of considerable value to the industries working in this area; through our dissemination plan we have ensured that both the academic circles and relevant industries are aware of this work. We also have established close links with the Defence Sector (through the current University Defence Research Centre in Signal Processing and its legacy) whose strategic area is the networked battlefield - some key enabling technologies for accurate estimation in this context are a subject of this proposal.
People |
ORCID iD |
Danilo Mandic (Principal Investigator) | |
Mary Morrell (Co-Investigator) |
Publications
Abdullah S
(2015)
A Multivariate Empirical Mode DecompositionBased Approach to Pansharpening
in IEEE Transactions on Geoscience and Remote Sensing
Adjei T
(2018)
The Female Heart: Sex Differences in the Dynamics of ECG in Response to Stress.
in Frontiers in physiology
Ahmed M
(2016)
A Multivariate Multiscale Fuzzy Entropy Algorithm with Application to Uterine EMG Complexity Analysis
in Entropy
Ahrabian A
(2015)
Synchrosqueezing-based time-frequency analysis of multivariate data
in Signal Processing
Ahrabian A
(2014)
Estimation of phase synchrony using the synchrosqueezing transform
Ahrabian A
(2013)
Bivariate Empirical Mode Decomposition for Unbalanced Real-World Signals
in IEEE Signal Processing Letters
Ahrabian A
(2015)
Selective Time-Frequency Reassignment Based on Synchrosqueezing
in IEEE Signal Processing Letters
Ahrabian A
(2015)
A Class of Multivariate Denoising Algorithms Based on Synchrosqueezing
in IEEE Transactions on Signal Processing
Alqurashi YD
(2018)
A novel in-ear sensor to determine sleep latency during the Multiple Sleep Latency Test in healthy adults with and without sleep restriction.
in Nature and science of sleep
Chanwimalueang T
(2017)
Stage call: Cardiovascular reactivity to audition stress in musicians.
in PloS one
Description | We have been able to identify sleep patterns from both standard Electroencephalogram (EEG) and our own in-ear EEG and to develop signal processing algorithms specially tailored for weak EEG signals in noise. This has been achieved in a data-driven manner and has lead to a number of new measures of signals association, such as intrinsic multivariate coherence, or intrinsic multivariate synchrony. This has also led to new "synchrosqueezing" transform for enhanced time-frequency analysis, and has enabled EEG estimation from the very convenient in-ear location. The findings have been supported by two patent applications. |
Exploitation Route | The findings may be commercialised, and we are talking with several big multinational companies in this direction (Bose, Huawei, Google, GSK). The theoretical findings are already picking up citations and will be helpful to any data analyst working on the detection and analysis of weak signals in noise. |
Sectors | Electronics Healthcare |
Description | We have been able to attract attention of leading international companies, such as Bose, Huawei, Google, GSK, Sonion and they have all visited at least twice and have shown interest in commercial aspects of this technology. |
First Year Of Impact | 2016 |
Sector | Healthcare |
Impact Types | Economic |
Description | Closed-loop multi-sensory Brain-Computer Interface for enhanced decision accuracy |
Amount | $6,000,000 (USD) |
Funding ID | EP/P008461/1 |
Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
Sector | Public |
Country | United Kingdom |
Start | 12/2016 |
End | 12/2019 |
Description | Imperial College Dementia Research Centre |
Amount | £20,000,000 (GBP) |
Funding ID | Imperial College Dementia Research Centre |
Organisation | Care UK |
Sector | Private |
Country | United Kingdom |
Start | 09/2019 |
End | 09/2026 |
Description | Low-cost high-tech concussion assessment and post-fall health monitoring, funded by The Racing Foundation |
Amount | £241,000 (GBP) |
Organisation | British Heart Foundation (BHF) |
Sector | Charity/Non Profit |
Country | United Kingdom |
Start | 09/2018 |
End | 11/2020 |
Description | Collaboration with Imperial Dementia Research Centre |
Organisation | UK Dementia Research Institute |
Country | United Kingdom |
Sector | Charity/Non Profit |
PI Contribution | We are part of the new Imperial Dementia Research Centre, a 20 million GBP centre funded by DRI Care & Technology Programme, 2019-2015 |
Collaborator Contribution | Our in-ear EEG system will be further developed and used to detect and predict the progress of dementia, in a 24/7 fashion |
Impact | No outputs yet, just started |
Start Year | 2019 |
Description | Collaboration with Prof Mary Morrell, Royal Brompton Hospital on patient recruitement and data analysis |
Organisation | National Heart and Lung Institute Foundation |
Country | United Kingdom |
Sector | Charity/Non Profit |
PI Contribution | We jointly collected data on sleep-related EEG |
Collaborator Contribution | Data analysis, protocol design, joint publications |
Impact | T. Nakamura, V. Goverdovsky, M. Morrell, and D. P. Mandic, ``Automatic sleep monitoring using ear-EEG'', IEEE Journal of Translational Engineering in Health and Medicine, minor revision, 2017. D. Looney, V. Goverdovsky, I. Rosenzweig, M. J. Morrell, and D. P. Mandic, ``A wearable in-ear encephalography sensor for monitoring sleep: Preliminary observations from nap studies'', Annals of the American Thoracic Society, vol. 13, no. 12, pp. 2229-2233, 2016. D. Looney, P. Kidmose, M. Morrell, and D. P. Mandic, ``Ear-EEG: Continuous Brain Monitoring'', in C. Guger et al. (editors), ``Brain-Computer Interface Research, pp. 63-71, Springer, 2014. |
Start Year | 2014 |
Description | Collaboration with SONY company, Japan |
Organisation | SONY |
Country | Japan |
Sector | Private |
PI Contribution | This is a funded work on the analysis of Ear-EEG data for entertainment systems |
Collaborator Contribution | SONY provided funding for research into artefact removal from ear-EEG |
Impact | N/A |
Start Year | 2020 |
Description | Collaboration with Sonion, manufacturer of hearing aid components |
Organisation | Sonion |
Country | Denmark |
Sector | Private |
PI Contribution | Sonion have provided us with their last generation MEMS microphones and in-ear receivers, which we will use in order to improve our current multimodal sensor |
Collaborator Contribution | Expert advice and parts, ca 600 GBP |
Impact | Improved multimodal sensor |
Start Year | 2017 |
Title | BIOSENSING ELECTRODES |
Description | A dual modality sensor comprises a tissue-contact electrode having a first surface configured for receiving an electrical signal from a user's tissue when attached thereto; and a mechanical sensor overlying the cutaneous electrode and configured to sense a mechanical displacement of the first surface through the electrode. The electrode and the mechanical sensor thereby provide electrical and mechanical signals which originate from precisely the same tissue location. |
IP Reference | EP3094235 |
Protection | Patent granted |
Year Protection Granted | 2016 |
Licensed | No |
Impact | A new collaboration with SONY corporation |
Title | BIOSENSING ELECTRODES |
Description | A dual modality sensor comprises a tissue-contact electrode having a first surface configured for receiving an electrical signal from a user's tissue when attached thereto; and a mechanical sensor overlying the cutaneous electrode and configured to sense a mechanical displacement of the first surface through the electrode. The electrode and the mechanical sensor thereby provide electrical and mechanical signals which originate from precisely the same tissue location. |
IP Reference | US2016331328 |
Protection | Patent granted |
Year Protection Granted | 2016 |
Licensed | No |
Impact | We have established a new collaboration with Sony |
Title | ELECTROCARDIOGRAM APPARATUS AND METHOD |
Description | The disclosure relates to a device and method of obtaining an electrocardiogram for a subject. The method comprises receiving electrical signals from at least two head- mounted sensors; and analysing said electrical signals to resolve shape and timing information for each of the P-, Q-, R-, S-, and T-waves available for the subject over a number of cardiac cycles, to derive a composite electrocardiogram, ECG, in which the composite electrocardiogram is derived using signals only from said head-mounted sensors. |
IP Reference | WO2019077363 |
Protection | Patent application published |
Year Protection Granted | 2019 |
Licensed | No |
Impact | n/a yet |
Title | FREQUENCY ESTIMATION |
Description | A method and an apparatus for estimating a frequency are disclosed. In one method, the frequency of an electrical signal at a node in an electrical system having a plurality of nodes is estimated. The method comprises determining a current function indicative of the one or more properties of the electrical signal at the node at the current instant in time in accordance with at least one determined property of the electrical signal at the node at the current instant in time, a previous function indicative of the one or more properties of the electrical signal at the node at a previous instant in time, and a function of the error in the previous frequency estimation. A current function indicative of one or more properties of an electrical signal at the at least one other node at the current instant in time is received from at least one other node of the plurality of nodes. The current function indicative of one or more properties of the electrical signal at the node and the current function indicative of one or more properties of the electrical signal at the at least one other node are then combined to produce a current combined function indicative of one or more properties of the electrical signal at the node at the current instant in time. Finally, a current frequency of the electrical signal at the node is estimated in accordance with the current combined function. |
IP Reference | WO2014053610 |
Protection | Patent application published |
Year Protection Granted | 2014 |
Licensed | No |
Impact | n/a |
Title | This is an open source Python library for manipulation of big data |
Description | HOTTBOX is a Python library for exploratory analysis and visualisation of multi-dimensional arrays of data, also known as tensors. It comprises methods ranging from standard multi-way operations through to multi-linear algebra based tensor decompositions and sophisticated algorithms for generalised multi-linear classification and data fusion such as Support Tensor Machine (STM) and Tensor Ensemble Learning (TEL). For user convenience, HOTTBOX offers a unifying API which establishes a self-sufficient ecosystem for various forms of efficient representation of multi-way data and corresponding decomposition and association algorithms. Particular emphasis is placed on scalability and interactive visualisation, to support multidisciplinary data analysis communities working on big data and tensors. HOTTBOX also provides means for integration with other popular data science libraries for visualisation and data manipulation. The source code, examples and documentation ca be found at https://github.com/hottbox/hottbox. |
Type Of Technology | Software |
Year Produced | 2021 |
Open Source License? | Yes |
Impact | It has been downloaded and used more than 1000 times |
URL | https://github.com/hottbox/hottbox |
Description | Tutorial at a foremost international conference : Multiscale Signal Processing for Wearable Health: Sleep, Stress, and Fatigue Applications |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Tutorial at ICASSP 2016 conference (2000 + attendenees in the conference) |
Year(s) Of Engagement Activity | 2016 |
URL | http://www.icassp2016.org/Tutorials.asp#T15 |
Description | Tutorial at an international conference |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | This was a tutorial at the World COngress on Computational Intelligence, entitled "Computational intelligence for wearable physiological sensing" |
Year(s) Of Engagement Activity | 2015 |
URL | http://www.ijcnn.org/assets/docs/ijcnn2015-program-v10.pdf |