Multidimensional endoscopy for early cancer detection
Lead Research Organisation:
University of Cambridge
Department Name: Physics
Abstract
Each year in the UK, over 55,000 patients will be diagnosed with oesophageal or lung cancer; a staggering 44,000 will die from their disease. The majority of patients are currently diagnosed when their disease is at a late stage (with regional or distant spread), yielding less than 10% survival over 5 years. Therefore, a key clinical unmet need remains for early disease diagnosis.
To detect these cancers early, patients that are at a high risk of disease might undergo endoscopy. An endoscope is a long, thin, flexible tube with a light source and video camera at one end to relay images from inside the body. They replicate what our eye could see if we were able to look inside the body. Endoscopy is vital for surveillance and diagnosis of cancer, however, many cancers of the oesophagus (food pipe) and lung start out as small abnormalities that are hard to spot using conventional endoscopes.
The focus of this research is to augment the vision of the endoscopist by designing a specialist video camera that detects the properties of light that our eyes are blind to. We will also use this new camera in a benchtop system that can analyse tissue that has been taken out of the body (biopsied) very quickly, before it is sent to a pathologist to check for cancer. Our new cameras are designed to be cost effective and provide extra information to help doctors with diagnosing cancer in the oesophagus and lung, and could ultimately replace the current standard-of-care.
To detect these cancers early, patients that are at a high risk of disease might undergo endoscopy. An endoscope is a long, thin, flexible tube with a light source and video camera at one end to relay images from inside the body. They replicate what our eye could see if we were able to look inside the body. Endoscopy is vital for surveillance and diagnosis of cancer, however, many cancers of the oesophagus (food pipe) and lung start out as small abnormalities that are hard to spot using conventional endoscopes.
The focus of this research is to augment the vision of the endoscopist by designing a specialist video camera that detects the properties of light that our eyes are blind to. We will also use this new camera in a benchtop system that can analyse tissue that has been taken out of the body (biopsied) very quickly, before it is sent to a pathologist to check for cancer. Our new cameras are designed to be cost effective and provide extra information to help doctors with diagnosing cancer in the oesophagus and lung, and could ultimately replace the current standard-of-care.
Planned Impact
Stakeholders that will benefit from this research include:
- Patients, through: more accessible testing (lower cost, less specialist equipment); reduced anxiety (better informed at point of test); improved outcomes (reduced mortality); and improved quality of life (fewer invasive interventions).
- Endoscopists and bronchoscopists, through: better imaging contrast that improves interpretation; reduced procedure time; and improvements in patient management.
- Endoscopy manufacturers, such as Olympus, Fuji and Pentax, and start-ups, such as Endochoice, who may benefit from our IP generation by increasing the capability of endoscopes to incorporate new techniques, involving additional research in image processing, visualisation, augmented reality as well as hardware, creating new jobs not currently present in the endoscopy field.
- Pathologists and surgeons, through: application of the MITA system, which could be deployed cost effectively, to help guide pathological sectioning and surgical margin delineation.
- Public service providers, such as the NHS, through: cost reductions in terms of procedure time as well as capital equipment. Our "5 in 1" endoscopy enables short procedures and cost effective deployment. Our MITA system has been designed through user engagement with endoscopists so is small, portable, and can fit into current facilities. Nanoplasmonic metapixel arrays should be highly robust so could also reduce service costs.
- General public, through: research engagement activites to demonstrate the power of multidisciplinary research, increasing awareness of the role of physical sciences in healthcare.
- Patients, through: more accessible testing (lower cost, less specialist equipment); reduced anxiety (better informed at point of test); improved outcomes (reduced mortality); and improved quality of life (fewer invasive interventions).
- Endoscopists and bronchoscopists, through: better imaging contrast that improves interpretation; reduced procedure time; and improvements in patient management.
- Endoscopy manufacturers, such as Olympus, Fuji and Pentax, and start-ups, such as Endochoice, who may benefit from our IP generation by increasing the capability of endoscopes to incorporate new techniques, involving additional research in image processing, visualisation, augmented reality as well as hardware, creating new jobs not currently present in the endoscopy field.
- Pathologists and surgeons, through: application of the MITA system, which could be deployed cost effectively, to help guide pathological sectioning and surgical margin delineation.
- Public service providers, such as the NHS, through: cost reductions in terms of procedure time as well as capital equipment. Our "5 in 1" endoscopy enables short procedures and cost effective deployment. Our MITA system has been designed through user engagement with endoscopists so is small, portable, and can fit into current facilities. Nanoplasmonic metapixel arrays should be highly robust so could also reduce service costs.
- General public, through: research engagement activites to demonstrate the power of multidisciplinary research, increasing awareness of the role of physical sciences in healthcare.
Publications
Bohndiek S
(2019)
Addressing photoacoustics standards
in Nature Photonics
Bohndiek Sarah
(2020)
A PHYSICIST TALKS TO A BIOLOGIST
in NATURE
Bombara D
(2021)
Automated real-time spectral characterization of phase-change tunable filters using a linear variable filter and IR camera for wide-field MWIR imaging
in Optical Engineering
Brown E
(2019)
Photoacoustic imaging as a tool to probe the tumour microenvironment.
in Disease models & mechanisms
Gataric M
(2019)
Reconstruction of Optical Vector-Fields With Applications in Endoscopic Imaging.
in IEEE transactions on medical imaging
Gordon G
(2019)
Characterizing Optical Fiber Transmission Matrices Using Metasurface Reflector Stacks for Lensless Imaging without Distal Access
in Physical Review X
Gordon G
(2019)
Coherent Imaging Through Multicore Fibres With Applications in Endoscopy
in Journal of Lightwave Technology
Gordon GSD
(2019)
Full-field quantitative phase and polarisation-resolved imaging through an optical fibre bundle.
in Optics express
Gordon GSD
(2019)
Quantitative phase and polarization imaging through an optical fiber applied to detection of early esophageal tumorigenesis.
in Journal of biomedical optics
Grigoroiu A
(2020)
Deep learning applied to hyperspectral endoscopy for online spectral classification.
in Scientific reports
Description | EU Biomed Alliance Task Force Member |
Geographic Reach | Europe |
Policy Influence Type | Membership of a guideline committee |
Impact | The Biomedical Alliance in Europe is the result of a unique initiative of leading European medical societies that together include more than 400,000 researchers and health professionals. The BioMed Alliance was created in 2010 to gather strength across different disciplines and areas in biomedical and health research at European level. |
URL | https://www.biomedeurope.org/ |
Description | Alliance for Cancer Early Detection Cambridge Member Centre |
Amount | £3,300,000 (GBP) |
Organisation | Cancer Research UK |
Sector | Charity/Non Profit |
Country | United Kingdom |
Start | 04/2019 |
End | 03/2024 |
Description | CRUK Spark Award |
Amount | £20,000 (GBP) |
Funding ID | C47594/A26851 |
Organisation | Cancer Research UK |
Sector | Charity/Non Profit |
Country | United Kingdom |
Start | 01/2019 |
End | 12/2019 |
Description | Cambridge-Tsinghua Joint Research Initiative |
Amount | £11,250 (GBP) |
Organisation | University of Cambridge |
Sector | Academic/University |
Country | United Kingdom |
Start | 01/2020 |
End | 12/2020 |
Description | EPSRC - Impact Acceleration Account Follow on Fund |
Amount | £60,000 (GBP) |
Organisation | ESRC Impact Acceleration Account Cambridge |
Sector | Academic/University |
Country | United Kingdom |
Start | 02/2021 |
End | 03/2022 |
Title | HySE - hyperspectral endoscopy |
Description | This method allows for the collection of hyperspectral imaging data from endoscopy, which enables the resolution of different lesion types in the gastrointestinal tract. |
Type Of Material | Technology assay or reagent |
Year Produced | 2019 |
Provided To Others? | Yes |
Impact | Methodology has been tested in 2 clinical studies resulting in 3 manuscripts Methodology has been patented and licenced to a spin out company supported by NLC |
Title | A background correction method to compensate illumination variation in hyperspectral imaging |
Description | Funder: FP7 People: Marie-Curie Actions; funder-id: http://dx.doi.org/10.13039/100011264; Grant(s): FP7-PEOPLE-2013-CIG-630729 |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | Not known |
URL | https://www.repository.cam.ac.uk/handle/1810/303457 |
Title | Dataset for: Characterizing optical fiber transmission matrices using metasurface reflector stacks for lensless imaging without distal access |
Description | |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | Yes |
URL | https://www.repository.cam.ac.uk/handle/1810/305512 |
Title | Dataset for: First experience in clinical application of hyperspectral endoscopy for evaluation of colonic polyps |
Description | All code and processed images used to prepare the figures associated with this manuscript. Raw endoscopy videos can be obtained on request from the corresponding author. |
Type Of Material | Database/Collection of data |
Year Produced | 2021 |
Provided To Others? | Yes |
Impact | Dataset was used for subsequent analysis that led to further publishable outputs and a patent filing. |
URL | https://www.repository.cam.ac.uk/handle/1810/323059 |
Title | Dataset for: First-in-human pilot study of snapshot multispectral endoscopy for early detection of Barrett's-related neoplasia. |
Description | Research data and code arising from a first-in-human endoscopy clinical trial (NCT03388047). Each of the data tables contains 9-point multispectral image spectra created from multispectral images captured using a snapshot multispectral endoscope. The raw images were processed into these image spectra using the code in 'PROCESSING CODE'. Raw image data can be provided upon request. The tissue spectra were captured in a first-in-human pilot study in the oesophagus. The colour spectra were captured from an X-rite colour target. Each row of the arrays contains the information about the spectra, most importantly, the labels from pathology. The different arrays contain spectra from averages over each multispectral image (processed_tissue_spectra.mat), each patient (processed_tissue_spectra_avg_per_trial.mat and processed_tissue_spectra_avg_per_trial_per_region_then_pooled.mat), and over all patients (processed_tissue_spectra_avg_overall.mat). Please see included readme files for further details. |
Type Of Material | Database/Collection of data |
Year Produced | 2021 |
Provided To Others? | Yes |
Impact | None yet. |
URL | https://www.repository.cam.ac.uk/handle/1810/329675 |
Title | Dataset for: Quantitative phase and polarization imaging through an optical fiber applied to detection of early esophageal tumorigenesis |
Description | |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | Yes |
URL | https://www.repository.cam.ac.uk/handle/1810/304918 |
Title | Dataset for: Spectral endoscopy combined with computational modelling and deep-learning yield high-contrast for oesophageal neoplasia |
Description | Data upload contains data tables of attenuation and reflection spectra. Also contains the code used to generate the manuscript Figures and Tables from the data. Raw endoscopy videos can be obtained upon request from the corresponding author of the manuscript. |
Type Of Material | Database/Collection of data |
Year Produced | 2021 |
Provided To Others? | Yes |
Impact | Supported investigations that subsequently led to further publications and a patent filing |
URL | https://www.repository.cam.ac.uk/handle/1810/323051 |
Title | Deep learning applied to hyperspectral endoscopy for online spectral classification. |
Description | Hyperspectral imaging (HSI) is being explored in endoscopy as a tool to extract biochemical information from tissue optical properties that may improve contrast for early cancer detection in the gastrointestinal tract. Motion artefacts during medical endoscopy have traditionally limited HSI application, however, recent developments in the field have led to real-time HSI deployments. Unfortunately, traditional HSI analysis methods remain unable to handle the volume of hyperspectral data in order to provide real-time feedback to the operator. Here, a convolutional neural network (CNN) is proposed to enable online classification of data obtained during HSI endoscopy. A five-layered CNN was trained and fine-tuned on a dataset of 300 hyperspectral endoscopy images acquired from a planar Macbeth ColorChecker chart and was able to distinguish between its 18 constituent colors with an average accuracy of 94.3% achieved at 8.8 fps. Performance was then tested on a set of images simulating an endoscopy environment, consisting of color charts warped inside a rigid tube mimicking a lumen. The algorithm improved robust to such variations, with classification accuracies over 90% being obtained despite the variations, with an average drop in accuracy of 2.4% being registered at the points of longest working distance and most inclination. For further validation of the color-based classification system, ex vivo videos of a methylene blue dyed pig esophagus and images of different cancer stages in the human esophagus were analyzed, showing spatially distinct color classifications. These results suggest that the CNN has potential to provide color-based classification during real-time HSI in endoscopy. |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | Enquiries regarding use of the dataset |
URL | https://www.repository.cam.ac.uk/handle/1810/302170 |
Title | Hyperspectral Colour Classification (HCC) |
Description | Dataset utilised for the training and testing of the spectral classification system for a line-scanning hyperspectral endoscope and the associated core algorithms. See the Readme file for a detailed description. |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | None known |
URL | https://www.repository.cam.ac.uk/handle/1810/303062 |
Title | Research data supporting "A background correction method to compensate illumination variation in hyperspectral imaging" |
Description | The dataset includes experimental and simulation data and the codes used in the paper "A background correction method to compensate illumination variation in hyperspectral imaging". All experimental data is captured using hyperspectral endoscopy (https://www.nature.com/articles/s41467-019-09484-4). See the Readme file for detailed information. |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | Not known |
URL | https://www.repository.cam.ac.uk/handle/1810/302032 |
Title | Robustness to misalignment of low-cost, compact quantitative phase imaging architectures |
Description | Non-interferometric approaches to quantitative phase imaging could enable its application in low-cost, miniaturised settings such as capsule endoscopy. We present two possible architectures and both analyse and mitigate the effect of sensor misalignment on phase imaging performance. This is a crucial step towards determining the feasibility of implementing phase imaging in a capsule device. First, we investigate a design based on a folded 4f correlator, both in simulation and experimentally. We demonstrate a novel technique for identifying and compensating for axial misalignment and explore the limits of the approach. Next, we explore the implications of axial and transverse misalignment, and of manufacturing variations on the performance of a phase plate-based architecture, identifying a clear trade-off between phase plate resolution and algorithm convergence time. We conclude that while the phase plate architecture is more robust to misalignment, both architectures merit further development with the goal of realising a low-cost, compact system for applying phase imaging in capsule endoscopy. |
Type Of Material | Database/Collection of data |
Year Produced | 2020 |
Provided To Others? | Yes |
Impact | None known |
URL | https://www.repository.cam.ac.uk/handle/1810/309619 |
Description | Johnson & Johnson |
Organisation | Johnson & Johnson |
Country | United States |
Sector | Private |
PI Contribution | We are bringing IP in optical filter fabrication to enable multispectral imaging using OVT sensors by direct deposition of filters using electron beam lithography and photolithography |
Collaborator Contribution | J&J (Auris) are providing bare silicon wafers for us to test our filter technologies for direct deposition onto CMOS image sensors |
Impact | Multidisciplinary, engineering, physics, medical. No outputs as yet |
Start Year | 2019 |
Description | Omnivision Technologies |
Organisation | OmniVision Technologies Inc |
Country | United States |
Sector | Private |
PI Contribution | We are bringing IP in optical filter fabrication to enable multispectral imaging using OVT sensors by direct deposition of filters using electron beam lithography and photolithography |
Collaborator Contribution | OVT are providing bare silicon wafers for us to test our filter technologies for direct deposition onto CMOS image sensors |
Impact | Multidisciplinary, engineering, physics, medical No outputs as yet |
Start Year | 2018 |
Title | A SINGLE STEP LITHOGRAPHY COLOUR FILTER |
Description | We disclose herein a method of producing an optical filter. The method comprising depositing a first mirror layer onto a substrate; depositing an insulating layeron the first mirror; exposing at least some of a plurality of portions of a surface of the insulating layer to a dose of energy; developing the insulating layer in order to remove a volume from the at least some of the plurality of portions of the insulating layer, wherein the volume of the insulating layer removed from each portion is related to the dose of energy exposed to each portion, and wherein a remaining thickness after the removal of the volume from each portion of the insulating layer is relatedto the dose of energy exposed to each portion. The method further comprising depositing a second mirror layer on the remaining thickness of each of the plurality of portions of the insulating layer such that the remaining thickness of each of the plurality of portions of the insulating layer define a profile of the optical filter. |
IP Reference | WO2019239139 |
Protection | Patent application published |
Year Protection Granted | 2019 |
Licensed | Yes |
Impact | none yet |
Title | APPARATUS AND METHOD FOR WIDE-FIELD HYPERSPECTRAL IMAGING |
Description | Embodiments of the present invention provide a hyperspectral endoscope system, comprising a memory for storing data therein, an endoscope arranged to, in use, receive radiation reflected from a sample and to output wide-field image data and line-scan hyperspectral data corresponding to the sample, a processor coupled to the memory, wherein the processor is arranged, in use, to determine registration information between portions of the wide-field image data, and determine wide-area hyperspectral image data in dependence on the registration information and the line-scan hyperspectral data. |
IP Reference | US2021374981 |
Protection | Patent application published |
Year Protection Granted | 2021 |
Licensed | Yes |
Impact | None yet |
Title | METHODS OF CHARACTERISING AND IMAGING WITH AN OPTICAL SYSTEM |
Description | A method of characterizing an optical system, wherein the optical system comprises an optical fibre having a proximal end and a distal end, wherein the optical fibre comprises a non-single-mode optical fibre, and comprising a reflector assembly comprising a stack of reflectors disposed at the distal end of the optical fibre, wherein the stack of reflectors is arranged to provide different reflector matrices in dependence on illumination wavelength. The method comprises projecting calibration patterns at a plurality of characterization wavelengths onto the proximal end, then obtaining, at the proximal end, data relating to reflected calibration patterns. An instantaneous transmission matrix of the optical fibre is then determined using the data relating to the reflected calibration patterns and the reflector matrices. |
IP Reference | US2021386277 |
Protection | Patent application published |
Year Protection Granted | 2021 |
Licensed | Yes |
Impact | none yet |
Title | ExVision |
Description | Ex vivo imaging platform for tissue characterisation |
Type | Diagnostic Tool - Imaging |
Current Stage Of Development | Initial development |
Year Development Stage Completed | 2022 |
Development Status | Under active development/distribution |
Impact | none yet |
Title | Hyperspectral Endoscopy |
Description | Endoscopy tool for early detection of cancer |
Type | Diagnostic Tool - Imaging |
Current Stage Of Development | Refinement. Clinical |
Year Development Stage Completed | 2020 |
Development Status | Actively seeking support |
Clinical Trial? | Yes |
Impact | First-in-human trial. Developing new approach for future applications. |
URL | https://clinicaltrials.gov/show/NCT03388047 |
Title | MAPS |
Description | We applied multispectral imaging to transnasal pituitary surgery, however, the results were equivocal so the method is on hold pending further analysis of the current trial data. |
Type | Diagnostic Tool - Imaging |
Current Stage Of Development | Initial development |
Year Development Stage Completed | 2022 |
Development Status | On hold |
Clinical Trial? | Yes |
UKCRN/ISCTN Identifier | n/a |
Impact | none yet |
Title | MuSE |
Description | Spectral endoscopy |
Type | Diagnostic Tool - Imaging |
Current Stage Of Development | Refinement. Clinical |
Year Development Stage Completed | 2020 |
Development Status | Under active development/distribution |
Clinical Trial? | Yes |
Impact | Application in 20 patient pilot clinical trial led to promising clinical data supporting further funding application and translational activities. One patent in preparation. |
Title | OptiMSFA |
Description | Multispectral imaging captures spatial information across a set of discrete spectral channels and is widely utilized across diverse applications such as remote sensing, industrial inspection, and biomedical imaging. Multispectral filter arrays (MSFAs) are filter mosaics integrated atop image sensors that facilitate cost-effective, compact, snapshot multispectral imaging. MSFAs are pre-configured based on application-where filter channels are selected corresponding to targeted absorption spectra-making the design of optimal MSFAs vital for a given application. Despite the availability of many design and optimization approaches for spectral channel selection and spatial arrangement, major limitations remain. There are few robust approaches for joint spectral-spatial optimization, techniques are typically only applicable to limited datasets and most critically, are not available for general use and improvement by the wider community. Here, we reconcile current MSFA design techniques and present Opti-MSFA: a Python-based open-access toolbox for the centralized design and optimization of MSFAs. Opti-MSFA incorporates established spectral-spatial optimization algorithms, such as gradient descent and simulated annealing, multispectral-RGB image reconstruction, and is applicable to user-defined input of spatial-spectral datasets or imagery. We demonstrate the utility of the toolbox by comparing against other published MSFAs using the standard hyperspectral datasets Samson and Jasper Ridge, and further show application on experimentally acquired fluorescence imaging data. In conjunction with end-user input and collaboration, we foresee the continued development of Opti-MSFA for the benefit of the wider research community. |
Type Of Technology | Software |
Year Produced | 2022 |
Open Source License? | Yes |
Impact | none yet |
URL | https://opg.optica.org/oe/fulltext.cfm?uri=oe-30-5-7591&id=469746 |
Description | CRUK-AACR Physical Sciences in Oncology |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Talk followed by interactive panel session |
Year(s) Of Engagement Activity | 2019 |
Description | Convergent Science in Surgical Oncology |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Conference presentation |
Year(s) Of Engagement Activity | 2019 |
Description | Departmental Seminar, University of Toronto Mississauga |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Professional Practitioners |
Results and Impact | Virtual talk |
Year(s) Of Engagement Activity | 2020 |
Description | HOPES Seminar |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Virtual talk |
Year(s) Of Engagement Activity | 2020 |
Description | Light@Bath Workshop |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | National |
Primary Audience | Postgraduate students |
Results and Impact | Participation in sandpit workshop including giving scene setting talk |
Year(s) Of Engagement Activity | 2019 |
Description | Photon 2020 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Virtual talk - plenary |
Year(s) Of Engagement Activity | 2020 |
Description | SPIE Photonics West 2020 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Professional Practitioners |
Results and Impact | Plenary session, two sub-sessions and an educational session delivered. Two sessions chaired. |
Year(s) Of Engagement Activity | 2020 |
Description | Sci-Phy-4-Health Seminar |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Postgraduate students |
Results and Impact | Educational session for PhD students |
Year(s) Of Engagement Activity | 2019 |
Description | University of Exeter Physics Colloquium |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Professional Practitioners |
Results and Impact | Local seminar |
Year(s) Of Engagement Activity | 2019 |