Multidimensional endoscopy for early cancer detection

Lead Research Organisation: University of Cambridge
Department Name: Physics

Abstract

Each year in the UK, over 55,000 patients will be diagnosed with oesophageal or lung cancer; a staggering 44,000 will die from their disease. The majority of patients are currently diagnosed when their disease is at a late stage (with regional or distant spread), yielding less than 10% survival over 5 years. Therefore, a key clinical unmet need remains for early disease diagnosis.

To detect these cancers early, patients that are at a high risk of disease might undergo endoscopy. An endoscope is a long, thin, flexible tube with a light source and video camera at one end to relay images from inside the body. They replicate what our eye could see if we were able to look inside the body. Endoscopy is vital for surveillance and diagnosis of cancer, however, many cancers of the oesophagus (food pipe) and lung start out as small abnormalities that are hard to spot using conventional endoscopes.

The focus of this research is to augment the vision of the endoscopist by designing a specialist video camera that detects the properties of light that our eyes are blind to. We will also use this new camera in a benchtop system that can analyse tissue that has been taken out of the body (biopsied) very quickly, before it is sent to a pathologist to check for cancer. Our new cameras are designed to be cost effective and provide extra information to help doctors with diagnosing cancer in the oesophagus and lung, and could ultimately replace the current standard-of-care.

Planned Impact

Stakeholders that will benefit from this research include:
- Patients, through: more accessible testing (lower cost, less specialist equipment); reduced anxiety (better informed at point of test); improved outcomes (reduced mortality); and improved quality of life (fewer invasive interventions).
- Endoscopists and bronchoscopists, through: better imaging contrast that improves interpretation; reduced procedure time; and improvements in patient management.
- Endoscopy manufacturers, such as Olympus, Fuji and Pentax, and start-ups, such as Endochoice, who may benefit from our IP generation by increasing the capability of endoscopes to incorporate new techniques, involving additional research in image processing, visualisation, augmented reality as well as hardware, creating new jobs not currently present in the endoscopy field.
- Pathologists and surgeons, through: application of the MITA system, which could be deployed cost effectively, to help guide pathological sectioning and surgical margin delineation.
- Public service providers, such as the NHS, through: cost reductions in terms of procedure time as well as capital equipment. Our "5 in 1" endoscopy enables short procedures and cost effective deployment. Our MITA system has been designed through user engagement with endoscopists so is small, portable, and can fit into current facilities. Nanoplasmonic metapixel arrays should be highly robust so could also reduce service costs.
- General public, through: research engagement activites to demonstrate the power of multidisciplinary research, increasing awareness of the role of physical sciences in healthcare.

Publications

10 25 50
 
Description EU Biomed Alliance Task Force Member
Geographic Reach Europe 
Policy Influence Type Membership of a guideline committee
Impact The Biomedical Alliance in Europe is the result of a unique initiative of leading European medical societies that together include more than 400,000 researchers and health professionals. The BioMed Alliance was created in 2010 to gather strength across different disciplines and areas in biomedical and health research at European level.
URL https://www.biomedeurope.org/
 
Description Alliance for Cancer Early Detection Cambridge Member Centre
Amount £3,300,000 (GBP)
Organisation Cancer Research UK 
Sector Charity/Non Profit
Country United Kingdom
Start 04/2019 
End 03/2024
 
Description CRUK Spark Award
Amount £20,000 (GBP)
Funding ID C47594/A26851 
Organisation Cancer Research UK 
Sector Charity/Non Profit
Country United Kingdom
Start 01/2019 
End 12/2019
 
Description Cambridge-Tsinghua Joint Research Initiative
Amount £11,250 (GBP)
Organisation University of Cambridge 
Sector Academic/University
Country United Kingdom
Start 01/2020 
End 12/2020
 
Description EPSRC - Impact Acceleration Account Follow on Fund
Amount £60,000 (GBP)
Organisation ESRC Impact Acceleration Account Cambridge 
Sector Academic/University
Country United Kingdom
Start 02/2021 
End 03/2022
 
Title HySE - hyperspectral endoscopy 
Description This method allows for the collection of hyperspectral imaging data from endoscopy, which enables the resolution of different lesion types in the gastrointestinal tract. 
Type Of Material Technology assay or reagent 
Year Produced 2019 
Provided To Others? Yes  
Impact Methodology has been tested in 2 clinical studies resulting in 3 manuscripts Methodology has been patented and licenced to a spin out company supported by NLC 
 
Title A background correction method to compensate illumination variation in hyperspectral imaging 
Description Funder: FP7 People: Marie-Curie Actions; funder-id: http://dx.doi.org/10.13039/100011264; Grant(s): FP7-PEOPLE-2013-CIG-630729 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact Not known 
URL https://www.repository.cam.ac.uk/handle/1810/303457
 
Title Dataset for: Characterizing optical fiber transmission matrices using metasurface reflector stacks for lensless imaging without distal access 
Description  
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
URL https://www.repository.cam.ac.uk/handle/1810/305512
 
Title Dataset for: First experience in clinical application of hyperspectral endoscopy for evaluation of colonic polyps 
Description All code and processed images used to prepare the figures associated with this manuscript. Raw endoscopy videos can be obtained on request from the corresponding author. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact Dataset was used for subsequent analysis that led to further publishable outputs and a patent filing. 
URL https://www.repository.cam.ac.uk/handle/1810/323059
 
Title Dataset for: First-in-human pilot study of snapshot multispectral endoscopy for early detection of Barrett's-related neoplasia. 
Description Research data and code arising from a first-in-human endoscopy clinical trial (NCT03388047). Each of the data tables contains 9-point multispectral image spectra created from multispectral images captured using a snapshot multispectral endoscope. The raw images were processed into these image spectra using the code in 'PROCESSING CODE'. Raw image data can be provided upon request. The tissue spectra were captured in a first-in-human pilot study in the oesophagus. The colour spectra were captured from an X-rite colour target. Each row of the arrays contains the information about the spectra, most importantly, the labels from pathology. The different arrays contain spectra from averages over each multispectral image (processed_tissue_spectra.mat), each patient (processed_tissue_spectra_avg_per_trial.mat and processed_tissue_spectra_avg_per_trial_per_region_then_pooled.mat), and over all patients (processed_tissue_spectra_avg_overall.mat). Please see included readme files for further details. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact None yet. 
URL https://www.repository.cam.ac.uk/handle/1810/329675
 
Title Dataset for: Quantitative phase and polarization imaging through an optical fiber applied to detection of early esophageal tumorigenesis 
Description  
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
URL https://www.repository.cam.ac.uk/handle/1810/304918
 
Title Dataset for: Spectral endoscopy combined with computational modelling and deep-learning yield high-contrast for oesophageal neoplasia 
Description Data upload contains data tables of attenuation and reflection spectra. Also contains the code used to generate the manuscript Figures and Tables from the data. Raw endoscopy videos can be obtained upon request from the corresponding author of the manuscript. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact Supported investigations that subsequently led to further publications and a patent filing 
URL https://www.repository.cam.ac.uk/handle/1810/323051
 
Title Deep learning applied to hyperspectral endoscopy for online spectral classification. 
Description Hyperspectral imaging (HSI) is being explored in endoscopy as a tool to extract biochemical information from tissue optical properties that may improve contrast for early cancer detection in the gastrointestinal tract. Motion artefacts during medical endoscopy have traditionally limited HSI application, however, recent developments in the field have led to real-time HSI deployments. Unfortunately, traditional HSI analysis methods remain unable to handle the volume of hyperspectral data in order to provide real-time feedback to the operator. Here, a convolutional neural network (CNN) is proposed to enable online classification of data obtained during HSI endoscopy. A five-layered CNN was trained and fine-tuned on a dataset of 300 hyperspectral endoscopy images acquired from a planar Macbeth ColorChecker chart and was able to distinguish between its 18 constituent colors with an average accuracy of 94.3% achieved at 8.8 fps. Performance was then tested on a set of images simulating an endoscopy environment, consisting of color charts warped inside a rigid tube mimicking a lumen. The algorithm improved robust to such variations, with classification accuracies over 90% being obtained despite the variations, with an average drop in accuracy of 2.4% being registered at the points of longest working distance and most inclination. For further validation of the color-based classification system, ex vivo videos of a methylene blue dyed pig esophagus and images of different cancer stages in the human esophagus were analyzed, showing spatially distinct color classifications. These results suggest that the CNN has potential to provide color-based classification during real-time HSI in endoscopy. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact Enquiries regarding use of the dataset 
URL https://www.repository.cam.ac.uk/handle/1810/302170
 
Title Hyperspectral Colour Classification (HCC) 
Description Dataset utilised for the training and testing of the spectral classification system for a line-scanning hyperspectral endoscope and the associated core algorithms. See the Readme file for a detailed description. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact None known 
URL https://www.repository.cam.ac.uk/handle/1810/303062
 
Title Research data supporting "A background correction method to compensate illumination variation in hyperspectral imaging" 
Description The dataset includes experimental and simulation data and the codes used in the paper "A background correction method to compensate illumination variation in hyperspectral imaging". All experimental data is captured using hyperspectral endoscopy (https://www.nature.com/articles/s41467-019-09484-4). See the Readme file for detailed information. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact Not known 
URL https://www.repository.cam.ac.uk/handle/1810/302032
 
Title Robustness to misalignment of low-cost, compact quantitative phase imaging architectures 
Description Non-interferometric approaches to quantitative phase imaging could enable its application in low-cost, miniaturised settings such as capsule endoscopy. We present two possible architectures and both analyse and mitigate the effect of sensor misalignment on phase imaging performance. This is a crucial step towards determining the feasibility of implementing phase imaging in a capsule device. First, we investigate a design based on a folded 4f correlator, both in simulation and experimentally. We demonstrate a novel technique for identifying and compensating for axial misalignment and explore the limits of the approach. Next, we explore the implications of axial and transverse misalignment, and of manufacturing variations on the performance of a phase plate-based architecture, identifying a clear trade-off between phase plate resolution and algorithm convergence time. We conclude that while the phase plate architecture is more robust to misalignment, both architectures merit further development with the goal of realising a low-cost, compact system for applying phase imaging in capsule endoscopy. 
Type Of Material Database/Collection of data 
Year Produced 2020 
Provided To Others? Yes  
Impact None known 
URL https://www.repository.cam.ac.uk/handle/1810/309619
 
Description Johnson & Johnson 
Organisation Johnson & Johnson
Country United States 
Sector Private 
PI Contribution We are bringing IP in optical filter fabrication to enable multispectral imaging using OVT sensors by direct deposition of filters using electron beam lithography and photolithography
Collaborator Contribution J&J (Auris) are providing bare silicon wafers for us to test our filter technologies for direct deposition onto CMOS image sensors
Impact Multidisciplinary, engineering, physics, medical. No outputs as yet
Start Year 2019
 
Description Omnivision Technologies 
Organisation OmniVision Technologies Inc
Country United States 
Sector Private 
PI Contribution We are bringing IP in optical filter fabrication to enable multispectral imaging using OVT sensors by direct deposition of filters using electron beam lithography and photolithography
Collaborator Contribution OVT are providing bare silicon wafers for us to test our filter technologies for direct deposition onto CMOS image sensors
Impact Multidisciplinary, engineering, physics, medical No outputs as yet
Start Year 2018
 
Title A SINGLE STEP LITHOGRAPHY COLOUR FILTER 
Description We disclose herein a method of producing an optical filter. The method comprising depositing a first mirror layer onto a substrate; depositing an insulating layeron the first mirror; exposing at least some of a plurality of portions of a surface of the insulating layer to a dose of energy; developing the insulating layer in order to remove a volume from the at least some of the plurality of portions of the insulating layer, wherein the volume of the insulating layer removed from each portion is related to the dose of energy exposed to each portion, and wherein a remaining thickness after the removal of the volume from each portion of the insulating layer is relatedto the dose of energy exposed to each portion. The method further comprising depositing a second mirror layer on the remaining thickness of each of the plurality of portions of the insulating layer such that the remaining thickness of each of the plurality of portions of the insulating layer define a profile of the optical filter. 
IP Reference WO2019239139 
Protection Patent application published
Year Protection Granted 2019
Licensed Yes
Impact none yet
 
Title APPARATUS AND METHOD FOR WIDE-FIELD HYPERSPECTRAL IMAGING 
Description Embodiments of the present invention provide a hyperspectral endoscope system, comprising a memory for storing data therein, an endoscope arranged to, in use, receive radiation reflected from a sample and to output wide-field image data and line-scan hyperspectral data corresponding to the sample, a processor coupled to the memory, wherein the processor is arranged, in use, to determine registration information between portions of the wide-field image data, and determine wide-area hyperspectral image data in dependence on the registration information and the line-scan hyperspectral data. 
IP Reference US2021374981 
Protection Patent application published
Year Protection Granted 2021
Licensed Yes
Impact None yet
 
Title METHODS OF CHARACTERISING AND IMAGING WITH AN OPTICAL SYSTEM 
Description A method of characterizing an optical system, wherein the optical system comprises an optical fibre having a proximal end and a distal end, wherein the optical fibre comprises a non-single-mode optical fibre, and comprising a reflector assembly comprising a stack of reflectors disposed at the distal end of the optical fibre, wherein the stack of reflectors is arranged to provide different reflector matrices in dependence on illumination wavelength. The method comprises projecting calibration patterns at a plurality of characterization wavelengths onto the proximal end, then obtaining, at the proximal end, data relating to reflected calibration patterns. An instantaneous transmission matrix of the optical fibre is then determined using the data relating to the reflected calibration patterns and the reflector matrices. 
IP Reference US2021386277 
Protection Patent application published
Year Protection Granted 2021
Licensed Yes
Impact none yet
 
Title ExVision 
Description Ex vivo imaging platform for tissue characterisation 
Type Diagnostic Tool - Imaging
Current Stage Of Development Initial development
Year Development Stage Completed 2022
Development Status Under active development/distribution
Impact none yet 
 
Title Hyperspectral Endoscopy 
Description Endoscopy tool for early detection of cancer 
Type Diagnostic Tool - Imaging
Current Stage Of Development Refinement. Clinical
Year Development Stage Completed 2020
Development Status Actively seeking support
Clinical Trial? Yes
Impact First-in-human trial. Developing new approach for future applications. 
URL https://clinicaltrials.gov/show/NCT03388047
 
Title MAPS 
Description We applied multispectral imaging to transnasal pituitary surgery, however, the results were equivocal so the method is on hold pending further analysis of the current trial data. 
Type Diagnostic Tool - Imaging
Current Stage Of Development Initial development
Year Development Stage Completed 2022
Development Status On hold
Clinical Trial? Yes
UKCRN/ISCTN Identifier n/a
Impact none yet 
 
Title MuSE 
Description Spectral endoscopy 
Type Diagnostic Tool - Imaging
Current Stage Of Development Refinement. Clinical
Year Development Stage Completed 2020
Development Status Under active development/distribution
Clinical Trial? Yes
Impact Application in 20 patient pilot clinical trial led to promising clinical data supporting further funding application and translational activities. One patent in preparation. 
 
Title OptiMSFA 
Description Multispectral imaging captures spatial information across a set of discrete spectral channels and is widely utilized across diverse applications such as remote sensing, industrial inspection, and biomedical imaging. Multispectral filter arrays (MSFAs) are filter mosaics integrated atop image sensors that facilitate cost-effective, compact, snapshot multispectral imaging. MSFAs are pre-configured based on application-where filter channels are selected corresponding to targeted absorption spectra-making the design of optimal MSFAs vital for a given application. Despite the availability of many design and optimization approaches for spectral channel selection and spatial arrangement, major limitations remain. There are few robust approaches for joint spectral-spatial optimization, techniques are typically only applicable to limited datasets and most critically, are not available for general use and improvement by the wider community. Here, we reconcile current MSFA design techniques and present Opti-MSFA: a Python-based open-access toolbox for the centralized design and optimization of MSFAs. Opti-MSFA incorporates established spectral-spatial optimization algorithms, such as gradient descent and simulated annealing, multispectral-RGB image reconstruction, and is applicable to user-defined input of spatial-spectral datasets or imagery. We demonstrate the utility of the toolbox by comparing against other published MSFAs using the standard hyperspectral datasets Samson and Jasper Ridge, and further show application on experimentally acquired fluorescence imaging data. In conjunction with end-user input and collaboration, we foresee the continued development of Opti-MSFA for the benefit of the wider research community. 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact none yet 
URL https://opg.optica.org/oe/fulltext.cfm?uri=oe-30-5-7591&id=469746
 
Description CRUK-AACR Physical Sciences in Oncology 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Talk followed by interactive panel session
Year(s) Of Engagement Activity 2019
 
Description Convergent Science in Surgical Oncology 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Conference presentation
Year(s) Of Engagement Activity 2019
 
Description Departmental Seminar, University of Toronto Mississauga 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Virtual talk
Year(s) Of Engagement Activity 2020
 
Description HOPES Seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Virtual talk
Year(s) Of Engagement Activity 2020
 
Description Light@Bath Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact Participation in sandpit workshop including giving scene setting talk
Year(s) Of Engagement Activity 2019
 
Description Photon 2020 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Virtual talk - plenary
Year(s) Of Engagement Activity 2020
 
Description SPIE Photonics West 2020 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Plenary session, two sub-sessions and an educational session delivered. Two sessions chaired.
Year(s) Of Engagement Activity 2020
 
Description Sci-Phy-4-Health Seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Educational session for PhD students
Year(s) Of Engagement Activity 2019
 
Description University of Exeter Physics Colloquium 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Local seminar
Year(s) Of Engagement Activity 2019