Correlative Refractive Index Light-sheet Microscopy
Lead Research Organisation:
University of Dundee
Department Name: Physics
Abstract
Developing future therapies, the first of EPSRC's Healthcare Technologies Grand Challenges, is essential to keep the National Health Service sustainable. Currently, an estimated 70% of the UK's healthcare expenditure goes towards the management of chronic diseases. Regenerative medicine is expected to significantly reduce costs as it can turn chronic, degenerative, diseases into curable conditions. Realising this potential requires the right tools to study the biological development process as it progresses from the single cell to the complex structure of entire organs.
The invention of the optical microscope, and in particular the phase contrast microscope, made it possible to highlight the fine features of living cells with unprecedented clarity. However, cells isolated on a microscope slide often do not behave as tissue in its natural, three-dimensional, environment. The recent development of the planar illumination light-sheet microscope enabled the visualisation of the intact, fluorescently-labelled, organisms during development. While light-sheet microscopy is highly successful for transparent zebrafish or chemically cleared tissue, many tissues are too opaque to be studied beyond the first layer of cells. A prime example is the chick embryo in its early stages of development. The complex collective behaviour of the cells in the initial layer can be studied in exquisite detail, yet as soon as the primitive streak forms, to grow the embryo in the third dimension, we lack the tools to keep track of the cell migration and differentiation. The images are too blurred.
Correlative refractive index light-sheet microscopy aims to make the invisible visible. It is based on the realization that the turbidity of biological samples is due to the same refractive index variations that yield structural information in phase-contrast microscopy. The distribution of optical properties within the specimen not only contains valuable structural information for the biologist, it also enables the adaptive wavefront correction needed for high resolution fluorescence imaging. By developing a hybrid instrument that maps the optical property distribution in parallel with the fluorescence image, this proposal will enable high-resolution deep-tissue imaging in turbid biological specimen. A direct view into the inner workings of the biological development process is essential to develop effective regenerative-medicine therapies.
The invention of the optical microscope, and in particular the phase contrast microscope, made it possible to highlight the fine features of living cells with unprecedented clarity. However, cells isolated on a microscope slide often do not behave as tissue in its natural, three-dimensional, environment. The recent development of the planar illumination light-sheet microscope enabled the visualisation of the intact, fluorescently-labelled, organisms during development. While light-sheet microscopy is highly successful for transparent zebrafish or chemically cleared tissue, many tissues are too opaque to be studied beyond the first layer of cells. A prime example is the chick embryo in its early stages of development. The complex collective behaviour of the cells in the initial layer can be studied in exquisite detail, yet as soon as the primitive streak forms, to grow the embryo in the third dimension, we lack the tools to keep track of the cell migration and differentiation. The images are too blurred.
Correlative refractive index light-sheet microscopy aims to make the invisible visible. It is based on the realization that the turbidity of biological samples is due to the same refractive index variations that yield structural information in phase-contrast microscopy. The distribution of optical properties within the specimen not only contains valuable structural information for the biologist, it also enables the adaptive wavefront correction needed for high resolution fluorescence imaging. By developing a hybrid instrument that maps the optical property distribution in parallel with the fluorescence image, this proposal will enable high-resolution deep-tissue imaging in turbid biological specimen. A direct view into the inner workings of the biological development process is essential to develop effective regenerative-medicine therapies.
Planned Impact
The immediate impact of this project will undoubtedly be felt in biology and in particular developmental biology where progress in the state-of-the-art in microscopy directly dictates the boundaries of knowledge. Knowledge that is of strategic importance for the development of effective therapies through regenerative medicine and crucial to reduce the financial pressures on the National Health Service. A microscope that can image large turbid samples with high contrast and resolution will break through the imaging depth barrier in biological samples. To facilitate adoption, the prototype instrument must however be easy to use and readily available to biology research groups and biotech or pharmaceutical companies. I therefore hope to leverage my ties with industry and employ my experience in commercializing imaging solutions to the outcomes of the grant. In particular life science instrumentation companies with an established market and contacts in the biomedical and pharmaceutical research will be able to bring the novel technology to market on the shortest possible time scale. To achieve this, M Squared Lasers Ltd is involved from the outset with this Fellowship.
Furthermore, this research project will yield much information about the optical properties of such samples. At present our knowledge of the optical properties within thick biological specimen is limited. It is anticipated that a tool to measure the optical properties of large specimen in detail will lead to a broader range of studies and greater insights into the diversity of the optical property distributions between species and their variation during development. This places us in a privileged position to direct the development of the next generation of microscope technology here in the UK.
Furthermore, this research project will yield much information about the optical properties of such samples. At present our knowledge of the optical properties within thick biological specimen is limited. It is anticipated that a tool to measure the optical properties of large specimen in detail will lead to a broader range of studies and greater insights into the diversity of the optical property distributions between species and their variation during development. This places us in a privileged position to direct the development of the next generation of microscope technology here in the UK.
Organisations
Publications
Hosny NA
(2020)
Planar Airy beam light-sheet for two-photon microscopy.
in Biomedical optics express
Sun Q
(2022)
Compact nano-void spectrometer based on a stable engineered scattering system
in Photonics Research
Sun Q
(2023)
A scattering spectrometer for white light interferometry
in Optics and Lasers in Engineering
Urban D
(2024)
Advancing Optical Coherence Tomography through Opto-Electronic Frequency Shifting
in EPJ Web of Conferences
Urban DR
(2024)
Widefield optical coherence tomography by electro-optical modulation.
in Biomedical optics express
| Title | Supplementary Video Mouse brain tissue.mp4 |
| Description | Three-dimensional visualization with rotating perspective of 0.60x1.15x0.60mm of Thy-1-GFP-expressing mouse brain tissue, imaged using the two-photon planar Airy light-sheet. |
| Type Of Art | Film/Video/Animation |
| Year Produced | 2020 |
| Impact | The novel, wide-field higher resolution, light-sheet microscopy technique is now commercialized by M Squared Ltd, a company with a global customer base. |
| URL | https://opticapublishing.figshare.com/articles/media/Supplementary_Video_Mouse_brain_tissue_mp4/1214... |
| Description | We discovered a method to turn the curved Airy beam into a planar sheet of light. Such illumination is important for light-sheet microscopes to capture high-resolution 3D images of biological samples, such as the brain, and study their working and development. Conventional laser beams spreads out due to diffraction, limiting the resolution and field-of-view that can be imaged, and thereby the biological conclusions. In contrast, the Airy beam, after Lord Airy, does not spread out as it propagates; however, it follows a curved trajectory. This creates distortion and the images need digital correction. More importantly, it limits the use of multi-photon excitation to see deeper into the sample. By twisting the beams in just the right way, our new technique illuminates the sample with a thin propagation-invariant Airy light-sheet. We used this to imaging neuronal connections in brain tissue on the mm-scale, yet with sufficient resolution to resolve the dendritic spines connecting the neurons. We also developed a method to study the scattering of light waves in complex materials. This is important for the development of the next generation of microscopy technique that allow us to see deeper into biological samples. Such tissue is typically a heterogenous mixture of materials with a range of different optical properties. This scatters the light, thus causing a blur that prevents conventional microscopes from visualising deeper layers. Under certain conditions, adaptive optics and wavefront shaping can undo the scattering, though these methods require precise knowledge of how the waves are scattered. Direct measurements are often impractical and always error prone. On the other hand, solving Maxwell's equations for light used to be a challenge for anything larger than a single biological cell. The computational method that we developed can scale up such calculations by more than two orders of magnitude. My mapping the vector Helmholtz equation onto the structure of the neural network, we were able to leverage tools from machine learning to bring such calculations to a scale relevant in microscopy. With this, we showed that machine learning tools can indeed be used to rigorously solve numeric problems. Neural networks are rapidly transforming our lives by mimicking how humans solve problems. Perhaps unsurprisingly, this carries with it the potential of errors and biases. We showed that a middle ground exists to solve challenging scientific problems with the mathematical rigour expected of scientific computations. |
| Exploitation Route | 1. The planar Airy beam light-sheet microscope is of immediate use in the life sciences and has commercial potential. 2. The open-source code made available with our paper can immediately be used to solve wave problems in acoustics and electrodynamics. We anticipate that it can also be used to solve more general problems such as diffusion in complex structures, and have published a pre-print to demonstrate how. |
| Sectors | Healthcare Other |
| URL | https://corilim.github.io/ |
| Description | The main example of impact from this project is the planar Airy beam light-sheet microscope that we developed. This has been commercialized by M Squared Ltd for their worldwide market. The availability of the instrument provides life scientists with a complete picture, both the large and the small. While developing this technology here is creating highly skilled jobs in a high-value-added UK industry. |
| First Year Of Impact | 2020 |
| Impact Types | Economic |
| Description | Computational Light Microscopy |
| Amount | £45,828 (GBP) |
| Funding ID | 2502350 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 07/2020 |
| End | 09/2023 |
| Description | Hybrid Optical-Digital Coherence Tomography |
| Amount | £50,000 (GBP) |
| Funding ID | 2607886 |
| Organisation | Engineering and Physical Sciences Research Council (EPSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 08/2021 |
| End | 09/2025 |
| Title | Light wave scattering on machine learning infrastructure |
| Description | Computing light scattering in complex materials such as biological tissue is challenging and difficult to scale up. We developed a method to do so more efficiently by leveraging advances in machine learning infrastructure. The method was integrated into the open source electromagnetic solver MacroMax, thereby expanding the functionality of both. The machine-learning enhanced version of MacroMax is freely available, including its complete source code. This enables researchers to scale up their electromagnetic wave calculations, in some cases by more than 2 orders of magnitude. We implemented and distributed this as a Python package, enabling rapid cloud deployment on e.g. Google Colab. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | The computational method is central to the UKRI Future Leaders Fellowship renewal grant award "Computational Refractive Index Light-sheet Microscopy (CORILIM)". We are aware that it is actively used by multiple research groups around the world. The Python package that we developed routinely is routinely downloaded over 400 times per month. |
| URL | https://pypi.org/project/macromax/ |
| Title | Data and code for widefield optical coherence tomography by electro-optical modulation |
| Description | The dataset contains the raw experimental data that support the findings in Urban et al.[1], as well as the Matlab scripts to process it. [1] D. R. Urban et al. "Widefield optical coherence tomography by electro-optical modulation," Biomed. Opt. Express 15 (11), 6573-6587 (2024). https://doi.org/10.1364/BOE.540278 |
| Type Of Material | Database/Collection of data |
| Year Produced | 2024 |
| Provided To Others? | Yes |
| Impact | Diseases of the retina often cause blindness and early detection is essential for effective treatment. The first symptoms typically appear in the perifery of the retina, outside the field-of-view of common instruments. The developed technologies improve optical coherence tomography, an important technique for the early detection of eye diseases such as glaucoma, (age-related) macular degeneration, and retinal detachment. These lead to reduced vision and eventually blindness, most certainly a life-altering condition. Early detection is essential to effective treatment. The methods developed and patented are particularly useful to broaden the field-of-view of optical coherence tomography. This can make it possible to study the periphery of the eye, where such diseases tend to develop first. At this stage, our experiments only cover proof-of-principle experiments. Further work is needed to integrate this in existing systems, and later, clinical procedures. The improved imaging capabilities could for instance be used to measure the angle between the iris and the cornea to detect Glaucoma in the anterior segment. Wide area imaging of detect retinal detachment where it first happens. Detection of macular degeneration could potentially benefit from improved signal-to-noise, though this may not be the strongest use case. Eye disease is often first noticed in the periphery. Since the retina and anterior segment are highly curved, any imaging system for the eye must be able to change the imaging depth dynamically during a scan to adapt to the curvature. The developed techniques allow fast (microsecond-scale) electro-optical refocussing of the swept-source OCT system. |
| URL | https://discovery.dundee.ac.uk/en/datasets/data-and-code-for-widefield-optical-coherence-tomography-... |
| Title | Dataset to support the publication 'Compact nano-void spectrometer based on a stable engineered scattering system' |
| Description | Falak, Przemyslaw Ludwik (2022) Dataset to support the publication 'Compact nano-void spectrometer based on a stable engineered scattering system'. University of Southampton doi:10.5258/SOTON/D2314 [Dataset] |
| Type Of Material | Database/Collection of data |
| Year Produced | 2022 |
| Provided To Others? | Yes |
| Impact | Qi Sun, Przemyslaw Falak, Tom Vettenburg, Timothy Lee, David B. Phillips, Gilberto Brambilla, Martynas Beresna "Compact nano-void spectrometer based on a stable engineered scattering system" Photonics Research (Impact Factor 7.25), 10, 2328-2336 (2022) doi:10.1364/PRJ.465322, https://opg.optica.org/prj/fulltext.cfm?uri=prj-10-10-2328&id=506534 |
| URL | https://eprints.soton.ac.uk/468356/ |
| Title | Datasets used in Scaling up wave calculations with a Scattering Network |
| Description | The complex-valued field distributions and matrices used to produce the figures in the manuscript "Scaling up wave calculations with a Scattering Network" Intelligent Computing (2024). Data is encoded in Python's NumPy format. Files with the .npy extension store individual complex-valued ndarrays with the field values and permittivity distribution descriptions. Files with the .npz extension store multiple ndarrays to describe the scattering and deposition matrices. The final high resolution / vector graphics versions of the figures are included in Portable Document Format (.pdf). |
| Type Of Material | Database/Collection of data |
| Year Produced | 2024 |
| Provided To Others? | Yes |
| Impact | The computational method is central to the UKRI Future Leaders Fellowship renewal grant award "Computational Refractive Index Light-sheet Microscopy (CORILIM)". We are aware that it is actively used by multiple research groups around the world. The Python package that we developed routinely is routinely downloaded over 400 times per month. |
| URL | https://discovery.dundee.ac.uk/en/datasets/datasets-used-in-scaling-up-wave-calculations-with-a-scat... |
| Title | AnySim - Framework for solving arbitrary linear systems |
| Description | A numerical solver for large linear problems of the form Ax = y, using the universal split preconditioner. |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| Impact | Tom Vettenburg, Ivo M. Vellekoop "A universal preconditioner for linear systems" https://arxiv.org/abs/2207.14222, doi: 10.48550/arXiv.2207.14222 |
| URL | https://github.com/corilim/anysim |
| Title | Pytorch integration in Macroscopic Maxwell Solver |
| Description | Open-source software to solve large-scale wave problems efficiently. By integrating a recurrent neural network approach using the machine learning library Pytorch, for the first time light-scattering can be computed on the millimetre scale that is relevant to microscopy. |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| Impact | By integrating a recurrent neural network approach using the machine learning library Pytorch, for the first time light-scattering can be computed on the millimetre scale that is relevant to microscopy. Valantinas and Vettenburg "A physics-defined recurrent neural network to compute coherent light wave scattering on the millimetre scale", https://arxiv.org/abs/2208.01118, doi: 10.48550/arXiv.2208.01118 |
| URL | https://github.com/corilim/MacroMax |
| Title | ZmxTools - A toolkit to open Zemax OpticStudio files |
| Description | Laborary optical components are often only described by Zemax OpticStudio zar files. This tool unpacks proprietary Zemax OpticStudio zar files and makes the information directly readible for the customers of such optical components. AGPL3 License: GNU Affero General Public License v3 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| Impact | This project has found use in the optics community. |
| URL | https://github.com/corilim/zmxtools |
| Description | Girls into physics - Monifeith |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Schools |
| Results and Impact | Eight hands-on workshops on optical spectroscopy in astronomy, aimed exclusively at female secondary school pupils. The aim of these workshops is to encourage them to consider STEM subjects for their further studies and improve the gender balance in physics. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Sculpting light from optically powered micro-robots to computational transparency |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | Sculpting light: from optically powered micro-robots to computational transparency a Science-on-the-Tay talk by Prof David Phillips - University of Exeter The purpose of this IOP sponsored event is to foster interest in physics and science as a whole. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://corilim.github.io/outreach/scienceonthetay/ |
| Description | Workshop: experimental physics kits for high schools in Scotland. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Weekend workshop with secondary school physics teachers and the institute of physics with the aim to develop and improve current physics teaching equipment available to schools. "Workshop on Photonics and Quantum Technologies" 19 Nov 2021 - 20 Nov 2021 |
| Year(s) Of Engagement Activity | 2021 |