Digital holographic microscopy for tracking micro-organisms in 3D
Lead Research Organisation:
University of Oxford
Department Name: Oxford Physics
Abstract
The majority of single-celled organisms actively navigate their environment. Many species of bacteria, for instance, use a tiny rotary motor to drive a helical filament to generate propulsion. In the case of bacteria, active navigation is required to source nutrients and to avoid toxins, to form symbiotic relationships with each other, and to move towards sites for pathogenic invasion. Currently, the only method we have of studying bacterial swimming is to look at 2D images from a conventional microscope and to infer the 3D nature of the swimming as the bacteria swim in and out of the microscope focus. A full 3D understanding of bacterial swimming is vital to probe many of the unanswered questions about how they move.
The aim of this project is to develop a new microscopy technique that is capable of recording the 3D positions of swimming bacteria at high speed. This will be achieved through developing a technique known as digital holographic microscopy. A hologram is an interference pattern between light emanating from a sample and a known reference beam. The fact that we know what this 'reference beam' is a priori means we can process the interference pattern in a computer to generate three-dimensional information about the sample we have imaged. A digital holographic microscope uses a microscope objective to greatly magnify the sample before forming the hologram. In this way we can obtain the 3D positions of organisms as small as bacteria, which are 1-2 microns long.
Two recent technological developments allow us to study these microorganisms in 3D, and at high speeds. The first is the advance in digital camera technology. New camera chips now have sufficient pixel resolution to capture the fine detail of these magnified holograms at very fast frame-rates (up to 2,000 frames per second). The second is the availability of graphical processing units (GPUs) for image processing. GPUs were originally invented for the computer gaming industry, but have now found a place in digital processing applications as they are far more efficient at processing large volumes of data than conventional CPUs. Our 3D holographic microscope will necessarily generate huge amounts of data (1GB per frame for a typical 3D hologram); hence, GPUs are integral to the handling of that information. One of the largest stumbling blocks that explains why digital holographic imaging has not yet been used in this manner is the lack of available GPU-compatible software to effectively process these volumes of data.
Our proposed technology will bring together microscopy, high-speed, high-resolution digital cameras and the processing power of GPUs to enable the three-dimensional study of bacteria and other microorganisms. It will involve a specific form of holography called 'off-axis holography'. This technique is more difficult to implement optically at higher magnifications, yet it offers improved resolution over 'inline holography' - the other main branch of the technology.
Having developed 3D microscopic imaging, we propose two methods in which it will be used to study how bacteria swim. The first uses a relatively low magnification microscope (40x) to capture holograms of many bacteria swimming in one field of view. We will use computer software to track the motion of these bacteria in 3D. In this way, we can investigate the collective swimming behaviour of a population of organisms. The second method uses higher magnification (225x) optics to investigate how individual bacteria interact with their fluid environment. This example would capture the motion of tiny 'tracer' particles that allow us to visualize the fluid flow caused by a single bacterium swimming past.
We anticipate that this technology would have a wider appeal to many branches of science interested in how microorganisms and other cells - such as sperm - move.
The aim of this project is to develop a new microscopy technique that is capable of recording the 3D positions of swimming bacteria at high speed. This will be achieved through developing a technique known as digital holographic microscopy. A hologram is an interference pattern between light emanating from a sample and a known reference beam. The fact that we know what this 'reference beam' is a priori means we can process the interference pattern in a computer to generate three-dimensional information about the sample we have imaged. A digital holographic microscope uses a microscope objective to greatly magnify the sample before forming the hologram. In this way we can obtain the 3D positions of organisms as small as bacteria, which are 1-2 microns long.
Two recent technological developments allow us to study these microorganisms in 3D, and at high speeds. The first is the advance in digital camera technology. New camera chips now have sufficient pixel resolution to capture the fine detail of these magnified holograms at very fast frame-rates (up to 2,000 frames per second). The second is the availability of graphical processing units (GPUs) for image processing. GPUs were originally invented for the computer gaming industry, but have now found a place in digital processing applications as they are far more efficient at processing large volumes of data than conventional CPUs. Our 3D holographic microscope will necessarily generate huge amounts of data (1GB per frame for a typical 3D hologram); hence, GPUs are integral to the handling of that information. One of the largest stumbling blocks that explains why digital holographic imaging has not yet been used in this manner is the lack of available GPU-compatible software to effectively process these volumes of data.
Our proposed technology will bring together microscopy, high-speed, high-resolution digital cameras and the processing power of GPUs to enable the three-dimensional study of bacteria and other microorganisms. It will involve a specific form of holography called 'off-axis holography'. This technique is more difficult to implement optically at higher magnifications, yet it offers improved resolution over 'inline holography' - the other main branch of the technology.
Having developed 3D microscopic imaging, we propose two methods in which it will be used to study how bacteria swim. The first uses a relatively low magnification microscope (40x) to capture holograms of many bacteria swimming in one field of view. We will use computer software to track the motion of these bacteria in 3D. In this way, we can investigate the collective swimming behaviour of a population of organisms. The second method uses higher magnification (225x) optics to investigate how individual bacteria interact with their fluid environment. This example would capture the motion of tiny 'tracer' particles that allow us to visualize the fluid flow caused by a single bacterium swimming past.
We anticipate that this technology would have a wider appeal to many branches of science interested in how microorganisms and other cells - such as sperm - move.
Technical Summary
Digital holographic microscopy
Off-axis is superior to inline holography in that the phase of a sample can be recovered, speckle noise is reduced for dense samples, and there is less uncertainty in 3D localization of particles. One of the difficulties with implementing high-magnification off-axis holography is that the unwanted illumination beam needs to be directed away from the camera having illuminated the sample. This is made difficult by the short focal length and high numerical aperture of high-powered microscope objectives. We will introduce a conjugate back focal plane in the imaging train and insert a mask that will eliminate the unwanted illumination beam, which is focused to a point in the back focal plane.
Holographic reconstruction on graphical processing units
Due to the large volume of 3D holographic data (~1GB/frame) GPUs are the only realistic option for reconstructing 3D holographic data in a sensible timeframe. Code will be written to interface with CUDA - the GPU language - to reconstruct the 3D light-field, identify particle positions through thresholding and centroid fitting, output 3D location data and track particles between frames. An interactive graphical user interface (GUI) will be made to facilitate use of the software by non-specialists.
Characterization of the capabilities of the DHM
We will combine theoretical modeling of light scattering to determine the performance of inline and off-axis methods as a function of particle size, illumination intensity and frame rate.
Application of the DHM to the study of swimming bacteria
We will track populations of swimming bacteria in a range of biological contexts relevant to chemotaxis and bio-film formation, and gold nano-spheres as tracer particles to map the flow-fields around individual swimming bacteria.
Possible wider applications
We will investigate other applications of the method, for example characterization of sperm motility, in academic and agricultural applications.
Off-axis is superior to inline holography in that the phase of a sample can be recovered, speckle noise is reduced for dense samples, and there is less uncertainty in 3D localization of particles. One of the difficulties with implementing high-magnification off-axis holography is that the unwanted illumination beam needs to be directed away from the camera having illuminated the sample. This is made difficult by the short focal length and high numerical aperture of high-powered microscope objectives. We will introduce a conjugate back focal plane in the imaging train and insert a mask that will eliminate the unwanted illumination beam, which is focused to a point in the back focal plane.
Holographic reconstruction on graphical processing units
Due to the large volume of 3D holographic data (~1GB/frame) GPUs are the only realistic option for reconstructing 3D holographic data in a sensible timeframe. Code will be written to interface with CUDA - the GPU language - to reconstruct the 3D light-field, identify particle positions through thresholding and centroid fitting, output 3D location data and track particles between frames. An interactive graphical user interface (GUI) will be made to facilitate use of the software by non-specialists.
Characterization of the capabilities of the DHM
We will combine theoretical modeling of light scattering to determine the performance of inline and off-axis methods as a function of particle size, illumination intensity and frame rate.
Application of the DHM to the study of swimming bacteria
We will track populations of swimming bacteria in a range of biological contexts relevant to chemotaxis and bio-film formation, and gold nano-spheres as tracer particles to map the flow-fields around individual swimming bacteria.
Possible wider applications
We will investigate other applications of the method, for example characterization of sperm motility, in academic and agricultural applications.
Planned Impact
The primary impact of this research will be knowledge, scientific advancement and the development of a novel micro-imaging instrument. The main beneficiaries of this research will be the UK and international scientific communities. There is also the possibility that a commercial application of the technology will emerge.
Since its invention in the 17th century, the microscope has revolutionized our understanding of the world. However, conventional microscopy is limited to 2D. Generally, cells and microorganisms inhabit a three-dimensional world. While some ground has been made in recent years towards developing 3D microscopic techniques, on the whole, these employ a slice-wise depth-scanning approach, which is too slow to observe the dynamic nature of swimming microorganisms. Fast, high-resolution digital cameras and the increased computing power of graphical processing units offer us the ability to probe, for the first time, the dynamic 3D world of microorganisms using digital holographic microscopy (DHM).
Bacterial swimming is an essential part of the lifestyle of most free living species and is used to move to sites for pathogenic invasion, to form symbiotic relationships or to form bio-films. A digital holographic microscope will give us the ability to probe these behaviours in 3D.
By having a 3D DHM facility in the University of Oxford, we will be able to offer the ability to capture 3D position data to workers in research fields such as; the study of the motility of other microorganisms, experimental verification of theoretical studies on anomalous diffusion, and categorizing flow in novel microfluidic devices.
The impact will spread to the broader scientific community as others make use of and adapt DHM to their research. Commercial applications may also be found for the 3D tracking of microscopic objects. For instance, there is much interest in the screening of sperm cells for a number of different veterinary applications.
Holographic technology could also be spread to the wider public. The raw materials to build a simple holographic microscope are not expensive. DHMs could be introduced to high school and undergraduate science laboratories, allowing students to observe the microworld in 3D, as well as providing a context for learning about optics, computer processing and biological processes. As a test case for this approach, we have already had a 6th form high school summer project student develop a simple holographic imaging setup with a laser pointer in our lab and are offering the opportunity to build a more extensive holographic microscope to a 4th year undergraduate project student in 2012.
Dissemination and Engagement Routes for Maximising Impact
1. Local university research community
Presentations at weekly seminars of the Oxford Physics biological physics community, which includes academics and students from Physics, Chemistry, Biochemistry, and Physiology.
2. UK academic community
Presentation at UK conferences, for example Institute of Physics, British Biophysical Society, Royal Microscopy Society meetings.
3. International academic community
Research papers in high-profile international journals. Presentation at one international conference, for example the Biophysical Society meeting.
4. General public
A regularly updated web-site that includes a description of the research at a level suitable for an educated layman, a list of publications, movies and illustrations.
5. BBSRC publicity team
Publications in top journals will be reported to the BBSRC publicity team.
Industrial Impact and Intellectual Property
The commercial potential of the methods developed will be assessed and managed with the assistance of ISIS, Oxford University's technology transfer subsidiary.
Since its invention in the 17th century, the microscope has revolutionized our understanding of the world. However, conventional microscopy is limited to 2D. Generally, cells and microorganisms inhabit a three-dimensional world. While some ground has been made in recent years towards developing 3D microscopic techniques, on the whole, these employ a slice-wise depth-scanning approach, which is too slow to observe the dynamic nature of swimming microorganisms. Fast, high-resolution digital cameras and the increased computing power of graphical processing units offer us the ability to probe, for the first time, the dynamic 3D world of microorganisms using digital holographic microscopy (DHM).
Bacterial swimming is an essential part of the lifestyle of most free living species and is used to move to sites for pathogenic invasion, to form symbiotic relationships or to form bio-films. A digital holographic microscope will give us the ability to probe these behaviours in 3D.
By having a 3D DHM facility in the University of Oxford, we will be able to offer the ability to capture 3D position data to workers in research fields such as; the study of the motility of other microorganisms, experimental verification of theoretical studies on anomalous diffusion, and categorizing flow in novel microfluidic devices.
The impact will spread to the broader scientific community as others make use of and adapt DHM to their research. Commercial applications may also be found for the 3D tracking of microscopic objects. For instance, there is much interest in the screening of sperm cells for a number of different veterinary applications.
Holographic technology could also be spread to the wider public. The raw materials to build a simple holographic microscope are not expensive. DHMs could be introduced to high school and undergraduate science laboratories, allowing students to observe the microworld in 3D, as well as providing a context for learning about optics, computer processing and biological processes. As a test case for this approach, we have already had a 6th form high school summer project student develop a simple holographic imaging setup with a laser pointer in our lab and are offering the opportunity to build a more extensive holographic microscope to a 4th year undergraduate project student in 2012.
Dissemination and Engagement Routes for Maximising Impact
1. Local university research community
Presentations at weekly seminars of the Oxford Physics biological physics community, which includes academics and students from Physics, Chemistry, Biochemistry, and Physiology.
2. UK academic community
Presentation at UK conferences, for example Institute of Physics, British Biophysical Society, Royal Microscopy Society meetings.
3. International academic community
Research papers in high-profile international journals. Presentation at one international conference, for example the Biophysical Society meeting.
4. General public
A regularly updated web-site that includes a description of the research at a level suitable for an educated layman, a list of publications, movies and illustrations.
5. BBSRC publicity team
Publications in top journals will be reported to the BBSRC publicity team.
Industrial Impact and Intellectual Property
The commercial potential of the methods developed will be assessed and managed with the assistance of ISIS, Oxford University's technology transfer subsidiary.
Organisations
Publications
Flewellen James Lewis
(2012)
Digital holographic microscopy for three-dimensional studies of bacteria
Flewellen JL
(2019)
A multi-mode digital holographic microscope.
in The Review of scientific instruments
| Description | Background: Holograms are a way of using interference with a reference laser beam to record the amplitude and phase of the light coming from an object, in contrast to an ordinary photograph that records only the light intensity (which is equal to the amplitude squared). Recording the phase as well allows a 3-dimensional reconstruction of the light field, which in turn allows 3D imaging of the object. Traditionally holograms were recorded on high-resolution emulsion film and reconstructed optically using laser illumination. More recently holograms of microscopic images have been recorded on digital cameras and reconstructed numerically in a computer. All reconstructions of holograms generate 3 separate images. One is the desired reconstruction of the original light from the object, also known as the "real image", one is a copy of this displaced in space, known as the "virtual image", and the third is the intensity image that would be recorded in an ordinary photograph. There are two types of hologram, inline and off-axis. In "inline" holograms the illumination beam doubles as the reference beam, and all 3 images are seen in the reconstruction. If care is taken that the intensity image is much less bright than the other two (by ensuring that the object beam is much less bright than the reference beam), and in addition that the virtual image is very out-of-focus at the location of the real image, the real image can be picked out from the background of the other two. Inline holography has been the dominant form in digital holographic microscopy, as it is relatively easy to implement. Off axis holography is preferred in traditional holograms, as the three images are in different places and it is possible to choose viewing conditions that show only the real image. However it has received little attention in digital holographic microscopy. Key findings: We have shown that it is possible to track swimming bacteria and 100 nm gold nanopartilces as tracers of fluid flow, in 3 dimensions, using digital holographic microscopy. Our inline holographic microscope can track at least dozens of swimming bacteria simultaneously at high speeds (up to 2000 frames per second) throughout volumes of order 100x100x100 microns. We have developed a new type of off-axis digital holographic microscopy, based on laser dark-field illumination to generate the object beam while removing the illumination beam. The key findings here are the optical techniques to create the darkfield object beam and interfere it with a separate reference beam and the computer methods to separate the real image entirely from the background images by filtering in the spatial frequency domain, also known as "Fourier Space". This method is still being tested, but early results show that full reconstructions of the amplitude and phase of the real image can be separated from the other images, and that the off-axis method successfully records the 3D locations of 100 nm gold nanoparticles where inline holography fails. We anticipate that lifting the limitation of inline holography that the object beam is much less bright than the reference beam will allow faithful recording of much higher densities of swimming bacteria and gold nanoparticles. We also aim to explore the possibilities of the new method to make 3D images of other biologically interesting samples. |
| Exploitation Route | Future potential exploitation routes include commercial tracking of microscopic swimmers, for example sperm, and other medical applications of 3D holographic imaging. Digital holographic microscopy also has great potential as an educational tool, and has already formed the basis of 3 student projects arising from this grant. The first exploitation route will be to study the swimming patterns and responses of a wide range of bacterial species under various conditions, including biofouling, chemotaxis and biofilm formation. This will include collaborations with other research groups studying various bacterial systems, who are currently limited by the inability to record swimming trajectories in 3 dimensions. Future potential exploitation routes include commercial tracking of microscopic swimmers, for example sperm, and other medical applications of 3D holographic imaging. These will depend substantially on the capabilities of the new off-axis method, which is currently under exploration under a follow-up grant. |
| Sectors | Agriculture, Food and Drink,Education,Environment,Healthcare |
| Description | We have developed two types of digital holographic microscope, inline and off-axis. We are planning to develop these commercially. |
| First Year Of Impact | 2013 |
| Sector | Education |
| Impact Types | Cultural |
| Description | JFF-Digital Holographic Microscopy |
| Amount | £46,915 (GBP) |
| Organisation | University of Oxford |
| Department | John Fell Fund |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 11/2013 |
| End | 11/2014 |
| Description | Tool Research and Development Fund Grant Award from the BBSRC: Digital Holographic Microscopy |
| Amount | £119,434 (GBP) |
| Organisation | Biotechnology and Biological Sciences Research Council (BBSRC) |
| Sector | Public |
| Country | United Kingdom |
| Start | 07/2012 |
| End | 11/2013 |
| Title | Dark field off-axis digital holographic microscopy through optical Fourier filtering |
| Description | Dark field microscopy is combined with off-axis holography to produce a novel technique for 3D imaging. dark field image is created by 1) a physical mask in the centre of a conjugate Fourier plane in the optical path,or 2) a hole in a mirror in a conjugate Fourier plane. Both methods have the result of removing the direct illumination component and allowing the dark field object signal to pass to form a hologram. This technique allows for the three-dimensional localisation of a dilute suspension of gold nanoparticles at high magnifications and of a suspension of motile bacteria at up to OD 0.04. Phase and amplitude reconstructions can be obtained of the light emanating from a given sample. |
| Type Of Technology | New/Improved Technique/Technology |
| Year Produced | 2012 |
| Impact | N/A |
| Title | Pym GUI |
| Description | Graphical User Interface (GUI) for processing and analysing hologram data. Provides a non-specialist-friendly interface to the Pym software package that pre-processes and reconstructs holographic data. |
| Type Of Technology | Software |
| Year Produced | 2011 |
| Impact | N/A |