Optimising acquisition speed in localisation microscopy

Lead Research Organisation: King's College London
Department Name: Randall Div of Cell and Molecular Biophy

Abstract

Fluorescence microscopy is a crucial tool for cell biologists because it allows them to label different proteins with fluorescent molecules (fluorophores) and observe them in live cells. This yields information which can help us to understand diseases, and find new drugs to treat them. Until recently fluorescence microscopy had a major flaw: it could not resolve any features below 200nm. While human cells are at least twenty times this size, there are many parts of a cell which are much smaller. Over the last ten years a number of methods have been developed which allow fluorescence microscopes to image structures below 200nm, and these methods are now becoming standard in fixed (dead) cells. A major challenge in microscopy development is how to apply these methods in live cells, in a way that is reproducible enough that it can be used in cell biology laboratories where there are no experts in the technique. In this proposal we attack this problem with two approaches.

Firstly, we will investigate the theoretical limits of localisation microscopy. Localisation microscopy works by taking many images of the sample. The behaviour of the fluorophores is controlled so that in each image only a few of the fluorophores are emitting light. Even though each fluorophore results in a blurred spot, we can find the position of the centre of the spot very accurately. The image of the sample is then built up by putting a point down at the position of all the fluorophores we identify across all the frames. At the moment, people think about localisation microscopy as being similar to other microscopy techniques; you illuminate with light and you get an image, with the quality of the image depending on how good your microscope is and how bright your light is. However, for a localisation image to achieve a high resolution, you have to find the position of lots of fluorophores. Less obviously, the number of frames it takes to get a certain number of fluorophores depends on the structure of your sample, since you cannot image two fluorophores which are too close together. This means that the maximum speed depends on the structure of your sample. We will carry out simulations to work out how the maximum speed depends on the structure, which will allow cell biologists to know in advance what speed can be achieved for a given sample.

Secondly, we will develop a method which can examine the raw data from an experiment and determine whether, if you analyse it, you will get an image which reflects the structure of the sample, or if you will get an image with features caused by fitting the positions of fluorophores inaccurately. Currently, it is very hard to work out if this has happened, particularly if you try to get data quickly, which is necessary for live cell experiments. It may be possible to perform a quick test by looking at how the number of fluorophores which is detected changes over time. However, we are likely to need a more sophisticated test. We will use the images from an experiment and create a simulated image where we add a single fluorophore at a known position. We can then run the data analysis and see if the new fluorophore is correctly detected. By moving the fluorophore round, and performing the test on different frames, we will determine if there are particular times or places in the images where the data analysis is not working well.

By taking these two approaches, we will give every cell biologist with a localisation microscopy system the tools they need to calculate the maximum speed at which they can image the structure they are interested in. This will bring live cell localisation microscopy out of specialist labs and into the reach of cell biologists. Fixed cell localisation microscopy has already shown us many new and unexpected structures in the cell; by extending this technique into live cells, we will be able to see how these structures change and evolve over time.

Technical Summary

In this project we aim to develop software which will enable researchers to carry out localisation microscopy in live cells. Currently the field has many analysis packages, and a few for experimental optimisation. However, there is no help for optimising live cell experiments, where speed is of the essence. Furthermore, the issue of whether artefacts might be present is very challenging to assess and has previously only been approached by doing correlative experiments or acquiring low density data, neither of which is practical in live cells.

To create this software we will need to develop our mathematical understanding of information transmission in localisation microscopy. This will allow us to quantify what speed of imaging is possible with a given localisation microscopy technique and a given sample structure. By building this information into a software tool, we will allow other cell biologists to optimise their localisation microscopy imaging of live cells.

Obtaining a reconstructed super-resolution image fast requires increasing the density of fluorophores in the raw data. However, these high densities are precisely the conditions most likely to lead to artefacts in the final reconstructed image. Therefore, we will also develop tests to identify when artefacts are likely to be present. We will check whether the fluorophore localisations follow Poisson statistics, since if they do not it implies there are incorrect numbers of fluorophores being fitted. We will also use simulations to add extra fluorophores to the raw data. If the position of the extra fluorophore is found by the fitting algorithm it will indicate that the data density is not too high in that region.

So, combining these two ideas, we will create an ImageJ plugin which can predict the maximum appropriate speed for a localisation microscopy experiment and can also take raw data and identify if there will be artefacts in the reconstruction.

Planned Impact

The initial impact is primarily expected to be seen in an increase in the ability to perform live cell localisation microscopy experiments among cell biologists. There are a number of biological systems that could benefit from live cell localisation microscopy imaging, including but not limited to: fascin (upregulated in all known human cancers, cannot be imaged in fixed cells), the immune synapse in T cells and B cells, and structures associated with adhesion and migration such as focal adhesions and podosomes. These are all systems of high biomedical importance and in the longer term the greatest impact of this project is likely to be enabling and accelerating new biomedical research. A number of our collaborators (e.g. Dr Maddy Parsons) have strong links with pharmaceutical companies and would be well placed to help us take these results into an industrial context.

We have links to a number of microscopy companies, and on this project we have partnered with Leica. Recently we acquired the body of a Leica GSD system, and have adapted it using our own lasers and control systems. Leica have expressed strong support for those developing open-source solutions for other researchers, and they have offered to host a visit for both myself and Patrick at their development headquarters in Mannheim so that we can share our results with them. Leica stand to benefit from our work because it would extend the experiments that could be carried out on their systems and give users greater confidence in their results. In turn, their users will benefit because we will be able to advise on changes to the hardware and software of the system which would optimise it for live cell localisation microscopy.

We will also carry out experiments at the Nikon Centre, and so will be able to directly advise those using Nikon microscopes, and those using the N-STORM system in general.

Dr Patrick Fox-Roberts, the post-doc on this project, has received training in both analysis of microscopy data and in carrying out experiments. Super-resolution is a rapidly growing area, with many jobs being created, and a shortage of people with in-depth training.

Publications

10 25 50
 
Description The award was to optimise the speed at which images could be acquired. In attempting to understand the limiting factors, we have discovered some characteristics of fluorophore behaviour which enable us to get much more accurate images of samples. Additionally, we have worked on an improved method for analysing the data once it has been acquired, so we can characterise it and understand if the system is performing optimally.

In order to improve our ability to acquire images at high speed without artefacts, we developed a different technique which pre-processes data before it is fitted, in a way that means the output will not have artefacts. This method, called HAWK, was published in Nature Methods and patented by KCL.
Exploitation Route Our methods have been released along with software which allows others doing microscopy to use the same methods.
Sectors Healthcare,Pharmaceuticals and Medical Biotechnology

 
Description Taught on DPG summer school
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact The DPG summer school was a unique one-off event designed to give students from the physical sciences an intensive introduction to biological microscopy. This is unusual since most courses are designed to introduce students from a biological background to physics-based concepts.
URL http://www.dpg-physik.de/dpg/pbh/aktuelles/S117.html?lang=en&
 
Description Taught on EMBL super-resolution microscopy course 2016
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact This course teaches students from across Europe about the principles of super-resolution optical microscopy, and how to apply these techniques in experiments. EMBL runs many prestigious courses for European researchers.
URL http://www.embl.de/training/events/2016/MIC16-03/
 
Description Taught on EMBL super-resolution microscopy course 2017
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact This course teaches students from across Europe about the principles of super-resolution optical microscopy, and how to apply these techniques in experiments. EMBL runs many prestigious courses for European researchers.
URL https://www.embl.de/training/events/2017/MIC17-03/
 
Description Taught on EMBO Advanced Optical Microscopy Course 2016
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact Each year the EMBO Advanced Optical Microscopy course gives around 20 students an intense, ten day course covering the relevant parts of physics, molecular biology, cell biology and chemistry. This broad and intense course is particularly good at kick-starting research careers in this highly interdisciplinary area. I went on this course (when I was a post-doc) in 2009 and it was an important factor in allowing me to rapidly establish myself in this new research area.
 
Description Taught on ESRIC summer school 2016
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact The ESRIC summer school is an intense five day course which introduces the students to different super-resolution methods, and then gives them hands-on practical sessions. In addition to the lectures there are question-and-answer sessions with researchers, which allow the students to ask experiment-specific questions. There is a strong emphasis on how to avoid images with artefacts, either through sample fixation or incorrect data analysis.
URL http://www.esric.org/summer-school.html
 
Description Taught on ESRIC summer school 2017
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact The ESRIC summer school is an intense five day course which introduces the students to different super-resolution methods, and then gives them hands-on practical sessions. In addition to the lectures there are question-and-answer sessions with researchers, which allow the students to ask experiment-specific questions. There is a strong emphasis on how to avoid images with artefacts, either through sample fixation or incorrect data analysis.
URL http://www.esric.org/summer-school.html
 
Description Architecture/force relationship and migration mechanics of macrophage podosomes
Amount $1,350,000 (USD)
Funding ID RGP0035/2016-MARIDONNEAU-PARINI 
Organisation Human Frontier Science Program (HFSP) 
Sector Charity/Non Profit
Country France
Start 11/2016 
End 10/2019
 
Title Accurate assessment of cluster size in localisation microscopy images 
Description We have developed a way of using the Renyi divergence to measure cluster size which is more accurate than methods currently used. Measuring cluster size is commonly performed on localisation microscopy images to assess the distribution of a receptor on a membrane during, for example, an immune response. However, we have found that standard measurements give values which have a significant bias. The Renyi divergence can be used to find a value which is highly accurate and precise. 
Type Of Material Technology assay or reagent 
Year Produced 2016 
Provided To Others? No  
Impact Method is currently under review, no impacts yet. 
 
Title Method to analyse localisation microscopy images without creating artifacts 
Description We developed a method, Haar Wavelet Kernel Analysis, which can be used to pre-process localisation microscopy data so that the image will not contain artificial sharpening artifacts. The method takes in a localisation microscopy dataset and outputs a longer dataset in which overlapping fluorophores have been separated into different frames, which can then be analysed with any fitting method. The software has been released with the paper as an imagej plugin and is fast and simple to use. 
Type Of Material Technology assay or reagent 
Year Produced 2018 
Provided To Others? Yes  
Impact The tool was only recently released so we have not yet been able to assess its impact on the field 
URL https://www.nature.com/articles/s41592-018-0072-5
 
Title METHOD AND SYSTEM FOR LOCALISATION MICROSCOPY 
Description Embodiments of the present invention provide a method and system for processing microscopy images to enable localisation analysis of high density raw data, and thereby achieve higher spatial resolution than would otherwise be the case. This is achieved by exploiting temporal redundancies in the image data resulting from close-to emitters that would otherwise be resolved as a single emitter were they to emit or fluoresce at the same time, but which, by virtue of emitting or fluorescing at slightly different (yet potentially overlapping) times, can be subject to temporal filtering by different filters of different temporal bandwidth to resolve the two emitters. Effectively, the different temporal filters have different time constants which work together to effectively highlight the different emission or fluorescence times of the two emitters, to thereby allow the two close-to emitters to be separately resolved. 
IP Reference WO2019135069 
Protection Patent application published
Year Protection Granted 2019
Licensed No
Impact None currently
 
Title Haar Wavelet Kernel Analysis for localisation microscopy data 
Description HAWK is a preprocessing software that allows high density localisation data to be fitted more accurately and with fewer artifacts. 
Type Of Technology Software 
Year Produced 2018 
Open Source License? Yes  
Impact Multiple groups have used our preprocessing method to obtain live cell localisation microscopy data without artifacts, leading to publications such as Frei et al Nature Comms 2019 and Kwon et al Nature Comms 2020. The method is available open source, but is covered by a patent (so development of any closed source version requires commercialisation). 
URL https://www.nature.com/articles/s41592-018-0072-5