Bayesian analysis of images to provide fluorescence ultramicroscopy

Lead Research Organisation: King's College London
Department Name: Randall Div of Cell and Molecular Biophy


Microscopy is a very powerful and important technique in modern cell biology. For example, electron microscopy allows cells to be imaged with a resolution of a few nanometres (nm) in dead cells and fluorescence microscopy allows the positions of specific proteins to be observed in live cells. In fluorescence microscopy the protein in the cell which interests us is dyed with a chemical which can glow (a fluorophore), giving us vital information about how cells function, divide and die.

Fluorescence microscopy is limited in resolution by the fundamental physics light. However, recently ways have been found to bypass this limit. For example, imagine that only molecule of dye is glowing. Even though an image of it is blurred into a blob, you can find the position of the centre of the blob very accurately. But this only works if the dot is alone, and so many thousands of images of blurred dots are needed to assemble a picture. In order to spread out the emission of fluorophores over thousands of images we use fluorophores which switch between a state where they are emitting light (on) and a state where they don't (off). This technique, localisation microscopy, can achieve resolutions of a few tens of nm.

Currently when a localisation microscopy image is reconstructed from a series of images all the information about when the fluorophores are emitting light is all discarded. We are proposing to create a new data analysis technique which will allow us to extract information about how often each fluorophore switches between the on and off states.

In order to extract information about how the fluorophore behaves over time, we will have to create a sophisticated model of the data. To do this we will use Bayesian statistics, in which we build all the information we have about a system into a series of models, and find which is most likely. For example, you could compare one model in which a fluorophore did not switch off and on at all and another in which it switched off and on rapidly. You can gradually change the model to work out which one is the most likely to be correct. The particular power of Bayesian statistics is that you can calculate this even if you do not know what the fluorophore looks like.

This information will allow cell biologists to carry out three new types of experiments:

1) For some fluorophores, the rate at which it switches between on and off changes as the chemical environment round the fluorophore changes. This would allow us to form an image of how the chemical environment of the cell changes, with a resolution of tens of nm.

2) Certain fluorophores change their intensity and the times at which they emit light when they are very close to a second type of fluorophore. This effect is already used to monitor when two different proteins are closer than 10 nm by looking for intensity changes. But if one of the types of fluorophore spontaneously switches between on and off, we will be able to form a localisation image of the cell, with a resolution of tens of nm, and also get information at each point about how close the second type of protein is to the first from the intensity and the times at which the fluorophores switch between on and off.

3) Certain fluorophores also change the times at which they emit light when two fluorophores of the same type are close together. This will give us information about where fluorophores are from two different sources: the localisation information and the information about how far apart molecules are. By modelling the fluorophores using both types of information we will create a new ultraresolution fluorescence microscopy technique with a resolution of 5 nm, similar to the resolution achieved by electron microscopy. But unlike electron microscopy, it will be possible to do experiments in live cells, allowing us to look at life in sharper focus than ever before.

Technical Summary

Advanced image analysis is taking an increasingly important role in fluorescence microscopy. One technique where image analysis is very important is localisation microscopy, which is a superresolution fluorescence technique that can achieve a resolution of tens of nm (as opposed to the diffraction limited value of hundreds of nm). In localisation microscopy the emission of the fluorophores is spread out over many thousands of images, and in each image there are a few well separated fluorophores whose positions can be fitted. This is possible because the fluorophores 'blink' between states where they emit light and remain dark. Currently the superresolution image is reconstructed from the fitted positions, and all the information about when in time the fluorophores blink is discarded.
We propose to create a new Bayesian image analysis method which will identify reappearances of individual fluorophores and use this information to characterise the behaviour of each fluorophore over time. This extra information about the sample can be built into the image (functional imaging). For example, some fluorophores change their blinking rate depending on the chemical environment of the cell, which would allow us to form a map of cell environment with a resolution of tens of nm.
Fluorescence resonance energy transfer is another technique which is used to make measurements of the distance between two fluorophores. Two different fluorophores are monitored and when they approach closer than 10nm, the intensity and the temporal behaviour of the fluorophores changes. If one of the pair of fluorophores blinks, it will be possible to combine localisation microscopy with a measure of interaction with another protein. If the two fluorophores are the same rather than different we will be able to combine localisation information with relative position information using Bayesian inference to give us a new microscopy technique with resolution down to 5nm.

Planned Impact

Having already developed Bayesian analysis of blinking and bleaching microscopy (3B), a leading technique in live cell superresolution imaging, the Cox lab is well placed to not only develop a new technique but to present it in a form which will make it available to the research community. Developing an ImageJ plugin for 3B has given us the necessary experience to translate new algorithms to this popular platform. In addition, our close links with the cell biology community allow us to target modifications to the technique to solve particular biological problems as they arise. This, together with our experience interacting with cell biologists who use our method, maximises the benefit to the imaging and cell biology community from our work.

We have identified several potential target systems in use by other researchers in the Randall division which would enable us to test and demonstrate the performance of different applications of temporal data mining. Firstly, the Ameer-Beg group works with fluorescence resonance energy transfer (FRET) sensors which can be switched to a non-emitting state (and where one of the pair is dark), enabling localisation measurements. Such data could be used to test the performance of simultaneous localisation microscopy, and FRET measurements which use both intensity and blinking rate information. Secondly previous superresolution measurements in podosomes, which are investigated by the Jones lab, have started to reveal the relative positions of different actin-associated proteins. However, to be verified and characterised in live cells these observations need a substantial boost in resolution, making this potentially a good test system for the performance to extract homo-FRET information. Finally the Owen lab is interested in the utility of blinking as a functional readout to gain information about lipid domains in T-cells, using a dye where the blinking rate of the fluorophore itself is known vary with the local molecular environment. The different biological questions we will investigate are all linked by the driving need to push data analysis for fluorescence microscopy beyond its current limitations by making use of temporal as well as spatial information.

The nature and location of the research are ideal for maximising the impact by allowing us to rapidly make the new techniques available to research groups at both King's and elsewhere in the UK community. The foundation of the Nikon Imaging Centre (NIC) - one of the largest global facilities of its kind and the only one in the UK - exemplifies the long term commitment to high resolution imaging at King's. The College has also benefited from a recent award of £360,000 from the Royal Society/Wolfson Trust to refurbish labs for the housing of our development systems. The work in this proposal would allow us to rapidly become a driving force in this increasingly important strategic area in the UK.

Cell biology and biophysics researchers will be informed about the algorithms and software packages developed in this project by conference presentations, publications, web sites and the provision of an online forum to encourage peer support when using techniques. Provision will be made for other researchers to make training visits to the laboratory.

Staff training: The PDRA will be trained in a broad range of skills required for developing software tools. Being embedded in the Randall will allow him to interface with cell biologists and biophysicists to better understand the needs of the end users while developing the software.


10 25 50
publication icon
Cox S (2015) Super-resolution imaging in live cells. in Developmental biology

publication icon
Fox-Roberts P (2014) Fixed pattern noise in localization microscopy. in Chemphyschem : a European journal of chemical physics and physical chemistry

publication icon
Man SM (2014) Inflammasome activation causes dual recruitment of NLRC4 and NLRP3 to the same macromolecular complex. in Proceedings of the National Academy of Sciences of the United States of America

Description Super-resolution fluorescence microscopy allows cells to be imaged down to a lengthscale of tens of nm. One such technique, localisation microsocpy, requires considerable image analysis. We have developed a new image analysis technique that allows data to be taken much more rapidly, and a method of data analysis which will warn users if their data is too dense to be analysed accurately. We have also investigated how the behaviour of fluorophores changes when they are close to other fluorophores.
Exploitation Route Improved super-resolution microscopy techniques will allow biological and biomedical research laboratories to imae cell structures at enhanced resolution, allowing improved characterisation of cell behaviour and of cell response to stimuli such as drugs.

Our method to assess the accuracy of data will be useful to cell biology and biomedical researchers who wish to use localisation microscopy and want to be sure that their results are reliable.
Sectors Healthcare,Pharmaceuticals and Medical Biotechnology

Description Taught on EMBL Super-Resolution Microscopy Course
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact This course teaches students from across Europe about the principles of super-resolution optical microscopy, and how to apply these techniques in experiments. EMBL runs many prestigious courses for European researchers.
Description Taught on EMBO Advanced Optical Microscopy Course
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact Each year the EMBO Advanced Optical Microscopy course gives around 20 students an intense, ten day course covering the relevant parts of physics, molecular biology, cell biology and chemistry. This broad and intense course is particularly good at kick-starting research careers in this highly interdisciplinary area. I went on this course (when I was a post-doc) in 2009 and it was an important factor in allowing me to rapidly establish myself in this new research area.
Description Taught on ESRIC summer school
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact The ESRIC summer school is an intense five day course which introduces the students to different super-resolution methods, and then gives them hands-on practical sessions. In addition to the lectures there are question-and-answer sessions with researchers, which allow the students to ask experiment-specific questions. There is a strong emphasis on how to avoid images with artefacts, either through sample fixation or incorrect data analysis.
Description Architecture/force relationship and migration mechanics of macrophage podosomes
Amount $1,350,000 (USD)
Organisation Human Frontier Science Program (HFSP) 
Sector Charity/Non Profit
Country France
Start 11/2016 
End 10/2019
Description King's College BBSRC Sparking Impact award
Amount £4,359 (GBP)
Organisation Biotechnology and Biological Sciences Research Council (BBSRC) 
Sector Public
Country United Kingdom
Start 01/2014 
End 06/2014
Description King's College London BBSRC Sparking Impact Award
Amount £6,600 (GBP)
Organisation Biotechnology and Biological Sciences Research Council (BBSRC) 
Sector Public
Country United Kingdom
Start 02/2016 
End 04/2016
Description Leverhulme Trust Research Project
Amount £390,000 (GBP)
Organisation The Leverhulme Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 05/2015 
End 05/2018
Title Assessment of fluorophore dynamics using DNA origami 
Description We have developed a method to use localisation microscopy data of DNA origami to assess how the behaviour of pairs of fluorophores is related to their spatial separation. It uses simultaneous imaging in two colour channels with DNA origami that has been labelled with two different dyes. 
Type Of Material Technology assay or reagent 
Provided To Others? No  
Impact The work has revealed that the behaviour of fluorophores depends on their spatial separation. This is important for molecule counting and nanoscale imaging applications. 
Title Dual density localisation microscopy imaging 
Description We have developed a method to simultaneously image a sample tagged with a photoswitchable fluorophore such as mEOS-2 in two different colour channels. This means that a high-density widefield image and a low-density single-molecule image can be acquired at the same time. This allows either tracking of the average motion of the protein from the high density image, and reconstruction of a super-resolution image from the low density image, or tracking of single molecule motion, depending on what type of data is required. 
Type Of Material Technology assay or reagent 
Year Produced 2015 
Provided To Others? Yes  
Impact We are using this method to track the dynamics of vinculin in focal adhesions in collaboration with the group of Maddy Parsons at KCL. The work is currently unpublished. 
Title Data quality assessment for localisation microscopy 
Description Our research revealed that analysis of localisation microscopy data can undergo 'silent failure', where the reconstruction does not reflect the underlying sample, but there is no warning given by the data analysis and no a priori way to analyse the raw data to detect that failure will occur. We have used a machine learning approach, coupling principle component analysis with random forests to allow user-driven classification of the data. This creates a map of where potentially incorrect results are, warning the user of where their results may not be reliable and allowing them to tune either the experiment or what data they choose to include in their analysis. 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact Paper published in Nature Communications. 
Title Improved Bayesian analysis of Blinking and Bleaching 
Description Bayesian analysis of blinking and bleaching is a data analysis method which allows super-resolution images to be reconstructed from a few hundred frames of raw data, rather than the thousands usually required for localisation microscopy. It is particularly suited to analysing data from live cell experiments where speed is critical. 
Type Of Technology Software 
Year Produced 2013 
Open Source License? Yes  
Impact The original Bayesian analysis of blinking and bleaching paper has been cited over 170 times since it was published in 2012 and has been taken up by multiple groups around the world. 
Title Improved background fitting for localisation microscopy 
Description We have modelled the background of localisation microscopy data as a Gaussian process, allowing improved background fitting. 
Type Of Technology Software 
Year Produced 2014 
Impact The paper describing this technique is currently under review so the software has not been released yet.