Bayesian analysis of images to provide fluorescence ultramicroscopy
Lead Research Organisation:
King's College London
Department Name: Randall Div of Cell and Molecular Biophy
Abstract
Microscopy is a very powerful and important technique in modern cell biology. For example, electron microscopy allows cells to be imaged with a resolution of a few nanometres (nm) in dead cells and fluorescence microscopy allows the positions of specific proteins to be observed in live cells. In fluorescence microscopy the protein in the cell which interests us is dyed with a chemical which can glow (a fluorophore), giving us vital information about how cells function, divide and die.
Fluorescence microscopy is limited in resolution by the fundamental physics light. However, recently ways have been found to bypass this limit. For example, imagine that only molecule of dye is glowing. Even though an image of it is blurred into a blob, you can find the position of the centre of the blob very accurately. But this only works if the dot is alone, and so many thousands of images of blurred dots are needed to assemble a picture. In order to spread out the emission of fluorophores over thousands of images we use fluorophores which switch between a state where they are emitting light (on) and a state where they don't (off). This technique, localisation microscopy, can achieve resolutions of a few tens of nm.
Currently when a localisation microscopy image is reconstructed from a series of images all the information about when the fluorophores are emitting light is all discarded. We are proposing to create a new data analysis technique which will allow us to extract information about how often each fluorophore switches between the on and off states.
In order to extract information about how the fluorophore behaves over time, we will have to create a sophisticated model of the data. To do this we will use Bayesian statistics, in which we build all the information we have about a system into a series of models, and find which is most likely. For example, you could compare one model in which a fluorophore did not switch off and on at all and another in which it switched off and on rapidly. You can gradually change the model to work out which one is the most likely to be correct. The particular power of Bayesian statistics is that you can calculate this even if you do not know what the fluorophore looks like.
This information will allow cell biologists to carry out three new types of experiments:
1) For some fluorophores, the rate at which it switches between on and off changes as the chemical environment round the fluorophore changes. This would allow us to form an image of how the chemical environment of the cell changes, with a resolution of tens of nm.
2) Certain fluorophores change their intensity and the times at which they emit light when they are very close to a second type of fluorophore. This effect is already used to monitor when two different proteins are closer than 10 nm by looking for intensity changes. But if one of the types of fluorophore spontaneously switches between on and off, we will be able to form a localisation image of the cell, with a resolution of tens of nm, and also get information at each point about how close the second type of protein is to the first from the intensity and the times at which the fluorophores switch between on and off.
3) Certain fluorophores also change the times at which they emit light when two fluorophores of the same type are close together. This will give us information about where fluorophores are from two different sources: the localisation information and the information about how far apart molecules are. By modelling the fluorophores using both types of information we will create a new ultraresolution fluorescence microscopy technique with a resolution of 5 nm, similar to the resolution achieved by electron microscopy. But unlike electron microscopy, it will be possible to do experiments in live cells, allowing us to look at life in sharper focus than ever before.
Fluorescence microscopy is limited in resolution by the fundamental physics light. However, recently ways have been found to bypass this limit. For example, imagine that only molecule of dye is glowing. Even though an image of it is blurred into a blob, you can find the position of the centre of the blob very accurately. But this only works if the dot is alone, and so many thousands of images of blurred dots are needed to assemble a picture. In order to spread out the emission of fluorophores over thousands of images we use fluorophores which switch between a state where they are emitting light (on) and a state where they don't (off). This technique, localisation microscopy, can achieve resolutions of a few tens of nm.
Currently when a localisation microscopy image is reconstructed from a series of images all the information about when the fluorophores are emitting light is all discarded. We are proposing to create a new data analysis technique which will allow us to extract information about how often each fluorophore switches between the on and off states.
In order to extract information about how the fluorophore behaves over time, we will have to create a sophisticated model of the data. To do this we will use Bayesian statistics, in which we build all the information we have about a system into a series of models, and find which is most likely. For example, you could compare one model in which a fluorophore did not switch off and on at all and another in which it switched off and on rapidly. You can gradually change the model to work out which one is the most likely to be correct. The particular power of Bayesian statistics is that you can calculate this even if you do not know what the fluorophore looks like.
This information will allow cell biologists to carry out three new types of experiments:
1) For some fluorophores, the rate at which it switches between on and off changes as the chemical environment round the fluorophore changes. This would allow us to form an image of how the chemical environment of the cell changes, with a resolution of tens of nm.
2) Certain fluorophores change their intensity and the times at which they emit light when they are very close to a second type of fluorophore. This effect is already used to monitor when two different proteins are closer than 10 nm by looking for intensity changes. But if one of the types of fluorophore spontaneously switches between on and off, we will be able to form a localisation image of the cell, with a resolution of tens of nm, and also get information at each point about how close the second type of protein is to the first from the intensity and the times at which the fluorophores switch between on and off.
3) Certain fluorophores also change the times at which they emit light when two fluorophores of the same type are close together. This will give us information about where fluorophores are from two different sources: the localisation information and the information about how far apart molecules are. By modelling the fluorophores using both types of information we will create a new ultraresolution fluorescence microscopy technique with a resolution of 5 nm, similar to the resolution achieved by electron microscopy. But unlike electron microscopy, it will be possible to do experiments in live cells, allowing us to look at life in sharper focus than ever before.
Technical Summary
Advanced image analysis is taking an increasingly important role in fluorescence microscopy. One technique where image analysis is very important is localisation microscopy, which is a superresolution fluorescence technique that can achieve a resolution of tens of nm (as opposed to the diffraction limited value of hundreds of nm). In localisation microscopy the emission of the fluorophores is spread out over many thousands of images, and in each image there are a few well separated fluorophores whose positions can be fitted. This is possible because the fluorophores 'blink' between states where they emit light and remain dark. Currently the superresolution image is reconstructed from the fitted positions, and all the information about when in time the fluorophores blink is discarded.
We propose to create a new Bayesian image analysis method which will identify reappearances of individual fluorophores and use this information to characterise the behaviour of each fluorophore over time. This extra information about the sample can be built into the image (functional imaging). For example, some fluorophores change their blinking rate depending on the chemical environment of the cell, which would allow us to form a map of cell environment with a resolution of tens of nm.
Fluorescence resonance energy transfer is another technique which is used to make measurements of the distance between two fluorophores. Two different fluorophores are monitored and when they approach closer than 10nm, the intensity and the temporal behaviour of the fluorophores changes. If one of the pair of fluorophores blinks, it will be possible to combine localisation microscopy with a measure of interaction with another protein. If the two fluorophores are the same rather than different we will be able to combine localisation information with relative position information using Bayesian inference to give us a new microscopy technique with resolution down to 5nm.
We propose to create a new Bayesian image analysis method which will identify reappearances of individual fluorophores and use this information to characterise the behaviour of each fluorophore over time. This extra information about the sample can be built into the image (functional imaging). For example, some fluorophores change their blinking rate depending on the chemical environment of the cell, which would allow us to form a map of cell environment with a resolution of tens of nm.
Fluorescence resonance energy transfer is another technique which is used to make measurements of the distance between two fluorophores. Two different fluorophores are monitored and when they approach closer than 10nm, the intensity and the temporal behaviour of the fluorophores changes. If one of the pair of fluorophores blinks, it will be possible to combine localisation microscopy with a measure of interaction with another protein. If the two fluorophores are the same rather than different we will be able to combine localisation information with relative position information using Bayesian inference to give us a new microscopy technique with resolution down to 5nm.
Planned Impact
Having already developed Bayesian analysis of blinking and bleaching microscopy (3B), a leading technique in live cell superresolution imaging, the Cox lab is well placed to not only develop a new technique but to present it in a form which will make it available to the research community. Developing an ImageJ plugin for 3B has given us the necessary experience to translate new algorithms to this popular platform. In addition, our close links with the cell biology community allow us to target modifications to the technique to solve particular biological problems as they arise. This, together with our experience interacting with cell biologists who use our method, maximises the benefit to the imaging and cell biology community from our work.
We have identified several potential target systems in use by other researchers in the Randall division which would enable us to test and demonstrate the performance of different applications of temporal data mining. Firstly, the Ameer-Beg group works with fluorescence resonance energy transfer (FRET) sensors which can be switched to a non-emitting state (and where one of the pair is dark), enabling localisation measurements. Such data could be used to test the performance of simultaneous localisation microscopy, and FRET measurements which use both intensity and blinking rate information. Secondly previous superresolution measurements in podosomes, which are investigated by the Jones lab, have started to reveal the relative positions of different actin-associated proteins. However, to be verified and characterised in live cells these observations need a substantial boost in resolution, making this potentially a good test system for the performance to extract homo-FRET information. Finally the Owen lab is interested in the utility of blinking as a functional readout to gain information about lipid domains in T-cells, using a dye where the blinking rate of the fluorophore itself is known vary with the local molecular environment. The different biological questions we will investigate are all linked by the driving need to push data analysis for fluorescence microscopy beyond its current limitations by making use of temporal as well as spatial information.
The nature and location of the research are ideal for maximising the impact by allowing us to rapidly make the new techniques available to research groups at both King's and elsewhere in the UK community. The foundation of the Nikon Imaging Centre (NIC) - one of the largest global facilities of its kind and the only one in the UK - exemplifies the long term commitment to high resolution imaging at King's. The College has also benefited from a recent award of £360,000 from the Royal Society/Wolfson Trust to refurbish labs for the housing of our development systems. The work in this proposal would allow us to rapidly become a driving force in this increasingly important strategic area in the UK.
Cell biology and biophysics researchers will be informed about the algorithms and software packages developed in this project by conference presentations, publications, web sites and the provision of an online forum to encourage peer support when using techniques. Provision will be made for other researchers to make training visits to the laboratory.
Staff training: The PDRA will be trained in a broad range of skills required for developing software tools. Being embedded in the Randall will allow him to interface with cell biologists and biophysicists to better understand the needs of the end users while developing the software.
We have identified several potential target systems in use by other researchers in the Randall division which would enable us to test and demonstrate the performance of different applications of temporal data mining. Firstly, the Ameer-Beg group works with fluorescence resonance energy transfer (FRET) sensors which can be switched to a non-emitting state (and where one of the pair is dark), enabling localisation measurements. Such data could be used to test the performance of simultaneous localisation microscopy, and FRET measurements which use both intensity and blinking rate information. Secondly previous superresolution measurements in podosomes, which are investigated by the Jones lab, have started to reveal the relative positions of different actin-associated proteins. However, to be verified and characterised in live cells these observations need a substantial boost in resolution, making this potentially a good test system for the performance to extract homo-FRET information. Finally the Owen lab is interested in the utility of blinking as a functional readout to gain information about lipid domains in T-cells, using a dye where the blinking rate of the fluorophore itself is known vary with the local molecular environment. The different biological questions we will investigate are all linked by the driving need to push data analysis for fluorescence microscopy beyond its current limitations by making use of temporal as well as spatial information.
The nature and location of the research are ideal for maximising the impact by allowing us to rapidly make the new techniques available to research groups at both King's and elsewhere in the UK community. The foundation of the Nikon Imaging Centre (NIC) - one of the largest global facilities of its kind and the only one in the UK - exemplifies the long term commitment to high resolution imaging at King's. The College has also benefited from a recent award of £360,000 from the Royal Society/Wolfson Trust to refurbish labs for the housing of our development systems. The work in this proposal would allow us to rapidly become a driving force in this increasingly important strategic area in the UK.
Cell biology and biophysics researchers will be informed about the algorithms and software packages developed in this project by conference presentations, publications, web sites and the provision of an online forum to encourage peer support when using techniques. Provision will be made for other researchers to make training visits to the laboratory.
Staff training: The PDRA will be trained in a broad range of skills required for developing software tools. Being embedded in the Randall will allow him to interface with cell biologists and biophysicists to better understand the needs of the end users while developing the software.
Organisations
People |
ORCID iD |
Susan Cox (Principal Investigator) |
Publications
Cox S
(2015)
Super-resolution imaging in live cells.
in Developmental biology
Fox-Roberts P
(2014)
Fixed pattern noise in localization microscopy.
in Chemphyschem : a European journal of chemical physics and physical chemistry
Fox-Roberts P
(2017)
Local dimensionality determines imaging speed in localization microscopy.
in Nature communications
Man SM
(2014)
Inflammasome activation causes dual recruitment of NLRC4 and NLRP3 to the same macromolecular complex.
in Proceedings of the National Academy of Sciences of the United States of America
Marsh RJ
(2018)
Artifact-free high-density localization microscopy analysis.
in Nature methods
Peddie CJ
(2017)
Correlative super-resolution fluorescence and electron microscopy using conventional fluorescent proteins in vacuo.
in Journal of structural biology
Reymond N
(2015)
RhoC and ROCKs regulate cancer cell interactions with endothelial cells.
in Molecular oncology
Staszowska AD
(2018)
The Rényi divergence enables accurate and precise cluster analysis for localization microscopy.
in Bioinformatics (Oxford, England)
Staszowska AD
(2017)
Investigation of podosome ring protein arrangement using localization microscopy images.
in Methods (San Diego, Calif.)
Description | Super-resolution fluorescence microscopy allows cells to be imaged down to a lengthscale of tens of nm. One such technique, localisation microsocpy, requires considerable image analysis. We have developed a new image analysis technique that allows data to be taken much more rapidly, and a method of data analysis which will warn users if their data is too dense to be analysed accurately. We have also investigated how the behaviour of fluorophores changes when they are close to other fluorophores. |
Exploitation Route | Improved super-resolution microscopy techniques will allow biological and biomedical research laboratories to imae cell structures at enhanced resolution, allowing improved characterisation of cell behaviour and of cell response to stimuli such as drugs. Our method to assess the accuracy of data will be useful to cell biology and biomedical researchers who wish to use localisation microscopy and want to be sure that their results are reliable. |
Sectors | Healthcare Pharmaceuticals and Medical Biotechnology |
URL | http://www.coxphysics.com/lr/ |
Description | Taught on EMBL Super-Resolution Microscopy Course |
Geographic Reach | Europe |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | This course teaches students from across Europe about the principles of super-resolution optical microscopy, and how to apply these techniques in experiments. EMBL runs many prestigious courses for European researchers. |
URL | http://www.embl.de/training/events/2016/MIC16-03/ |
Description | Taught on EMBO Advanced Optical Microscopy Course |
Geographic Reach | Europe |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | Each year the EMBO Advanced Optical Microscopy course gives around 20 students an intense, ten day course covering the relevant parts of physics, molecular biology, cell biology and chemistry. This broad and intense course is particularly good at kick-starting research careers in this highly interdisciplinary area. I went on this course (when I was a post-doc) in 2009 and it was an important factor in allowing me to rapidly establish myself in this new research area. |
URL | http://www.mba.ac.uk/embo-course/ |
Description | Taught on ESRIC summer school |
Geographic Reach | National |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | The ESRIC summer school is an intense five day course which introduces the students to different super-resolution methods, and then gives them hands-on practical sessions. In addition to the lectures there are question-and-answer sessions with researchers, which allow the students to ask experiment-specific questions. There is a strong emphasis on how to avoid images with artefacts, either through sample fixation or incorrect data analysis. |
URL | http://www.esric.org/summer-school.html |
Description | Architecture/force relationship and migration mechanics of macrophage podosomes |
Amount | $1,350,000 (USD) |
Funding ID | RGP0035/2016-MARIDONNEAU-PARINI |
Organisation | Human Frontier Science Program (HFSP) |
Sector | Charity/Non Profit |
Country | France |
Start | 11/2016 |
End | 10/2019 |
Description | King's College BBSRC Sparking Impact award |
Amount | £4,359 (GBP) |
Organisation | Biotechnology and Biological Sciences Research Council (BBSRC) |
Sector | Public |
Country | United Kingdom |
Start | 01/2014 |
End | 06/2014 |
Description | King's College London BBSRC Sparking Impact Award |
Amount | £6,600 (GBP) |
Organisation | Biotechnology and Biological Sciences Research Council (BBSRC) |
Sector | Public |
Country | United Kingdom |
Start | 02/2016 |
End | 04/2016 |
Description | Leverhulme Trust Research Project |
Amount | £390,000 (GBP) |
Organisation | The Leverhulme Trust |
Sector | Charity/Non Profit |
Country | United Kingdom |
Start | 04/2015 |
End | 05/2018 |
Title | Assessment of fluorophore dynamics using DNA origami |
Description | We have developed a method to use localisation microscopy data of DNA origami to assess how the behaviour of pairs of fluorophores is related to their spatial separation. It uses simultaneous imaging in two colour channels with DNA origami that has been labelled with two different dyes. |
Type Of Material | Technology assay or reagent |
Provided To Others? | No |
Impact | The work has revealed that the behaviour of fluorophores depends on their spatial separation. This is important for molecule counting and nanoscale imaging applications. |
Title | Dual density localisation microscopy imaging |
Description | We have developed a method to simultaneously image a sample tagged with a photoswitchable fluorophore such as mEOS-2 in two different colour channels. This means that a high-density widefield image and a low-density single-molecule image can be acquired at the same time. This allows either tracking of the average motion of the protein from the high density image, and reconstruction of a super-resolution image from the low density image, or tracking of single molecule motion, depending on what type of data is required. |
Type Of Material | Technology assay or reagent |
Year Produced | 2015 |
Provided To Others? | Yes |
Impact | We are using this method to track the dynamics of vinculin in focal adhesions in collaboration with the group of Maddy Parsons at KCL. The work is currently unpublished. |
Title | Data quality assessment for localisation microscopy |
Description | Our research revealed that analysis of localisation microscopy data can undergo 'silent failure', where the reconstruction does not reflect the underlying sample, but there is no warning given by the data analysis and no a priori way to analyse the raw data to detect that failure will occur. We have used a machine learning approach, coupling principle component analysis with random forests to allow user-driven classification of the data. This creates a map of where potentially incorrect results are, warning the user of where their results may not be reliable and allowing them to tune either the experiment or what data they choose to include in their analysis. |
Type Of Technology | Software |
Year Produced | 2016 |
Open Source License? | Yes |
Impact | Paper published in Nature Communications. |
URL | http://www.coxphysics.com/lr/ |
Title | Improved Bayesian analysis of Blinking and Bleaching |
Description | Bayesian analysis of blinking and bleaching is a data analysis method which allows super-resolution images to be reconstructed from a few hundred frames of raw data, rather than the thousands usually required for localisation microscopy. It is particularly suited to analysing data from live cell experiments where speed is critical. |
Type Of Technology | Software |
Year Produced | 2013 |
Open Source License? | Yes |
Impact | The original Bayesian analysis of blinking and bleaching paper has been cited over 170 times since it was published in 2012 and has been taken up by multiple groups around the world. |
URL | http://www.coxphysics.com/3b/ |
Title | Improved background fitting for localisation microscopy |
Description | We have modelled the background of localisation microscopy data as a Gaussian process, allowing improved background fitting. |
Type Of Technology | Software |
Year Produced | 2014 |
Impact | The paper describing this technique is currently under review so the software has not been released yet. |