Enabling Reliable Testing Of SMLM Datasets

Fluorescence microscopy is a key technology for biomedical research due to its ability to image the distribution of specific proteins in cells. Over the last twenty years the resolution achievable with fluorescence microscopy has been improved from around 200nm down to 20nm. The most popular of these methods, single molecule light microscopy (SMLM), collects a series of images and then processes it to create a reconstructed image of the sample with improved resolution. While this method is experimentally simple and can achieve very good resolution, it has a major drawback: the image processing it relies on can lead to artifacts (false structures) in the reconstructed image that are hard to spot, leading biologists to draw incorrect conclusions about their data. It is extremely difficult to test for these artifacts, and methods to do so have only recently been developed. Here the two leading developers of data assessment for SMLM want to combine forces to create a new tool which enables testing using both of their approaches.
We will create an integrated software package that tests for resolution (using a metric improved in accuracy compared to the standard one used) and two types of artifact. The first, artificial sharpening, is common issue encountered in multicolour and live cell SMLM, and any reconstruction method that doesn't find positions of fluorophores, but instead modifies the input images. The second examines how well the SMLM reconstructed image agrees with a widefield image of the sample, and is good at spotting missing structure, variation in the proportion of fluorophores being detected across the image, and issues caused by the background. By testing for all of these different types of artifact, users will be able to have confidence that their data analysis is not introducing errors into their images.
Code will be developed on a Github repository and source code will be made available. This is in keeping with the long track record of the applicants in releasing open source software. The software will be released in two different packages: as an ImageJ/Fiji plugin (the most common system used by microscopists and biological/biomedical researchers) and also in napari (a new python-based image processing package, which is increasingly used by methods developers). We will seek to establish our method as a standard test before publication, and encourage the field to develop further extensions as our understanding of SMLM improves, with a process of code review ultimately determining which modifications are accepted.
We will also assess a range of publicly available data, and acquire new exemplar datasets to allow us to illustrate to potential users likely artifacts and under what circumstances they might be encountered. While there is an increasing amount of SMLM data available, this usually focuses on high quality raw data which others might want to process. We also want to generate datasets in which raw data likely to produce errors is deliberately generated.
The lack of consistent standards and tests contributes to the reproducibility crisis in science: it is impossible to benchmark or validate the data analysis in SMLM studies because there are no agreed standards to which such studies should adhere. Our approach, uniting the best of the current tests available, has the support and confidence of the SMLM developer community and support from facility managers and researchers who rely on SMLM. Setting such a standard for testing will allow journals to finally put in place a consistent set of requirements for SMLM data verification, which is both actually possible for non-specialist researchers to fulfill, and provides rigorous quality checks.

Super-resolution microscopy is one of the biggest advances in fluorescence microscopy techniques from the last twenty years. Single molecule localization microscopy (SMLM) offers the highest resolving power in this family of methods, but is the most challenging to gather reliable results from. This is because the analysis stage of SMLM, where the centres of individual fluorescent molecules are computationally localized with nanometer precision, is prone to inaccuracy when the raw data deviates from ideal, low-density conditions. When this failure occurs it can lead to artifacts such as artificial sharpening, where closely separated structures collapse onto a single artificial structure, and a nonlinear relationship between the super-resolution image intensities and the true number of fluorescent molecules. Such artifacts can look like realistic structures, and as a result are difficult to detect in reconstructed SMLM images, especially for non-specialists. While methods to detect artifacts in SMLM data have been developed, these are currently spread between different software packages, with each method sensitive to a different type of artifact. We propose to integrate the current best tests of SMLM image quality to create a single resource that allows users to detect a range of different artifacts, and an accurate resolution measure, using a single package. No such package currently exists, with tests being split between a number of different plugins, and in the case of resolution measure often implementing a simplified version of the algorithm prone to reporting a better resolution than has actually been achieved. Our package will give people a clear overview of the resolution they have achieved and any artifacts that may be present in their image, allowing users to routinely test the quality of their data before publication.