Quantitative and Real-Time Image Analysis for Advanced Light Microscopy.
Lead Research Organisation:
University of Oxford
Department Name: Weatherall Inst of Molecular Medicine
Abstract
In microscopy based research it is common to acquire and analyze microscopy images of cells sampled from populations which have undergone differential treatments. A non-exhaustive number of samples (i.e. images of cells) are often acquired due to experimental limitations that include: restricted acquisition periods, provision of expensive equipment or pressures to move onto the next experiment. Random sampling is a methodology that is used to sample representatively. There are some good reasons however for not sampling randomly. For example, cells maybe expressing a reporter protein and the copy number (quantity) of this molecule maybe too high or too low for subsequent analysis and so preference is shown toward those examples where the expression levels are within a certain range. Or, alternatively, the cells in an area maybe compromised by an artifact, be unhealthy, or too dense or sparse for a particular analysis. There are many factors that will influence the choices of an imaging researcher that are fully justifiable, but it is a problem as to how this information is documented and shared with other scientists. Removing the scientist from the pipeline of acquisition and analysis is counter-productive, a better solution is to provide tools that allow them to document and describe any experimental subjectivity as well as boosting reproducibility through automation. The objectives of this first part of the project (1-4) revolve around solving these issues of documenting, describing and automating image acquisition so that experimental design can be better communicated between users and laboratories.
Furthermore, we would like to develop tools and approaches, which allow better visualization and real-time feedback for advanced approaches so as to better inform imaging scientists as they perform their research and make experimental decisions. The second part of the project relates to this goal, objectives 5-6.
Furthermore, we would like to develop tools and approaches, which allow better visualization and real-time feedback for advanced approaches so as to better inform imaging scientists as they perform their research and make experimental decisions. The second part of the project relates to this goal, objectives 5-6.
Technical Summary
In terms of microscopy analysis, great inroads have been made in terms of impartial and systematic analysis but little has been done to ensure that cells under the microscope are selected impartially. For this project DW will develop algorithms that can statistically quantify and describe cellular appearances utilizing the latest machine learning, computer vision (CV) and signal processing techniques and technologies.
DW has in ongoing work investigated the use of CV algorithms in microscopy and has shown very promising results can be achieved by utilizing object detection convolutional neural networks for cellular detection. DW will extend on his use of neural networks for localizing cells (objectives 1-2) and will also develop methods to statistically describe cellular appearance using networks derived from auto-encoders (objective 3), a type of compression network. Neural networks are implemented in several ways, a popular method is to use Tensorflow and DW is an expert in this language. To effectively train neural networks, powerful GPUs are required. Fortunately the WIMM has various computational facilities and DW has access to two powerful GPU equipped servers.
For developing CV and real-time analysis approaches to work with camera and detector hardware, DW will work toward developing algorithms that can be embedded in miniaturized electronics (objective 4). The Nvidia Jetson TX2 Developer kit is a resource which allows you to create and develop neural networks and distribute them onto small hardware boards. DW will develop and test this hardware technique with microscopy based hardware and algorithms.
For objectives 5 and 6 GPU code will be systematically produced in CUDA and will be made as compatible and distributable as possible. Oxford University has several microscopy software development projects (e.g. in Micron) and it is intended that the software and libraries produced by this project will be made compatible with these other projects.
DW has in ongoing work investigated the use of CV algorithms in microscopy and has shown very promising results can be achieved by utilizing object detection convolutional neural networks for cellular detection. DW will extend on his use of neural networks for localizing cells (objectives 1-2) and will also develop methods to statistically describe cellular appearance using networks derived from auto-encoders (objective 3), a type of compression network. Neural networks are implemented in several ways, a popular method is to use Tensorflow and DW is an expert in this language. To effectively train neural networks, powerful GPUs are required. Fortunately the WIMM has various computational facilities and DW has access to two powerful GPU equipped servers.
For developing CV and real-time analysis approaches to work with camera and detector hardware, DW will work toward developing algorithms that can be embedded in miniaturized electronics (objective 4). The Nvidia Jetson TX2 Developer kit is a resource which allows you to create and develop neural networks and distribute them onto small hardware boards. DW will develop and test this hardware technique with microscopy based hardware and algorithms.
For objectives 5 and 6 GPU code will be systematically produced in CUDA and will be made as compatible and distributable as possible. Oxford University has several microscopy software development projects (e.g. in Micron) and it is intended that the software and libraries produced by this project will be made compatible with these other projects.
Publications


Carravilla P
(2019)
Molecular recognition of the native HIV-1 MPER revealed by STED microscopy of single virions.
in Nature communications

Chagraoui H
(2018)
SCL/TAL1 cooperates with Polycomb RYBP-PRC1 to suppress alternative lineages in blood-fated cells.
in Nature communications

Chen YL
(2019)
Proof-of-concept clinical trial of etokimab shows a key role for IL-33 in atopic dermatitis pathogenesis.
in Science translational medicine

Hailstone M
(2020)
CytoCensus, mapping cell identity and division in tissues and organs using machine learning.
in eLife

Sezgin E
(2019)
Measuring nanoscale diffusion dynamics in cellular membranes with super-resolution STED-FCS.
in Nature protocols



Waithe D
(2020)
Object detection networks and augmented reality for cellular detection in fluorescence microscopy.
in The Journal of cell biology
Description | Chair of the Image Analysis Focused Interest Group of the Royal Microscopy Society |
Geographic Reach | National |
Policy Influence Type | Membership of a guideline committee |
Impact | The bioimage analysis community is not well represented within the research community in the UK and beyond. The goal of this focused interest group, which I was asked to chair by the Royal Microscopy society, is designed to highlight and draw together the community of bioimage analysts in the UK. So far we have developed a number of questionnaires, staged events and contributed talks at a number of scientific conferences to raise awareness for this group. |
URL | http://iafig-rms.org/ |
Description | Organised and taught on a Python Bioimage Analysis course |
Geographic Reach | Europe |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | We trained 46 bioimage scientists to become fully trained Bioimage Analysts. They developed Python image analysis skills and will apply these research skills across the UK and continental Europe. |
URL | https://github.com/IAFIG-RMS/Python-for-Bioimage-Analysis |
Description | Training of Bioimage Analysis Instructors. |
Geographic Reach | National |
Policy Influence Type | Influenced training of practitioners or researchers |
Impact | We trained a number of bioimage analyst (16) to be trainers. This has lead to subsequent enhanced and additional training. |
URL | https://analyticalscience.wiley.com/do/10.1002/micro.2914/full/ |
Description | UK management committee member for NEUBIAS |
Geographic Reach | Europe |
Policy Influence Type | Membership of a guideline committee |
Impact | Neubias is an COST funded initiative designed to raise awareness for and to support image analysis on the EU and worldwide level. I along with Graeme Ball represent the UK in this scheme. I attend meetings representing the UK and contribute to workgroups which influence how this group contributes to things like ontologies, training and databases for software, as well as best practise for the community. e.g. https://twitter.com/matuskalas/status/1093479463155912711 |
URL | http://eubias.org/NEUBIAS/venue/who-are-we/uk-members/ |
Title | Automated Microscope Control Algorithm software repository. |
Description | This is a software framework which allows a microscope to be autonomously controlled using the feedback from a machine learning/computer vision algorithm. https://github.com/dwaithe/amca It relates to the paper published as a pre-print here: https://www.biorxiv.org/content/10.1101/544833v1 |
Type Of Material | Technology assay or reagent |
Year Produced | 2019 |
Provided To Others? | Yes |
Impact | There is a lot of interest from the research community. The paper pre-print has already an altmetric of 42 and has been viewed around the world: https://biorxiv.altmetric.com/details/55202241 |
URL | https://github.com/dwaithe/amca |
Title | Fluorescence Microscopy Data for Cellular Detection using Object Detection Networks. |
Description | As a by-product of my research we have developed a public collection of bioimage datasets with annotations. This can be freely downloaded and used by the community as a whole. |
Type Of Material | Database/Collection of data |
Year Produced | 2019 |
Provided To Others? | No |
Impact | No notable impact yet, but publication citations should follow. |
URL | https://zenodo.org/record/2548493#.XIZMUxOwnJw |
Description | Application of image analysis algorithms for Molecular recognition of the native HIV-1 MPER. |
Organisation | University of the Basque Country |
Country | Spain |
Sector | Academic/University |
PI Contribution | Algorithms developed by myself for the analysis of STED images were used in their publication. |
Collaborator Contribution | They used the algorithms to great affect and we developed together based on their feedback a more optimised and expansive solution. |
Impact | Carravilla P, Chojnacki J, Rujas E, Insausti S, Largo E, Waithe D..... Nieva JL, (2019). Molecular recognition of the native HIV-1 MPER revealed by STED microscopy of single virions.. Nature communications, 10 (1), pp. 78 |
Start Year | 2018 |
Description | Twitter posts relating to research |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Tweeting links to papers and providing interactive resources: https://twitter.com/dwaithe/status/1094195706905194500 |
Year(s) Of Engagement Activity | 2018,2019 |
URL | https://twitter.com/dwaithe/status/1094195706905194500 |