Quantitative and Real-Time Image Analysis for Advanced Light Microscopy.

Lead Research Organisation: University of Oxford
Department Name: Weatherall Inst of Molecular Medicine

Abstract

In microscopy based research it is common to acquire and analyze microscopy images of cells sampled from populations which have undergone differential treatments. A non-exhaustive number of samples (i.e. images of cells) are often acquired due to experimental limitations that include: restricted acquisition periods, provision of expensive equipment or pressures to move onto the next experiment. Random sampling is a methodology that is used to sample representatively. There are some good reasons however for not sampling randomly. For example, cells maybe expressing a reporter protein and the copy number (quantity) of this molecule maybe too high or too low for subsequent analysis and so preference is shown toward those examples where the expression levels are within a certain range. Or, alternatively, the cells in an area maybe compromised by an artifact, be unhealthy, or too dense or sparse for a particular analysis. There are many factors that will influence the choices of an imaging researcher that are fully justifiable, but it is a problem as to how this information is documented and shared with other scientists. Removing the scientist from the pipeline of acquisition and analysis is counter-productive, a better solution is to provide tools that allow them to document and describe any experimental subjectivity as well as boosting reproducibility through automation. The objectives of this first part of the project (1-4) revolve around solving these issues of documenting, describing and automating image acquisition so that experimental design can be better communicated between users and laboratories.
Furthermore, we would like to develop tools and approaches, which allow better visualization and real-time feedback for advanced approaches so as to better inform imaging scientists as they perform their research and make experimental decisions. The second part of the project relates to this goal, objectives 5-6.

Technical Summary

In terms of microscopy analysis, great inroads have been made in terms of impartial and systematic analysis but little has been done to ensure that cells under the microscope are selected impartially. For this project DW will develop algorithms that can statistically quantify and describe cellular appearances utilizing the latest machine learning, computer vision (CV) and signal processing techniques and technologies.
DW has in ongoing work investigated the use of CV algorithms in microscopy and has shown very promising results can be achieved by utilizing object detection convolutional neural networks for cellular detection. DW will extend on his use of neural networks for localizing cells (objectives 1-2) and will also develop methods to statistically describe cellular appearance using networks derived from auto-encoders (objective 3), a type of compression network. Neural networks are implemented in several ways, a popular method is to use Tensorflow and DW is an expert in this language. To effectively train neural networks, powerful GPUs are required. Fortunately the WIMM has various computational facilities and DW has access to two powerful GPU equipped servers.
For developing CV and real-time analysis approaches to work with camera and detector hardware, DW will work toward developing algorithms that can be embedded in miniaturized electronics (objective 4). The Nvidia Jetson TX2 Developer kit is a resource which allows you to create and develop neural networks and distribute them onto small hardware boards. DW will develop and test this hardware technique with microscopy based hardware and algorithms.
For objectives 5 and 6 GPU code will be systematically produced in CUDA and will be made as compatible and distributable as possible. Oxford University has several microscopy software development projects (e.g. in Micron) and it is intended that the software and libraries produced by this project will be made compatible with these other projects.
 
Description Chair of the Image Analysis Focused Interest Group of the Royal Microscopy Society
Geographic Reach National 
Policy Influence Type Membership of a guideline committee
Impact The bioimage analysis community is not well represented within the research community in the UK and beyond. The goal of this focused interest group, which I was asked to chair by the Royal Microscopy society, is designed to highlight and draw together the community of bioimage analysts in the UK. So far we have developed a number of questionnaires, staged events and contributed talks at a number of scientific conferences to raise awareness for this group.
URL http://iafig-rms.org/
 
Description Organised and taught on a Python Bioimage Analysis course
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact We trained 46 bioimage scientists to become fully trained Bioimage Analysts. They developed Python image analysis skills and will apply these research skills across the UK and continental Europe.
URL https://github.com/IAFIG-RMS/Python-for-Bioimage-Analysis
 
Description Training of Bioimage Analysis Instructors.
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact We trained a number of bioimage analyst (16) to be trainers. This has lead to subsequent enhanced and additional training.
URL https://analyticalscience.wiley.com/do/10.1002/micro.2914/full/
 
Description UK management committee member for NEUBIAS
Geographic Reach Europe 
Policy Influence Type Membership of a guideline committee
Impact Neubias is an COST funded initiative designed to raise awareness for and to support image analysis on the EU and worldwide level. I along with Graeme Ball represent the UK in this scheme. I attend meetings representing the UK and contribute to workgroups which influence how this group contributes to things like ontologies, training and databases for software, as well as best practise for the community. e.g. https://twitter.com/matuskalas/status/1093479463155912711
URL http://eubias.org/NEUBIAS/venue/who-are-we/uk-members/
 
Title Automated Microscope Control Algorithm software repository. 
Description This is a software framework which allows a microscope to be autonomously controlled using the feedback from a machine learning/computer vision algorithm. https://github.com/dwaithe/amca It relates to the paper published as a pre-print here: https://www.biorxiv.org/content/10.1101/544833v1 
Type Of Material Technology assay or reagent 
Year Produced 2019 
Provided To Others? Yes  
Impact There is a lot of interest from the research community. The paper pre-print has already an altmetric of 42 and has been viewed around the world: https://biorxiv.altmetric.com/details/55202241 
URL https://github.com/dwaithe/amca
 
Title Fluorescence Microscopy Data for Cellular Detection using Object Detection Networks. 
Description As a by-product of my research we have developed a public collection of bioimage datasets with annotations. This can be freely downloaded and used by the community as a whole. 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? No  
Impact No notable impact yet, but publication citations should follow. 
URL https://zenodo.org/record/2548493#.XIZMUxOwnJw
 
Description Application of image analysis algorithms for Molecular recognition of the native HIV-1 MPER. 
Organisation University of the Basque Country
Country Spain 
Sector Academic/University 
PI Contribution Algorithms developed by myself for the analysis of STED images were used in their publication.
Collaborator Contribution They used the algorithms to great affect and we developed together based on their feedback a more optimised and expansive solution.
Impact Carravilla P, Chojnacki J, Rujas E, Insausti S, Largo E, Waithe D..... Nieva JL, (2019). Molecular recognition of the native HIV-1 MPER revealed by STED microscopy of single virions.. Nature communications, 10 (1), pp. 78
Start Year 2018
 
Description Twitter posts relating to research 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Tweeting links to papers and providing interactive resources:
https://twitter.com/dwaithe/status/1094195706905194500
Year(s) Of Engagement Activity 2018,2019
URL https://twitter.com/dwaithe/status/1094195706905194500