Active Microscopy: Machine learning optimization of cell-based imaging microscopy.

In cell biology, scientists investigate cells and cellular components to understand the biological mechanisms that enable these organisms to function. It is very common to grow cells in culture systems and then to stain certain parts of these cells with fluorescent or contrasting molecules. The staining procedure highlights specific parts of the cell or organism, which can then be subsequently viewed and imaged with a microscope. Microscopy experiments often require a skilled user to setup the measurements, but once configured, involve repetitive steps to be performed to complete the experimentation. Often cell biologists will spend a lot of their time manually seeking out the position of cells of interest (e.g. with specific properties) on their sample slide before imaging them. Cell biologists are highly trained individuals but often this procedure of image acquisition and optimisation can be time consuming and wasteful use of their time, and might also become objective. Could this process be automated? There have already been attempts to automate cell biology imaging assays. Unfortunately, these systems are designed to automate a particular type of imaging experiment and/or use specialised equipment that is not flexible enough to use for day-to-day prototyping of cell biology experiments. The purpose of this project is to integrate machine learning techniques, which are intelligent and adaptive, with conventional microscopy. Due to advances in the fields of computer science and artificial intelligence over the last 10 years, the technology required to achieve the proposed goals is very feasible. These algorithms that can learn from example will form the foundation of the approach. With minimal human guidance the proposed microscope system will be able to perform cell biology imaging experiments and the process will be fully logged, accountable and reproducible. The proposed system will be a significant contribution to the arsenal of tools which imaging scientists can use to perform their experiments and this is the perfect time to develop such a technology.

Machine learning (ML) is a sub-field of Artificial Intelligence (AI) and its purpose is to describe algorithms that can learn without being explicitly programmed. Due to recent advances in ML we are now at a stage whereby we can consider embedding AI into the equipment we use to support our productivity. In this project we propose to create a microscope system that is tightly integrated with state-of-the-art ML algorithms so that it will be able to autonomously perform imaging experiments. Using ML we can create a tool that is powerful and flexible and which can directly learn from human scientists.
We will take an already existing microscope in the applicant's lab and propose to upgrade the stage, camera and illumination of the microscope so that the acquisition can be fully computerised and interfaced through a desktop computer. This computer will run LabView, a powerful hardware software interface and will also run python scripts that will facilitate the algorithmic control of the system and connect to the user interface.
Our proposed algorithmic framework is divided into two parts (Assessment and Fine-Grain). For the Assessment, we intend to use a ML density estimation technique, which will coarsely locate the cells. This algorithm is implemented using fast decision tree ensemble methods. For the Fine-Grain classification we intend to use a combination of Convolutional and Recurrent Neural Networks that will learn to distinguish cells based on the association of structures within the cell and the human classification of the same cell. The user will show the system how to perform the experiment by selecting the first examples, from which the system will then learn and then will be able to perform the rest of the experimentation. The system's algorithms, which have been proven to work in other areas of computer science, are of great interest to the community and resulting findings will not only inform the biomedical community but the ML community also.

Our project will have a considerable impact on the microscopy, cell biology and biophysics academic communities. Although automated microscopes have been created in the past, this is the first time that a substantial amount of artificial intelligence (AI) has been applied to control a conventional microscope. Because of this, we expect the project to be appealing not only as a cutting-edge piece of research but also as a tool that can be leveraged in many different projects due to the fields in which our collaborations reside (e.g. peroxisome, immunology, cancer and haematology fields). Future implementation of our Active Microscopy approach on microscopes within the host institutes open-user image facility will make it accessible to a broad range of researchers within Oxford and UK-/world-wide. As a result, we expect this project to have a substantial impact on the community within Oxford, nationally and also internationally. The developments of this project will be communicated through peer-reviewed publications, conferences and through social media. Both Prof. Christian Eggeling (CE) and Dr. Dominic Waithe (DW) have a long history of publishing in these domains.
Because we are focusing on the acquisition phase of the experimentation and this involves directly interacting with microscopy hardware we expect that this research will be of interest to microscopy vendors. As part of this project we have established a project-specific collaboration with PicoQuant and we also expect interest with other microscopy companies such as Zeiss, Leica, Nikon and Olympus who are active in this domain and with whom we have collaborations. Furthermore we propose to develop a novel augmented reality system for the microscopes that we expect will have a big impact on the microscopy community as a whole due to its potential for improving user-interactivity. CE has significant experience working with microscopy companies and both CE and DW will communicate the developments of this project directly with these companies and through conference portals.
We expect this project to also have an impact on the AI academic research community. The algorithms we are using are cutting edge and because of the synergies this approach has with work in other areas we expect to make an impact in many sub-fields of machine learning discipline. DW already has a record of publishing within the machine learning community and will continue to do so. Because we are using cutting-edge machine learning algorithms which are being actively applied in other domains, we expect that this project will have a significant impact on companies which focus on applying machine learning in commerce and media (e.g. Microsoft, Alphabet DeepMind and also Facebook). These companies are interested in accessing new markets and so it is likely that this project will be of interest. We will communicate our findings to these companies through our academic connections with these firms as well as at biophysics, microscopy, computer vision and machine learning conferences where there is an opportunity to meet with the vendors.
Besides using conventional academic portals to ensure that our research is communicated we are also going to make use of social media. DW has significant experience using social media and has some 300,000 views for his scientific communication Youtube account and also maintains a popular Twitter profile. Furthermore DW also maintains an active Github and Quora account which can also be leveraged to share the findings of this project and to answer questions from interested parties. We intend to develop several short movies that will document and highlight the various aspects of this work and then will publicise these using all of the mentioned social media platforms along with CEs group page (http://www.nano-immunology.org/). AI and robotics is currently a very popular topic on the internet and so we are sure to have some impact in these arenas with our project.