LeMuR: Plant Root Phenotyping via Learned Multi-resolution Image Segmentation

Plant phenotyping - the measurement of quantitative data on plant structure and function from image and sensor data - is a key bottleneck holding back efforts towards global food security; that is, providing enough food for a growing population. The roots of food crops are clearly important for the development of the crop itself, yet root phenotyping is particularly challenging, as the roots grow in soil. Though methods of imaging roots in soil are emerging, they remain slow and expensive. Large-scale experiments are still performed using artificial growth media (gel, filter paper etc.) that allow the root to be imaged using conventional equipment. Analysis of the resulting images requires the root to be separated from its background and a structural description of the root architecture to be produced and presented to the user. But doing this fully automatically is a challenge, and most software tools written to date work with very specific sets of images, and tend to break if used outside of the scenarios they were designed for.

In this proposal we will develop cutting-edge deep learning analysis approaches to build a much more general software tool. So-called deep approaches are revolutionising image analysis, with large companies developing similar techniques to analyse other image sets, such as for diagnosing medical conditions, to great effect. The proposed approach, LeMuR (Learned Multi-Resolution image segmentation), will exploit the common structure of root image analysis tasks, and recent advances in deep machine learning, to produce a flexible plant root phenotyping tool that can be easily adapted, without re-writing code, to new laboratory environments and imaging techniques.

We propose two main developments. First, a software tool LeMuRoot which will be designed to work across a wide variety of root system data sets right out of the box, compared to the limited application of traditional tools. Second, a software framework (LeMuRLearn) to allow biologists themselves to adapt the tool to even more images beyond those that LeMuRoot was designed to work with. By supplying their own image data sets annotated using a novel user interface which will form part of LeMuRLearn, biologists will be able to re-train the core model underlying the tools, allowing them to improve the quality of results for their particular data.

In a further novel process, biologists will be able to seamlessly share their newly trained tool with the community, which in turn can be used as a base for further development. This will allow LeMuRLearn to incrementally improve over time, and for it to use different underlying models for a wider variety of image data sets than was conceived of at initial release.

This is exciting for two main reasons. First, previous development of software tools has by necessity been limited to computer scientists who are capable programmers - here we put the continued development of the tool in the hands of the biology community themselves. Second, by sharing the core model underlying the tool (called the LeMuRNet), biologists can share with the community without fear of sharing raw data or results.
Combined with hiding the computational complexity of both the analysis and evolution process behind an accessible user interface, the potential to disrupt the current process for image analysis tool development and use with plant science (and beyond) is high.

The proposed approach, LeMuR, will exploit the common structure of root image analysis tasks and recent advances in deep machine learning to produce a flexible plant root phenotyping tool that can be easily adapted, without re-writing code, to new laboratory environments and imaging techniques. It comprises two novel software components.

The tool, LeMuRoot, will bring together i) a novel, learned multi-resolution root image segmentation method based on a convolutional neural net, ii) optimal path finding to identify the root skeleton, and iii) RSML format description of root architectures. By considered development in this proposal, LeMuRoot will be broadly applicable to any common 2D, image-based, root system architecture phenotyping task, subsuming previous tools. When necessary, users will adapt the tool to their own situation by further training of the convolutional net, a process that will not require specialist knowledge. Training will be conducted via the LeMuRLearn learning framework, which will both allow biologist users to produce their own training data and automatically update the network. This additional training will increase the power and generality of LeMuRoot, which will be made available to the community via a variant of the standard Open Source process.

There are three main areas of development (which correspond to the work packages of the proposal):

1. Developing and training the initial deep network for root system segmentation (LeMuRNet)

2. Building the generic 2D root system analysis tool using the output from (1) to segment root systems (the LeMuRoot tool)

3. Developing an accessible, extensible tool for the community (LeMuRLearn)

In this proposal we aim to not only produce a more widely useful root system analysis tool than has been produced previously, but also to provide an approach and software environment biologist users can use to improve the tool themselves and, if they wish, share their improved tool with the community. LeMuR's impact then lies both with the tool and algorithms, and in changing the nature of the way such tools evolve and are shared over time.

Who will benefit from these outputs, and how?

Plant and Crop Scientists (Academic and Industry):
The project will impact these scientists in two sizeable ways (1) by providing a ready-to-use software tool which is able to analyse a wide variety of 2D root architecture (2) by providing the means with which a biologist (rather than a computer scientist) can improve it further. This new improved tool will directly impact the plant scientist who develops it but, by re-publishing the new version of the tool, it will also likely see uptake by other biologists. This is a step-change from the standard open source model of software dissemination, as altering existing tools has typically meant that a qualified computer scientist must program the tool directly. To increase impact for the scientists, we will track who changes a particular version of the tool, to maintain a citation trace to assign credit to be used in future publications.
The use of such an Artificial Intelligence-based approach will help to raise awareness of the impact of this new suite of tools within the plant (and wider biological) sciences. Such approaches are already having great impact in other domains (such as medicine (e.g. Google's DeepMind Health project).

We are already aware of the demand for such deep learning approaches within industry, particularly because of the vast datasets being captured. There is no doubt there is a 'big data' bottleneck, and LeMuR addresses this directly. Any newly trained LeMuRNet deriving from industrial work can be kept private, allowing industry to retain IP.

Farmers (Industry)
In the long term, the insight provided by this tool can be expected to lead to the identification of new crop phenotypes, which could lead to new crop varieties being available for farmers. These could address problems such as soil nutrition or water deficiency by identification of traits that affect root system architecture.

Computer scientists and Image analysts (Academic and Industry):
We will be developing cutting-edge deep learning approaches to underpin the tool's main functionality. This will be publishable in high-impact technical journals, and by doing so we will encourage other computer scientists to tackle similar biological challenges. Computer scientists working in the deep learning field will also be able to take the open source networks trained using this system and apply them in other domains. The network architecture developed is likely to be applicable beyond plant sciences, to the analysis of other narrow, elongated structures. The core design of LeMuRNet is likely to find application in the analysis of medical (e.g. arterial), remote sensing (e.g. roads and rivers) and document (e.g. drawings and sketches) images.
The LeMuRNet version control tracking approach, whilst built on existing computing technologies, will see novel adaptation and application in this proposal, and may impact the software engineering research sector of computer science.

The public
In the long term, new and improved crop varieties can lead to more stable and efficient food production, which could both lead to improved nutritional content of food and potentially lower food prices.

Schools
There is an opportunity to present to young scientists the idea that you can develop cutting edge, exciting computer science which can have a real impact on biological experiments which are important for the future of life on the planet. Demonstrating that such a career choice exists is powerful impact.