An affordable stereoscopic camera array system for capturing real-time 3D responses to vegetation dense environments

To feed our growing global population, the productivity of staple crops will require an increase in yields of ca. 60% by 2050. Realisation of this significant challenge is currently hampered by insufficient capacity of the plant science and Agri-Tech communities to analyse existing plant genetic resources for their interaction with the environment. Plant phenotyping is an emerging science that links genomics with plant ecophysiology and agronomy. The collection of architectural traits of a plant throughout the life cycle (the phenotype) is a result of dynamic interactions between the genetic background (the genotype) and the physical world in which the plant develops (the environment). These interactions determine plant performance and productivity measured as accumulated biomass and commercial yield, and resource use efficiency. A critical parameter of a plant's phenotype is the shape in three dimensions - the plant architecture - which reflects the adaption of a plant to environmental conditions, such as light quantity and quality, and temperature. We will bring together plant scientists, and image capture and machine vision experts to develop an affordable and user-friendly system for progressing on one of the most important issues in biology, our capacity to enhance the productivity of plants. We will combine a robust photometric stereo approach with software using optimised learning algorithms that will produce standardised outputs that can be readily integrated across scales with other biological data sets.

There is an urgent need for new technologies that allow us to monitor and predict the impact of abiotic and biotic stresses on seasonal plant growth. Such tools could provide quantitative data on crop yield traits and plant fitness markers in natural habitats. The current project is a collaboration between plant researchers, modellers, engineers and image analysis specialists that aims to take a new approach, by producing i) an affordable and robust photometric stereo platform for 3D capture of rosette growth in Arabidopsis that can be extended to other crops, such as related Brassica species (e.g. Brassica rapa), and ii) a standardised open source software based in optimised machine learning algorithms and fronted by a biologist-friendly user interface for simplifying the extraction of traits of interest. To test the system we will target an important plant adaptive response that can negatively impact plant biomass and crop yield - the shade avoidance response (SAR). Initial computational work will focus on processing image data to gather 3D information for plant reconstruction through intelligent integration over the surface normal field. Subsequent image/model analysis will centre on the optimisation of leaf segmentation algorithms using features commonly used in machine vision and new approaches rooted in unsupervised feature learning. Following segmentation, we will quantify the plant phenotypic traits required for evaluating the key features of SAR, including: leaf emergence rates, positioning, orientation, as well as alterations in leaf length, width, shape and expansion. The novel algorithms resulting from this research will be integrated into the existing UoE PhenoTiki package (http://phenotiki.com) so that we reach a wide base of users. We aim to combine 3D phenotype data with mathematical modelling to predict growth outcomes in differing growth scenarios and in unseasonal weather, including controlled environments and outside in field or natural habitats.

Who will benefit from this research?
1. Academics and researchers in all fields of plant research and machine learning.
2. UK and international science base.
4. The Agro-Tech industry including biotechnologists and plant breeders seeking to increase plant productivity, and metabolic engineers.
5. Agricultural community and advisors.
6. The University of Edinburgh.
6. The postdoctoral researcher (PDRA), and research associate (RA).
7. Public.

How will they benefit?
1. The research will have a major impact on understanding of light signalling and shade avoidance, and enhance the UK's international standing in plant science. Currently no other equipment can perform these measurements elsewhere in the world, nor integrate them with the wealth of molecular data and integrated modelling expertise available at UoE.
2. The technology is designed to enable labs in universities, research institutes and the Agro-Tech industry to pursue phenotyping approaches that have previously been largely inaccessible to them. To acquire the technology following publication, researchers will access an already established open source online resource (PhenoTiki) with an already existing user community to download a user-friendly software and have the option to reproduce the phenotyping system in their own labs, using our published designs that where possible will be made available for 3D printing.
3. Our system will provide a means of standardisation across the phenotyping field, to inform research strategies towards enhanced food security, which will be a platform for direct translation to achieving this goal in food crop plants through yield improvements. The public interest in food security and sustainable crop production, the broad biological scope of the project, and the predictive, systems approach already highlight areas of Impact.
4. The PDRA and RA will receive a wide training in plant integrative biology, hardware design and machine vision approaches, full access to professional skills and wider training courses, and the opportunity to work with two different research institutes.

What will be done to ensure they benefit from this research?
1. Publish results in high-impact journals in a timely fashion, with open access.
2. Present research results at UK and international meetings and institutions.
3. Exploit existing contacts with other UK and international academics with relevant research interests as soon as any exploitable results/materials are generated.
4. Make informal contacts with industrialists as soon as exploitable results/materials are generated; recognise and protect intellectual property to ensure wise and fruitful exploitation.
5. Provide PDRA and RA training and mentoring available at UoE and CMV, including regular reviews to monitor progress and a career development plan. We will also encourage participation in all aspects of the dissemination of research results, and understanding of the wider implications and applications of the research.
7. Use results as part of our regular engagement with non-academic audiences, e.g. local interest groups, schools, local and national shows, science showcases, media.