CropQuant - Next-generation cost-effective crop monitoring system for breeding, crop research and digital agriculture

Lead Research Organisation: Earlham Institute
Department Name: Research Faculty

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
 
Description (1) We have contracted a marketing company to conduct a market research to verify the commercial value of the CropQuant technology; (2) we arranged several visits and Skype conferences to discuss with industrial companies regarding the technology; (3) we successfully engaged with Bayer Crop Science through our interactions with the industry and jointly applied and was awarded a Bayer G4T focused project, which only awarded to two research groups globally in 2017. (4) Through our interactions with the industry, we were invited to showcase our CropQuant breeding version and CropQuant farming robot in REAP 2017.

2018-2019: new i-team market report suggests that CropQuant technology is likely to be fully exploited by viticulture. So, we are starting discuss with a viticulture link through UEA (https://www.vinewineconsultancy.com).
Exploitation Route (1) For academic EU users, CropQuant hardware and software will be freely available for BBSRC's Designing Future Wheat programme as well as a broader plant research community to use through out online repository at Earlham Institute. (2) For industrial users and non-EU users, we are in discussion to license the CropQuant technology to further developing and resolving different agricultural problems in these non-EU countries.

2018-2019: through DFW as well as open source publications.
Sectors Agriculture, Food and Drink,Digital/Communication/Information Technologies (including Software),Environment,Financial Services, and Management Consultancy,Manufacturing, including Industrial Biotechology,Other

URL http://www.earlham.ac.uk/zhou-group
 
Description The CropQuant technology has been required by leading bio-tech and breeding companies including Bayer Crop Sciences, Syngenta, Limagrain and ADAS. We were also awarded Bayer's G4T focus grant in 2017 based on CropQuant technology. A number of non-EU companies are interested in purchasing our CropQuant patent and discussions are ongoing. 2018-2019: The hardware and software solutions of the IoT-based crop monitoring solution called CropQuant (UKIPO, GB1709756.9, international PCT Patent Application No. PCT/GB2018/050985) and derived AI-based Agri-Robot (CropQuant-R) have attracted much industrial interest. The East Asia licence is in the process of being sold to a research organisation in China for US$80,000 plus VAT, plus 5% royalty.
First Year Of Impact 2019
Sector Agriculture, Food and Drink,Digital/Communication/Information Technologies (including Software),Education,Environment,Other
Impact Types Societal,Economic

 
Title CropQuant distributed phenotyping workstations 
Description Cost effective automated phenotyping device, CropQuant, is capable of providing continuous and precise measurements of traits that are key to todays crop research, breeding and agronomic practices. The high-frequency and high-precision phenotypic analysis can enable the accurate delineation of the genotype-to-phenotype pathway and the identification of genetic variation influencing environmental adaptation and yield potential. To manage distributed infield experiments and crop-climate data collection, we have developed a web-based control system called CropMonitor to provide a unified graphical user interface (GUI) to enable real-time interactions between users and their experiments. Furthermore, we established a high-throughput trait analysis pipeline for phenotypic analyses so that lightweight machine-learning modelling can be executed on CropQuant workstations to study the dynamic interactions between genotypes (G), phenotypes (P), and environmental factors (E). 
Type Of Material Improvements to research infrastructure 
Year Produced 2017 
Provided To Others? Yes  
Impact The bio preprint version (https://www.biorxiv.org/content/early/2017/09/01/161547) has been introduced to the public in September 2017. We have utilised the CropQuant technology to enable the acquisition of high-quality sensor- and image-based plant data in the field as well as in greenhouses (e.g. the published Speed Breeding paper). It was presented in a number of Agri-Tech and innovation conferences such as KTN Automation in Agriculture as well as REAP 2017. We have also successfully established collaborations with ADAS and Bayer Crop Science to apply CropQuant in their breeding and crop research programmes. 
URL https://github.com/Crop-Phenomics-Group/CropQuant/releases/
 
Title CropSight - an automated data collation, storage, and phenotyping management system 
Description CropSight is a PHP and SQL based server platform, which provides automated data collation, storage, and information management through distributed IoT sensors and phenotyping workstations. It provides a two-component solution to monitor biological experiments through networked sensing devices, with interfaces specifically designed for distributed plant phenotyping and centralised data management. Data transfer and annotation are accomplished automatically though an HTTP accessible RESTful API installed on both device-side and server-side of the CropSight system, which synchronise daily representative crop growth images for visual-based crop assessment and hourly microclimate readings for GxE studies. CropSight also supports the comparison of historical and ongoing crop performance whilst different experiments 
Type Of Material Improvements to research infrastructure 
Year Produced 2019 
Provided To Others? Yes  
Impact CropSight is a scalable and open-source information management system that can be used to maintain and collate important crop performance and microclimate information. Big data captured by diverse technologies known collectively as the Internet of Things (IoT) is extremely difficult to calibrate, annotate and aggregate. This presents a major challenge for plant scientists trying to understand the dynamics between crop performance, genotypes and environmental factors and for agronomists and farmers monitoring crops in fluctuating agricultural conditions. The new system developed by researchers from the Earlham Institute, John Innes Centre, and University of East Anglia (UEA) provides near real time environmental and crop growth monitoring. 
URL https://github.com/Crop-Phenomics-Group/CropSight
 
Title CropSight - a crop phenotyping management system 
Description As a scalable and open-source information management system, CropSight can be used to maintain and collate important crop performance and microclimate datasets captured by IoT sensors and distributed phenotyping installations. It provides near real-time environmental and crop growth monitoring in addition to historical and current experiment comparison through an integrated cloud-ready server system. Accessible both locally in the field through smart devices and remotely in an office using a personal computer, CropSight has been applied to field experiments of bread wheat prebreeding since 2016 and speed breeding since 2017. We believe that the CropSight system could have a significant impact on scalable plant phenotyping and IoT-style crop management to enable smart agricultural practices in the near future. 
Type Of Material Data handling & control 
Year Produced 2018 
Provided To Others? Yes  
Impact CropSight can enable the accessibility of locally in the field through smart devices and remotely in an office using a personal computer. It has been applied to field experiments of bread wheat prebreeding since 2016 and speed breeding since 2017. We believe that the CropSight system could have a significant impact on scalable plant phenotyping and IoT-style crop management to enable smart agricultural practices in the near future. We have used the CropSight system in our collaboration with BASF and Bayer Crop Science. 
URL https://github.com/Crop-Phenomics-Group/CropSight
 
Description Awarded Bayer G4T Focus grant for hybrid wheat research, only two were funded globally in 2017 
Organisation Bayer
Department Bayer CropScience Ltd
Country United Kingdom 
Sector Private 
PI Contribution Utilising the CropQuant technology to develop novel image-based machine learning technologies to enable trait measurements of spikes per unit area, spikelet number, and anther extrusion for hybrid wheat seed production breeding at Bayer. The Zhou lab is the sole academic partner on this grant. My lab will provide (1) the improved CropQuant technology to collect high-quality wheat growth images in the greenhouse and the field; (2) advanced image analysis and machine learning algorithms to detect anther extrusion over time; (3) a deep learning based analysis solution to quantify spikes per unit area and spikelet/spike numbers in field trials.
Collaborator Contribution Dr M. Schmolke and Dr M. Kerns at the European Wheat Breeding Centre, Crop Science Division, Bayer AG, will support the awarded project using (1) male and female breeding lines for hybrid seed production at Bayer; (2) the deployment of hardware and software toolkit for Breeding & Trait Develop in the hybrid wheat seed production; (3) the integration of this project in the breeding programme at Bayer.
Impact This project will provide a unique opportunity to incorporate genetics, computing sciences (computer vision, deep learning, remote sensing, and growth modelling), and breeding in one multidisciplinary R&D project. The project is just initiated in Jan 2018.
Start Year 2018
 
Title CropQuant - data processing of crop images 
Description Field of the invention: the present invention relates to data processing of images of a crop, in particular a cereal crop such as wheat, maize or rice, for use in image-based field phenotyping. The method comprises retrieving a series of images of a crop captured over a period of time and identifying, in an image (or "initial image") selected from the series of images to be used as a reference image, a reference system against which other images can be compared, the reference system including an extent of a crop plot and/or one or more reference points. The method also comprises, for each of at least one other image in the series of images, calibrating or adjusting the image using the reference system, and determining a height of a canopy of the crop in the image, a main orientation of the crop and/or a value indicative of vegetative greenness (for example, a normalised green value in an RGB colour space and/or excessive greenness). This can afford greater flexibility when monitoring a crop, particularly large numbers of crops, over periods of months. For example, the method can be used to process images of a crop which have been captured in the field and, thus, subject to vagaries of weather. Moreover, the method can be used for each crop and, thus, allow large data to be processed for large numbers of crops. 
IP Reference GB1709756.9 
Protection Patent application published
Year Protection Granted 2018
Licensed Commercial In Confidence
Impact CropQuant technology has been reported by UK national and EU media over a dozen of times since 2016. It has been featured as a stand-out example of UK-based Agri-Tech innovations and was presented BBSRC's Harvest 2050 and other industry-related activities. Recently, our bespoke farming robot CropQuant Sheila was invited to exhibit at REAP 2017, an event organised by Agri-Tech East, KTN and Innovate UK.
 
Title CropQuant to measure cereal growth 
Description A method of processing (batch processing) images of a crop (pot, plot or field) comprising: retrieving a series of images of a crop (pot, plot or field) captured over a period of time; identifying, in an (initial) image selected from the series of images to be used as (a) reference image, a reference system against which other images can be compared, the reference system including an extent of a crop plot and a set of (key reference points, such as the plot region, the canopy space and) height markers ; and for each of at least one other image in the series of crop growth images. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2017
Licensed Commercial In Confidence
Impact The hardware and software solutions developed by my lab, i.e. the IoT-based crop monitoring solution called CropQuant (UKIPO, GB1709756.9, international PCT Patent Application No. PCT/GB2018/050985) and derived AI-based Agri-Robot (CropQuant-R) have attracted much industrial interest. The East Asia licence is in the process of being sold to a research organisation in China for US$80,000 plus VAT, plus 5% royalty.
 
Title CropQuant Software System 
Description The CropQuant in-field phenotyping platform provides a cost-effective Internet of Things (IoT) powered crop monitoring system for wheat and other cereal crops, designed to be easily used and widely deployed in any environment. To manage and process data generated by the platform, we developed an automatic control system, high-throughput trait analysis algorithms, and machine-learning based modelling to explore the dynamics between genotypes, phenotypes and environment. This technology can be applied to breeding, cultivation, crop research, and digital agriculture. 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2017 
Impact Since September 2017, the preprint of the CropQuant manuscript has been tweeted for 35 times, downloaded for more than 1272 times. CropQuant was reported by BBSRC and other industry-related activities such as Syngenta and Bayer Internal Seminar Series. CropQuant has also been featured as a stand-out example of UK-based Agri-Tech innovations. Recently, our bespoke farming robot using the CropQuant software system, Project Sheila, was invited to exhibit at REAP 2017, an event organised by Agri-Tech East, KTN and Innovate UK. 
URL https://github.com/Crop-Phenomics-Group/CropQuant/releases
 
Title CropSight - a scalable and open-source information management system for distributed plant phenotyping and IoT-based crop management 
Description CropSight is a PHP and SQL based server platform, which provides automated data collation, storage, and information management through distributed IoT sensors and phenotyping workstations. It provides a two-component solution to monitor biological experiments through networked sensing devices, with interfaces specifically designed for distributed plant phenotyping and centralised data management. Data transfer and annotation are accomplished automatically though an HTTP accessible RESTful API installed on both device-side and server-side of the CropSight system, which synchronise daily representative crop growth images for visual-based crop assessment and hourly microclimate readings for GxE studies. CropSight also supports the comparison of historical and ongoing crop performance whilst different experiments are being conducted. 
Type Of Technology Webtool/Application 
Year Produced 2019 
Open Source License? Yes  
Impact As a scalable and open-source information management system, CropSight can be used to maintain and collate important crop performance and microclimate datasets captured by IoT sensors and distributed phenotyping installations. It provides near real-time environmental and crop growth monitoring in addition to historical and current experiment comparison through an integrated cloud-ready server system. Accessible both locally in the field through smart devices and remotely in an office using a personal computer, CropSight has been applied to field experiments of bread wheat prebreeding since 2016 and speed breeding since 2017. We believe that the CropSight system could have a significant impact on scalable plant phenotyping and IoT-style crop management to enable smart agricultural practices in the near future. 
URL https://academic.oup.com/gigascience/advance-article/doi/10.1093/gigascience/giz009/5304887
 
Title Leaf-GP Software Distribution 
Description Leaf-GP software is published with the software paper entitled "Leaf-GP: an open and automated software application for measuring growth phenotypes for arabidopsis and wheat" in Plant Methods in last December. The software is distributed under the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The software is a sophisticated software application that provides three approaches to quantify growth phenotypes from large image series. We demonstrate its usefulness and high accuracy based on two biological applications: (1) the quantification of growth traits for Arabidopsis genotypes under two temperature conditions; and (2) measuring wheat growth in the glasshouse over time. The software is easy-to-use and cross-platform, which can be executed on Mac OS, Windows and HPC, with open Python-based scientific libraries preinstalled. It presents the advancement of how to integrate computer vision, image analysis, machine learning and software engineering in plant phenomics software implementation. To serve the plant research community, our modulated source code, detailed comments, executables (.exe for Windows; .app for Mac), and experimental results are freely available at https://github.com/Crop-Phenomics-Group/Leaf-GP/releases. 
Type Of Technology Software 
Year Produced 2017 
Open Source License? Yes  
Impact The Leaf-GP paper generated 18169 downloads within a month and received an attention score of 21 (https://biomedcentral.altmetric.com/details/30811558#score), No.1 amongst all Plant Methods papers (48 in total) published from October 2017 and the attention scoring of the paper is higher than 91% of its contemporaries published from October 2017, worldwide. 
URL https://github.com/Crop-Phenomics-Group/Leaf-GP/releases
 
Description AI in life sciences - Presentation at the Royal Institute of Great Britain, organised by Royal Society of Biology 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Artificial Intelligence - Can AI save the world? Dr Ji Zhou, Phenomics Project Leader at the Earlham Institute. Discussions regarding the inclusion of self-learning algorithms into systems that underpin modern life could help us address many of the global challenges society faces. But they also carry risks that may have devastating results. Can AI truly save the world, or is it too risky to depend on machine learning to solve our problems?

This event is presented in partnership with the Royal Society of Biology, the Biochemical Society and the British Pharmacological Society for Biology Week 2018.

Biology Week 2018 is from 6th-14th October and showcases the important and amazing world of the biosciences, getting everyone from children to professional biologists involved in fun and interesting life science activities.
Year(s) Of Engagement Activity 2018
URL https://www.rsb.org.uk/get-involved/biologyweek/royal-institution-debate
 
Description Automation and Robotics Event Organised by KTN, BBSRC and Innovate UK 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact I gave a talk entitled "An integrated approach to understand our crops, from sky to field", at the KTN Agrifood Automation & Robotics Event in Peterborough on Friday 4th July. Robotic based material handling and processing systems are presented to Agri-Food industry in order to give a major uplift in productivity for food manufacturers.
Year(s) Of Engagement Activity 2017
URL https://www.oalgroup.com/events/ktn-agrifood-automation-robotics-event-belfast-axleb
 
Description BBSRC Harvest 2050 - Science Media Centre 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Interviewed by the media in terms of the Zhou Lab's Agri-Tech innovative products, which were then reported by Guardian and other national media.
Year(s) Of Engagement Activity 2017
URL https://www.theguardian.com/environment/2017/nov/14/miniature-robots-could-cut-pesticide-use-on-farm...
 
Description Exhibit in Royal Norfolk Show 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact My lab exhibited technologies invented by the Zhou laboratory at Norwich Research Park and was interviewed by Royal Norfolk Show, Agri-Tech East, and EDP.
Year(s) Of Engagement Activity 2018
URL https://royalnorfolkshow.rnaa.org.uk/
 
Description NRP Translational Fund public engagement night 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Supporters
Results and Impact Over 200 people attended the NRP translational funding night aimed for raising further funding to support Agri-Tech innovations. I presented CropQuant technologies derived from NRP Translational Fund in the event.
Year(s) Of Engagement Activity 2019
URL http://www.nrp.ac.uk/nrp-funded-projects/
 
Description REAP 2017 - CropQuant Robot 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact At REAP 2017 we presented our newly invented CropQuant farming robot as an emerging agri-tech innovation for the UK's Agri-Food sector. We also shared our experiences and interacted with business and industrial practitioners in terms of crop monitoring, robotics, machine learning, future visions, and how the CropQuant technology could provide insights into emerging agri-tech developments.
Year(s) Of Engagement Activity 2017
URL https://www.agritech-east.co.uk/cropquant-grows-with-crop-to-provide-robot-eye-on-performance/