14TSB_ESAP Agrivision Inspector- Development of a system for low cost, remotely managed, automated crop stress monitoring and detection

Lead Research Organisation: Rothamsted Research
Department Name: Biointeractions and Crop Protection

Abstract

The TSB project is a business led collaborative project that addresses the three strands of the scope, enhancing efficiency, maximising market yield and minimising potential negative environmental impacts. The project involves an SME, Rail Vision Europe Ltd (RVL), a not for profit RTO, Rothamsted Research (RR) & a large crop production company, Certis UK Ltd (CUK) a UK subsidiary of Certis Europe BV. The project will be led by RVL. The project brings together a consortium with highly established crop specialists from RR and CUK, and electronic sensor, photonics, data processing and analytics systems specialism from RVL. The industrial research project is targeted at improving the efficieny of food production by engineering a novel, flexible multi sensor imaging system (HD, IR, flourescence) for mounting on a mobile platform (manual or robotic) for use initially in a protected crop environment. The imaging system will work with an advanced data analytics system to automatically detect and alert identified plant stresses and present the results and reports to end users in a traceable & recordable manner for automated crop monitoring & stress detection. The system will initially be targeted at protected tomato crop stresses, but is expected to be rapidly expanded to additional protected crops including cucumbers, fruits etc and eventually to an increasing range of crops. By facilitating increased crop monitoring, automated analysis and an earlier detection of crop stresses, the project facilitates enhanced efficiency in use of resources by reducing the use of herbicides/pesticides, more targeted application of resources and increased yield for a given acreage. The project supports the application of increasing integrated pest management in the protected crop environment and minimising the potentially negative impacts associated with food production. Within just the tomato growing market in the UK, worth £175m p.a., a minimum of 10% losses costs growers £17.5m per annum. Improving yield in this sector by only 2% increases tomato yields by £3.5m p.a. UK market is small proportion of EU potential. The project engineers a solution for pest and disease control building on RR and other recent research into the use of intelligent multi-sensors for the detection of crop diseases (West et al 2003; Moshou et al 2011; Sankaran et al 2010) and RVL's expertise in the development of sensor platforms design, sensor analytics, image processing and presentation (Warsop & Singh (VISAPP, 2009), Singh et al (CIS 2009 - 8th IEEE Intnl Conf. on Cybernetic Intelligent Systems). The project combines these areas of expertise to provide a unique capability for automated crop monitoring initially within a protected crop environment.
The project will provide new information on the phenotyping of crop stresses, including the timescales for development of certain symptoms, including at microscopic scales and using fluorescence imaging, which could provide methods for assessing cultivar resistance. The timescales for detection thresholds will be identified under different conditions to allow production of simple epidemiology models to predict risk of disease spread around detected foci.

Technical Summary

The TSB project is a business led collaborative project involving an SME, Rail Vision Europe Ltd (RVL), a not for profit RTO, Rothamsted Research (RR) & a large crop production company, Certis UK Ltd (CUK) a UK subsidiary of Certis Europe BV. The project will be led by RVL. The project brings together a consortium comprising highly established crop specialists from RR and CUK, and electronic sensor, photonics, data processing and analytics systems specialism from RVL. The project is targeted at improving the efficieny of food production by engineering a novel, flexible multi sensor imaging system (HD, IR, flourescence) for mounting on a mobile platform (manual or robotic) for use initially in a protected crop environment. The imaging system will work with an advanced data analytics system to automatically detect and alert identified plant stresses and present the results and reports to end users in a traceable & recordable manner for automated crop monitoring & stress detection. The system will initially be targeted at protected tomato crop stresses, but is expected to be rapidly expanded to additional protected crops including cucumbers, fruits etc and eventually to an increasing range of crops. By facilitating increased crop monitoring, automated analysis and an earlier detection of crop stresses, the project facilitates enhanced efficiency in use of resources by reducing the use of herbicides/pesticides, more targeted application of resources and increased yield for a given acreage. The project supports integrated pest management in the protected crop environment and will minimise the potentially negative impacts associated with food production. The project will provide new information on the phenotyping of crop stresses, including the timescales for development of certain symptoms under different conditions to allow production of simple epidemiology models to predict risk of disease spread around detect foci.

Planned Impact

Plant stresses cause a substantial loss of food production globally (Strange et al (2005), Oerke (2005)). Competing demands for increased productivity, reduced cost & environmental impact, increasing pathogen resistance & legislation (e.g. Sustainable Use Directive, 2009/128/EC), are leading to increased use of integrated pest management (IPM). Early stress detection leads to more effective intervention. Currently, monitoring is carried out by regular visual inspections either by farm workers or by specialised inspectors. Inspections are laborious, time consuming & expensive, limiting frequency & effectiveness. The proposed project develops a novel, flexible multi-sensor imaging system (HD, IR & fluorescence) for application on a mobile platform (manual or robotic) for use initially in a protected crop environment. The system will be initially aimed at protected tomato crops. Total UK glass house area is 1,779ha (Defra, 2013), of which ~200ha is for tomatoes, by 40 key growers. UK tomato production is worth £175m. Losses amount to at least 10%, worth £17.5m p.a. (£87,500/ha/p.a.). Global tomato production increasing, 36% up 2002-2011. EU & UK production stable at ~16m Tonnes. China grows 30% of global demand. CUK provides crop protection solutions for 70% of UK growers. The sector is capital intensive, scientifically progressive, technically advanced & environmentally responsible. Tomato Growers Assoc. identify increasing IPM & reduced pesticide use as key objectives. CUK have team of specialist crop inspector's on client's sites on a 2 weekly cycle. A remote monitoring system accurately auto-assessing, alerting & presenting crop stress incidents & identified anomalies is extremely attractive to end users, it does not currently exist & will be transformational. With a target cost per platform with equipment: £70k, plus S/W £25K for focussed crops version (£50K for multiple crop). Projected sales yr 1 £450K, yr 2 £1m, & yr 3 £2.5m providing ROI for the project 3 yrs from product launch with <40% of CUK's existing UK tomato growers. The solution will rapidly expand to more crops & new markets.
The industrial partners of the project will benefit immediately. For RVL & CUK it provides a new product/service range in a large & growing global market, with expected T/O in the UK of > £2.5m within 3 years of product launch. Operations are planned to be at RVL's & CUK's current facilities & are expected to create >12 new high skilled jobs, plus additional revenues for h/w supply chain companies. For RR it provides additional research opportunities and these could lead to improved methods for plant health inspectors to detect potentially infected plant material that can then be targeted for sampling and application of diagnostic tests (see also Academic Beneficiaries Section). For end users it provides reduced losses from crop stresses, a 2% increase in tomato yields ~ £3.5m/p.a increase in revenue. Additional benefits are expected from expansion into further crops. Social Benefits-existing surveys are undertaken manually, these are time consuming & labour intensive in a sector with skill shortages. This new approach reduces the manual labour content whilst improving outcomes & allowing highly skilled employees to accurately assess crop stresses. Environmental benefits are also expected - the project increases yield of locally grown produce, reducing food miles. Continuous monitoring, early identification & intervention for crop stresses reduces requirements for herbicides/ pesticides supporting localised crop intervention. Remote monitoring reduces transport CO2 for specialist inspectors. Additional crop monitoring opportunities are envisaged e.g. growth/yield from automated surveys can focus fertiliser inputs.
 
Description We developed a robotic system that can move in a glasshouse, record its location and record images of plants that can be analysed to highlight anomalies such as pest or disease damage. The project was a very applied project and the IP was protected by the commercial partner, Railvision Ltd. For the academic organisation, Rothamsted Research, the project partly supported two members of staff, allowing the continuation of a group working on integrated pest management, and surveillance and monitoring of diseases in crops. It has already led to invited presentations at conferences, results formed part of general review articles in refereed international journals about detection of plant diseases. It also led to two conference publications. Two papers are in the final stages of preparation for submission to refereed international journals and possibly one or two others will follow.
Refereed review article: West JS, Canning GGM, Perryman SA, King K. 2017. Novel Technologies for the detection of Fusarium head blight disease and airborne inoculum. Tropical Plant Pathology 42 (3): 203-209. DOI 10.1007/s40858-017-0138-4. Conference paper: West JS, Canning GGM, Perryman SA, Kaczmarek A, King KM (2017) Precision disease detection systems for agriculture and horticulture. Aspects of Applied Biology: Precision systems in agricultural and horticultural production v135, p1-8 (2017).



At least one additional paper is planned for submission to refereed international journals . The project reinforced and expanded our expertise and understanding of optical sensing for crop stresses, particularly disease detection. This has already led to a new funding opportunity with a different commercial company. We also applied for new InnovateUK funding with Railvision but so far have been unsuccessful.
Exploitation Route We have already secured a new InnovateUK project on a related subject, which is using optical sensing to monitor tomato and strawberry fruit ripeness and early disease detection.
This is InnovateUK project 104571 "Feasibility of using 3D MultiSpectral Imaging for enhanced classification of fruit ripeness and disease".
Sectors Agriculture, Food and Drink

URL https://www.youtube.com/watch?v=8EKERDyk2uk
 
Description The project developed a prototype system, which needs further work to commercialise, which will be pursued by the commercial partners. It also helped some commercial partners to improve various existing methods. Equipment used in the project or developed in the project is also available for future research.
First Year Of Impact 2017
Sector Agriculture, Food and Drink,Transport
Impact Types Economic

 
Description Development of Aerial Risk Inspection,Evaluation & Surveying (ARIES) platform for Cocoa Plantations
Amount £704,442 (GBP)
Funding ID 105655 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 11/2019 
End 10/2022
 
Description InnovateUK - Feasibility of using 3D MultiSpectral Imaging for enhanced classification of fruit ripeness and disease
Amount £100,000 (GBP)
Funding ID 15290 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 01/2019 
End 12/2019
 
Title Image capture for plant disease assessment 
Description We developed a facility for inoculation of potted plants with pathogen inoculum, incubation under suitable LED lighting (red and blue lights) and image capture using a digital SLR with additional lighting, including UV lighting and selective light filters in front of the camera to improve contrast or to separate fluorescence from reflectance. We also developed proficiency in processing images taken of the same subject under different lighting or filter combinations to highlight differences caused by reflectance or fluorescence attributed to pest or disease presence. This proficiency can be useful for plant phenotyping and applied detection of pests and diseases. 
Type Of Material Improvements to research infrastructure 
Provided To Others? No  
Impact This os still in development but will lead to at least one publication 
 
Description ARIES project 
Organisation Certis UK
Country United Kingdom 
Sector Private 
PI Contribution Part of a UKRI InnovateUK project (105655) developing surveillance and monitoring tools for cocoa swollen shoot virus and airborne fungal diseases in Ghana, using satellite & drone imagery data analytics and spore sampling linked with DNA-base diagnostics. This project builds on a previous collaboration InnovateUK/TSB project 'Agrivision Inspector' which ended in 2017.
Collaborator Contribution Railvision will do the image analytics Certis will provide CSSV crop protection solutions Positive Agro will provide airborne imaging and spore sampling platforms
Impact none yet - project started in Nov 2019
Start Year 2019
 
Description ARIES project 
Organisation Rail Vision Europe Ltd.
Country United Kingdom 
Sector Private 
PI Contribution Part of a UKRI InnovateUK project (105655) developing surveillance and monitoring tools for cocoa swollen shoot virus and airborne fungal diseases in Ghana, using satellite & drone imagery data analytics and spore sampling linked with DNA-base diagnostics. This project builds on a previous collaboration InnovateUK/TSB project 'Agrivision Inspector' which ended in 2017.
Collaborator Contribution Railvision will do the image analytics Certis will provide CSSV crop protection solutions Positive Agro will provide airborne imaging and spore sampling platforms
Impact none yet - project started in Nov 2019
Start Year 2019
 
Description Agrivision Inspector Project 
Organisation Certis UK
Country United Kingdom 
Sector Private 
PI Contribution Rothamsted has conducted experiments to inoculate tomato plants with different diseases and pests in order to test the best imaging methods for their detection and automated classification in collaboration with Railvision and CertisUK. The work has provided training on working with tomato diseases and pest for Rothamsted staff, Prof Jon west and Dr Sarah Perryman, and has provided training to Dr Perryman on bioImaging techniques and image processing.
Collaborator Contribution Railvision have provided hardware for imaging and lighting plants and also provided software to allow collection of hundreds of images and their transfer to a server. Railvision have also performed methods for automated classification of pest and disease features using image analysis methods. CertisUK have provided commercial glasshouse sites for testing and information to help the design and use of the system.
Impact The project is multidisciplinary, involving biologists (plant pathologists), engineers, image analysis and computing experts. Several presentations made to the public or professional audiences on the broader subject of pest and disease detection for smart crop protection, have included information obtained in the project. So far, one refereed paper has been published that included some information from the project, although it also covered other aspects of disease detection. Other publications are in preparation. Rothamsted's role has already provided feedback to the lead partner, Railvision, to identify issues with data communication and storage, which has helped them to improve methods applied to other areas of Railvision's work.
Start Year 2014
 
Description Agrivision Inspector Project 
Organisation Rail Vision Europe Ltd.
Country United Kingdom 
Sector Private 
PI Contribution Rothamsted has conducted experiments to inoculate tomato plants with different diseases and pests in order to test the best imaging methods for their detection and automated classification in collaboration with Railvision and CertisUK. The work has provided training on working with tomato diseases and pest for Rothamsted staff, Prof Jon west and Dr Sarah Perryman, and has provided training to Dr Perryman on bioImaging techniques and image processing.
Collaborator Contribution Railvision have provided hardware for imaging and lighting plants and also provided software to allow collection of hundreds of images and their transfer to a server. Railvision have also performed methods for automated classification of pest and disease features using image analysis methods. CertisUK have provided commercial glasshouse sites for testing and information to help the design and use of the system.
Impact The project is multidisciplinary, involving biologists (plant pathologists), engineers, image analysis and computing experts. Several presentations made to the public or professional audiences on the broader subject of pest and disease detection for smart crop protection, have included information obtained in the project. So far, one refereed paper has been published that included some information from the project, although it also covered other aspects of disease detection. Other publications are in preparation. Rothamsted's role has already provided feedback to the lead partner, Railvision, to identify issues with data communication and storage, which has helped them to improve methods applied to other areas of Railvision's work.
Start Year 2014
 
Description Feasibility of using 3D MultiSpectral Imaging for enhanced classification of fruit ripeness and disease - InnovateUK project 
Organisation Fotenix
Country United Kingdom 
Sector Private 
PI Contribution Growth of tomato and strawberry plants. Inoculation with pathogens (Botrytis cinerea, Oidium neolycopersici) and assessment of symptom development against time in combination with optical reflectance data obtained by industry partners (Fotenix and CHAP).
Collaborator Contribution Collection and analysis of image data.
Impact none yet
Start Year 2019
 
Description Smart detection of airborne diseases in protected crops Waitros CTP PhD studentship 
Organisation University of Warwick
Department Warwick Crop Centre
Country United Kingdom 
Sector Academic/University 
PI Contribution Supervision and provision of lab and plant growth facilities for PhD student, jointly with the Warwick Crop Centre and APS produce Ltd.
Collaborator Contribution Provision of access to commercial glasshouses (APS Produce Ltd), provision of seeds and diseased plant material and supervision; Provision of research glasshouse (Warwick crop centre) and isolates of fungal pathogens and supervision.
Impact conference presentations
Start Year 2018
 
Description 21/Apr/2021 Presented and was a panellist at the Waitrose PhD student career conference 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact 21/Apr/2021 Gave a presentation on working with industry as an academic and was a panellist at the Waitrose PhD student career conference
Year(s) Of Engagement Activity 2021
 
Description 5th International Symposium on Fusarium Head Blight, Florianopolis, 6-10 April 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Keynote presentation made on the subject: Technologies for pathogen and disease detection
Year(s) Of Engagement Activity 2016
URL https://www.youtube.com/watch?v=8EKERDyk2uk
 
Description AAB precision Agriculture conference October 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Jon West made an invited keynote presentation at the conference and this was associated with the following publication: West JS, Canning GGM, Perryman SA, Kaczmarek A, King KM (2017) Precision disease detection systems for agriculture and horticulture. Aspects of Applied Biology: Precision systems in agricultural and horticultural production v135, p1-8 (2017).
Year(s) Of Engagement Activity 2017
 
Description Agritec East pollenator talk 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Presentation made to a group of 50 stakeholders on the subject: the need for farmers' perspectives in applied research
Year(s) Of Engagement Activity 2016
URL https://www.cambridgenetwork.co.uk/events/agri-tech-east-pollinator-pipettes-and-ploughs-taking-the-...
 
Description Agrosavia Colombia 2019 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Agrosavia, Colombia invited me to make two keynote presentations in Villavicencio, Colombia - one primarily for farmers and one for scientists to help with the management and control of diseases of rubber "Seminario Científico Internacional: ALERTA temprana para el manejo de plagas y enfermedades en Caucho"
I also gave some training on air sampling techniques.
Year(s) Of Engagement Activity 2019
 
Description Airborne Pest & Disease Surveillance Workshop, Adelaide, Australia 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Jon West gave a Keynote presentation at the Workshop on Airborne Pest & Disease Surveillance and also chaired a session on day 2.
My expertise was sought by Australia to help them identify current and future methods for pest and disease surveillance.
Year(s) Of Engagement Activity 2016
 
Description Association of Applied Biology Conference Oct 2021 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Jon West gave a 20-minute presentation and there is also an abstract in the conference proceedings. West JS, Canning GGM, King KM, Singh M, Myers T, Kennedy R (2021) Smart monitoring of airborne plant pathogens to enhance crop protection decisions. Crop production with reduced pesticide and fertiliser inputs without compromising yield and quality. Virtual meeting: October 13th-14th 2021. Association of Applied Biologists, Wellesbourne.
Year(s) Of Engagement Activity 2021
 
Description BCPC Diseases Review, 7 December, NIAB innovation farm 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Jon West co-organised this conference as secretary of the BCPC diseases working group.
Year(s) Of Engagement Activity 2016
URL http://www.bcpc.org/expert-groups/diseases
 
Description British Science Association Festival, 6 September 2016, Swansea 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Jon West made a presentation on Farming in the future: Part 1 - Innovations in crop protection
Year(s) Of Engagement Activity 2016
URL https://www.britishscienceassociation.org/british-science-festival
 
Description Crop Protection Association/Fresh Produce Consortium annual conference, Peterborough 23 January 2018 (invited presentation) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Invited presentation about Advances in pathogen detection, monitoring and forecasting.
Year(s) Of Engagement Activity 2018
URL https://freshproduce.org.uk/media/694099/Briefing-Document-23-Jan.pdf
 
Description Cropworld Global, Amsterdam, 25 october 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Jon West made a presentation on Precision Agriculture for Enhanced Crop Protection
Year(s) Of Engagement Activity 2016
URL http://cropworld2016.conference-websites.co.uk/files/2016/01/Cropworld-Exhibitor-Brochure-Last.pdf
 
Description GCRF Africa Workshops June 2020 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact A series of talks online, organizes by the UK KTN to brief people on upcoming funding opportunities from GCRF, Agritech Catalyst and UK KTP.
Jon West gave a pitch in the East Africa and Southern Africa sessions. This led to an application to Agritech Catalyst round 10.
Example 2-minute video pitch is 15 mins 35 seconds into this YouTube link: https://www.bing.com/videos/search?q=YouTube+UK+short+pitches+GCRF+Africa&docid=608032572298429384|=821100F0382FEAA133C3821100F0382FEAA133C3&view=detail&FORM=VIRE
Year(s) Of Engagement Activity 2020
URL https://www.bing.com/videos/search?q=YouTube+UK+short+pitches+GCRF+Africa&docid=608032572298429384&m...
 
Description Intelligent Sensing Systems for Early Detection of Animal and Plant Health Threats Workshop, Feb 2017 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact BBRSC organised a workshop to bring together key experts with knowledge and techniques of potential for surveillance and monitoring of plant and animal health threats. The workshop included some short talks, discussions and break-out groups. It was attended by over 100 scientists and potential funders. Outcomes included an invitation to co-organise a related conference (Sensor 100, which was in July 2018), ideas for potential new projects and collaborators were also made. Feedback was provided to the BBSRC.
Year(s) Of Engagement Activity 2017
 
Description New IPM Conference, Swansea 12-14 September 2022 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact 12-14 Sept 2022 Chaired the opening session at the New IPM Conference and presented on surveillance and monitoring methods to detect crop diseases, Swansea 12-14 September 2022
Year(s) Of Engagement Activity 2022
URL https://www.swansea.ac.uk/business-and-industry/linc/integrated-pest-management/
 
Description • 22/Jun/2021. Invited seminar: Halting crop disease epidemics before they start. University of Hertfordshire, School of Life & Medical Sciences, Virtual Research Conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact 22/Jun/2021. Invited seminar: Halting crop disease epidemics before they start. University of Hertfordshire, School of Life & Medical Sciences, Virtual Research Conference
Year(s) Of Engagement Activity 2021