16AGRITECHCAT5: GrassVision: Automated application of herbicides to broad-leaf weeds in grass crops

Lead Research Organisation: University of the West of England
Department Name: Bristol Robotics Laboratory

Abstract

The aim of the project is to develop a novel spray apparatus for precision application of herbicides to broad-leaf weeds in
grass crops. Such a system, using a spray boom covering a 10x.5m area of ground, could feasibly run at upwards of 1m/s,
allowing precision spraying of weeds at between 1 and 2 hectares/hr.

The consortium is comprised of three partners; (1) Centre for Machine Vision (CMV), a leading research centre in 3D
machine vision, with past success in the agri-tech field, (2) Aralia Systems Ltd (AS), an international security company with
a wide R&D portfolio in data-mining and complex feature analysis on videos, (3) Soil Essentials Ltd (SE), a leading
precision agriculture company specialising in GPS machinery guidance, implement control, cloud based decision support
systems and emerging grassland agronomy technologies.

The primary focus will be to detect weeds in grass such as dock and ragwort using novel 3D machine vision techniques. Initially the
project will use off-the-shelf machinery to spray an area of roughly 50x50cm around each weed, activating the relevant
nozzles on the boom with an estimated aimed decrease in herbicide use of around 75%.The project will then look to
determine the limits of precision by refining the boom itself, allowing nozzles to move linearly across the boom as on an
inkjet printer, or in rotation using servo motors. Using this approach, we aim to provide potential reductions in herbicide use
in excess of 90%.

The role of the CMV team will be to realise novel 3D imaging hardware and software for detecting the weeds in the grass
while moving on a tractor. We also aim to recognise the weed species. This data will be used to both direct the automated
weeding system in real-time (developed by SE) and to create a detailed weed data map (developed by AS) of the entire
field.

Technical Summary

The aim of the project is to develop a novel spray apparatus for precision application of herbicides to broad-leaf weeds in
grass crops.

The consortium is comprised of three partners; (1) Centre for Machine Vision (CMV), a leading research centre in 3D
machine vision, with past success in the agri-tech field, (2) Aralia Systems Ltd (AS), an international security company with
a wide R&D portfolio in data-mining and complex feature analysis on videos, (3) Soil Essentials Ltd (SE), a leading
precision agriculture company specialising in GPS machinery guidance, implement control, cloud based decision support
systems and emerging grassland agronomy technologies.

The primary focus will be to detect weeds in grass such as dock and ragwort using novel 3D machine vision techniques. The
project will use off-the-shelf machinery to spray an area around each weed, activating the relevant
nozzles on the boom.

The role of the CMV team will be to realise novel 3D imaging hardware and software for detecting the weeds in the grass
while moving on a tractor. We also aim to recognise the weed species.

Planned Impact

The main impact will be on beef and dairy farmers who will be provided with a tool to control weeds in pasture, thereby
offering increased productivity of grass swards. End-user farmers are expected to see substantial reductions in herbicide
use in excess of 90%.

There are also wider environmental and social benefits. Environmental benefits will accrue from a more efficient, precision
application of herbicides and increased pasture yield. Precision application will reduce the environmental impact and
economic losses from the industry and help to improve feed conversion efficiency from pasture, boosting food security.
Considerable environmental benefits will accrue from a more efficient, precision application of herbicides. Benefits include,
reduced environmental contamination, eg of water sources, ability to treat larger grassland areas, increasing yields
reducing the need for purchasing feed, ability to spray in environmentally sensitive areas where spraying is currently not
possible, reduction of herbicide costs and clover damage. Better management of pasture will allow longer herd life will
reduce the environmental impact and economic losses from the industry and help to improve feed conversion efficiency in
beef and milk production, boosting food security. Benefits will accrue beyond the project as improvements in pasture and
its management grow. The economic and environmental potential of grassland weed control is very high due to the
extensive area grassland covers, the economic loss of untreated weeds and the high cost (both economic and
environmental) of blanket application of herbicide.

Economic Impacts - The development of the technology, data handling, benchmarking systems and pasture weed control
and health monitoring will increase the SME partners' profile in R&D, leading to further opportunities and developments within the
value chain. An increased turnover and profitability following the project will help to increase employment and job security.
 
Description The Grassvision project successfully delivered a proof of concept technical demonstrator capable of identifying broad leaf weeds in pasture using CNNs that triggered a solenoid via a microcontroller to deliver herbicide via a targeted spray.

Some of the non-commercially sensitive key findings have been disseminated in a conference paper:

Lyndon N. Smith, Arlo Byrne, Mark F. Hansen, Wenhao Zhang, and Melvyn L. Smith "Weed classification in grasslands using convolutional neural networks", Proc. SPIE 11139, Applications of Machine Learning doi.org/10.1117/12.2530092

Two high quality student projects have been based around this investigation using the dataset that was created via funding of this project:

1. Weed Classification in Grasslands, with Convolutional Neural Networks. Arlo Byrne (BEng), 2018
2. Automated Weed Detection, Classification and Localisation in Pasture Using Machine Vision and Deep Learning Neural Networks. Kieron Rai (MSc), 2019

The promising results obtained from the funding, led to immediate follow on consultancy work, to build a lower cost implementation that demonstrated the same functionality, as well as a distributed framework for sharing images and trained neural networks in order to update smart camera modules remotely. This was identified as a key requirement for commercialisation of the system and successful delivery of this combined with the initial findings from BB/P005039/1 led to the successful application of a further follow on funding from Innovate UK to deliver an ambitious deep-learning based agri-tech platform (SKAINet) with SoilEssentials and three other commercial partner:
A retrainable, smart-camera, vision system for agriculture - SKAi, the SoilEssentials KORE Artificial Intelligence platform (105154)
Exploitation Route Likely to be exploited by the commercial partners, led by Soil Essentials Ltd. A beta version of the automated weeding system developed via this project and it's follow-on InnovateUK project is scheduled to be launched at Royal Highland Show in June 2020
Sectors Agriculture, Food and Drink

URL http://www.soilessentials.com/sector/research/
 
Description This technology is now being sold under the name of SKAi and will serve to reduce environmental impact through targeted herbicide delivery. There is an urgent agronomic (reducing the amount of plant protection products applied to crops), environmental (pollution reduction), economic (lowering the cost of food production) and political (continuing public pressure for a reduction in ag-chem use) need to modernise and update agrochemical applications to crops. This requires a move from traditional practice of applying a uniform rate of treatments across the whole crop to a much more targeted approach. A demonstrator system that was proven under this 2-year feasibility project and developed on towards a marketable product under the follow-on project [GrassVision 2 - A retainable, smart-camera, vision system for agriculture - SKAi, the Soil Essentials KORE Artificial Intelligence platform, InnovateUK (25989). £891,051(£173,951) has completed in-field trials by Soil Essentials. The follow-on 3-year InnovateUK project, 'GrassVision2', received funding to develop the technology towards a marketable device, 'SKAi', to allow herbicide to be precisely targeted only where needed, (ie only on the weeds), increasing productivity and reducing the environmental impact. SKAi uses a re-trainable smart camera vision system developed at UWE for agriculture, and builds on the InnovateUK feasibility study project of 'GrassVision', which successfully developed a low-cost machine vision system to recognise and precision-apply herbicides to broad leaved weeds in grassland. Soil Essentials is meeting this need by utilising a smart camera and artificial intelligence platform for use by farmers, agronomists and agrochemical applicators. This platform is being integrated into the existing KORE (www.koresolution.com) precision farming platform to extend its functionality to allow the support of in-field smart cameras using image transfer and machine learning. The system can dramatically reduce the total amount of crop protection products applied to crops by ~90%. This project formed a key component of a UWE REF Impact Case Study.
First Year Of Impact 2019
Sector Agriculture, Food and Drink
Impact Types Societal,Economic

 
Description A retrainable, smart-camera, vision system for agriculture - SKAi
Amount £891,051 (GBP)
Funding ID 25989 
Organisation Biotechnology and Biological Sciences Research Council (BBSRC) 
Sector Public
Country United Kingdom
Start 04/2019 
End 03/2022
 
Description the GCRF Agri-tech Catalyst Seeding Award competition.
Amount £142,000 (GBP)
Funding ID GCRF-SA-2020-UWE 
Organisation Biotechnology and Biological Sciences Research Council (BBSRC) 
Sector Public
Country United Kingdom
Start 03/2020 
End 07/2021
 
Title SKAi 
Description New technology and software in the form of a re-trainable smart camera system solution for agriculture, marketed under the brand name of 'SKAi' (pronounced: Sky).Utilises smart cameras, trained in the recognition of target weed species, to control an agricultural crop sprayer as it passes over a field. Initially utilised to target dock infestations in grassland, SKAi is now being put to work in high value arable crops. 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2022 
Impact Reduced level of weed in grasslands. 
URL https://www.soilessentials.com/product/skai-retrainable-camera-system/
 
Description "Applied Machine Vision for the Real World", Invited speaker at The 38th Research and Academic Conference Research and Technology 2020. December 2020 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Mark Hansen gave an invited talk entitled "Applied Machine Vision for the Real World" at The 38th Research and Academic Conference Research and Technology 2020. December 2020
Year(s) Of Engagement Activity 2020
 
Description Grass Vision project was featured on BBC Radio 4 Farming today as part of COP 26 on 3rd November 2021 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Our Grass Vision project was featured on BBC Radio 4's Farming Today program as part of COP 26 on 3rd November 2021. The item talked our technology and the benefits offered and appeared as part of an item on precision farming. Here is a link to the recording: https://www.bbc.co.uk/sounds/play/m00114dc

Following the broadcast a number of related articles appeared in several trade journals.
Year(s) Of Engagement Activity 2021
URL https://www.bbc.co.uk/sounds
 
Description KTN Emerging Imaging Technologies in Agri-Food Workshop in Birmingham 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Wenhao Zhang presented our work on using machine vision / learning in agri-tech at the KTN Emerging Imaging Technologies in Agri-Food Workshop in Birmingham
Year(s) Of Engagement Activity 2018
URL https://ktn-uk.co.uk/news/emerging-imaging-technologies-in-agri-food
 
Description KTN Networking evenet - Robotics and Artificial Intelligence for Agriculture Event, 26th July (London) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact showcasing some Recent successful InnovateUK projects for AgriFood.

Lots of interest in our work.
Year(s) Of Engagement Activity 2018
URL https://ktn-uk.co.uk/events/19061
 
Description N8 Precision Ags and Robotics Doctoral Training Seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact A keynote talk was delivered by Wenhao Zhang at the N8 doctoral training seminar at Manchester on agri-robotics and automation.
Year(s) Of Engagement Activity 2018
URL https://www.n8research.org.uk/
 
Description Virtual presentation to UK-RAS Strategic Task group in Agri-Robotics 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Melvyn Smith to give a virtual presentation to UK-RAS Strategic Task group in Agri-Robotics that included our work on automated weed detection, cattle condition monitoring and pig face recognition and expression detection on 29th Sept.
Year(s) Of Engagement Activity 2020
 
Description White paper - Agricultural Robotics: The Future of Robotic Agriculture 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact UK-RAS Network White Papers, ISSN 2398-4414
Year(s) Of Engagement Activity 2018
URL https://arxiv.org/abs/1806.06762