13TSB_AgriFood Developing a vision system to enhance phenotyping in apples (Pomevision)

Lead Research Organisation: National Institute of Agricultural Botany
Department Name: Centre for Research

Abstract

The UK is not self-sufficient in apples, even during the cropping season, providing only one third of our own consumption
with the shortfall made up by imports. A large proportion of this is due to our inability to meet the stringent specification set
by supermarkets for Class 1 fruit, which represents 80% of sales. Our consortium represents 20% of the apple crop in the
UK; our records show that similar orchards can have outputs that vary by a factor of 2 or 3, with reject fruit going to waste
or low value processing. A significant amount of this variation is down to management practice. Our distribution members
believe that, by standardising best-practice orchard management, and with a strategic approach to breeding new cultivars,
we could gear up our orchards to take back at least 100,000T of the imported volume.
This project is designed to enhance the pace of improvement of quality and output parameters in UK apple orchards
through improving management practices and strategic data capture. For this, we need detailed management information,
which has been sorely lacking due to the labour-intensive and subjective nature of manual checks to date. The project
develops a novel vision-based crop measurement technology based on the convergence between state-of-the-art cost
effective image capture technology (now achievable using consumer grade cameras instead of expensive scientific
instruments) and our new image processing algorithms. These tools will be used to identify and record commercially
relevant phenotype traits in detail. This data can be used for yield and quality prediction and management in season, for
the optimisation of commercial yields across the UK through the transfer of 'best practice' over a longer timescale, and
finally to tie in with the recently completed sequencing of the apple genome, to identify the best markers to more effectively
breed new elite cultivars with the best commercial (as well as biological) traits.
The project objectives include the identification and optimisation of the most commercially effective traits, the development
of an automated in-field system to regularly measure the status and development of these traits in response to stress and
management activities, decision support outputs and the new knowledge required to start the UK strategic development
programme for new types of commercial cultivar.
The consortium is led by business, in the form of Worldwide Fruit, the UK's largest apple producer and supply group, joined
by technology developers who will commercialise the novel platform technology, and supported by East Malling Research,
world experts in pomology and apple cultivar development, who will set out the phenotype maps and disseminate the
academic outputs relating to genetic markers for improved cultivars. The group has the skills to research, develop,
commercialise and exploit the technology in the UK, and will exploit within Europe and/or licence to other manufacturers as
volumes grow. TSB support is the catalyst needed to drive this innovation across the business-academia gap, while the
exposure the scheme brings will help us to penetrate the market more quickly.
Potential benefits to the UK apple growers' industry are: raising production quality (percentage within spec.) to better meet
supermarket size and uniformity specifications, saving up to £40M of wasted product; increasing the UK average cropping
intensity by up to 50% through 'best practice' identification and transfer, growing capacity by a further £50M; and giving the
UK the capability to strategically lead the world in the rapid development of new cultivars, shortening the introduction time by potentially up to 5 years

Technical Summary

The UK is not self sufficient in apples, even during the high season, providing only one third of our own consumption, with the shortfall made up by imports. Our consortium, representing one third of the growers, beieves that, by standardising best-practice orchard management, and with a strategic approach to helping breed new cultivars, we could gear up our orchards to take back at least 100,000T of lost import volume, worth £50M. This project develops a novel vision based crop measurement technology for apple growers, capable of measuring commercially relevant phenotype traits in the field. The detailed data captured by tree over the season and between seasons will be correlated with environmental factors and the apple genome. It will provide growers the opportunity to better manage the quality and yield of their crops, to strategically increase cropping intensity and to inform breeders of the desired routes for the accelerated development of new elite cultivars. This new technology will allow the UK to lead the world in the precision management and development of pome crops, and help increase production capacity by up to 50% from the current acreage.

Planned Impact

Beneficiaries from this research will be primarily the UK apple growers. There are 400 orchards in the UK that could take
up our technology. Our consortium members represent over 30% of the UK crop (70 significant-sized growers). Potential
benefits to the UK apple growers industry are: raising production quality (percentage within spec.) to better meet
supermarket size and uniformity specifications, saving up to £40M of wasted product; increasing the UK average cropping
intensity by up to 50% through 'best practice' identification and transfer, growing capacity by a further £50M; and giving the
UK the capability to strategically lead the world in the rapid development of new cultivars, shortening the introduction time
by potentially up to five years.
UK apple growers will benefit from using the system and knowledge to:
Firstly, improve the management of this year's crop to produce a maximised, more reliable yield of Class 1 fruit to meet the
supermarket specification at highest price. This yield improvement is expected to be up to 25%.
Secondly, have the ability to improve winter pruning activity to optimise the structural / cropping wood ratio and to train
branch angles, in order to increase the yield potential of the tree for next year's crop.
Thirdly, apple breeders will gain benefit by the QTL work carried out here to connect commercially important phenotypic
traits to the apple genome, allowing more focussed breeding programmes. The ability of the system to provide early
prediction of physiological growth characteristics will also enable breeding programmes to produce experimental results
more quickly, speeding the introduction of commercially relevant new varieties.
Fourthly, equipment providers will be able to exploit a novel methodology and hardware system that can capture and
analyse high definition images in very unstructured and variable environments. This will be sellable across tree fruit and
other horticultural applications, as well as being transferable into other sectors such as security.
The wider public will benefit from the provision of more fruit with good organoleptic quality and eating more fruit and
vegetables has been shown to play a key role in fighting obesity, heart disease and diabetes by providing consumers with a
healthy and sustainable diet option. Reduced food miles for 100,000Tonnes of product (transported 400km by road from
European countries) would reduce CO2 emissions by 5,000 tonnes (estimated at 125g CO2 per T.km: AECOM Freight
emissions benchmarking report). The UK need to intensify its cropping, as land for production is not increasing. The
government wants UK horticulture to double its production and become more "Food Secure".

Publications

10 25 50
 
Description This project allowed us to develop key genetic resources (replicated plantings or progenies) and tools (genetic linkage maps) for use in future research into improving apple fruit quality attributes. It also allowed us to carry out the necessary phenotyping work to develop two scientific papers on the genetic control of apple russetting and bitter pit disorder, the first of which will be submitted in the coming months. It is unfortunate that the version of the phenotyping system that we had access to (before final improvements were made) did not work very well on our segregating apple progenies or on small experimental plots. Our trained operator made several suggestions on how the system could be improved. Being part of this project has allowed NIAB EMR researchers to maintain their internationally-important contributions to apple genetics research and it has provided us with the opportunity to exploit and expand our existing genetic and phenotyping resources. New areas of research interest for us, as result of this project, are skin finish/russetting and bitter-pit in apple, imaging to improve harvest forecasts and prediction of fruit quality in soft fruit, and the combination of imaging and modelling of environmental metrics to predict yields.
Exploitation Route The TRC will use the ground-truth data generated by NIAB EMR to further refine the PomeVision system so that the accuracy and precision of apple counts in commercial orchards are are improved. The progenies phenptyped in this project will also be available for future research into developing new elite varieties with improved storage potential that will facilitate the development of low-stringency, more energy-efficient long-term storage strategies.
Sectors Agriculture, Food and Drink,Digital/Communication/Information Technologies (including Software),Education,Electronics,Environment

 
Description Goals, Achievements & Impact 1.Developed image measuring systems based on standard industrial vision camera hardware used in manufacturing to increase the economic value of tree fruit. We have successfully developed a working prototype system using ruggedised industrial hardware. This enabled us to develop an affordable outdoor vision system for real time scanning of apples on trees. This will in turn enable us to develop a commercially viable solution for SME fruit growers & cultivar research & development. 2.Developed a unique image analysis algorithm to count and size fruit on trees from video footage in real time in a complex images of fruits, leaves and other features against complex, high contrast backgrounds in the field. Our solution has met our main objectives, with certain limitations associated with light level extremes, rate of change in light levels, fruit size early in growth phase, and seeing branches in full leaf. Overall, we achieved a step change improvement in state of the art, which has led us close to a commercially viable mark 1 system. 3. Once the developed system can be employed over several seasons, then the research data can be linked in a longitudinal (in time) description of crop performance and quality criteria with the genomic information. This will enable breeders to target their programmes and bring commercially interesting cultivars to market up to five years earlier than otherwise. Our system is the first that will be capable of being used in this way to generate affordable time series big data. This will enable NIAB EMR to meet one of its strategic aims. 4. This project enabled NIAB EMR to carry out the necessary phenotyping work to improve our understanding of the genetic control of apple russetting and bitter pit disorder; two scientific papers will be submitted to peer-reviewed, high quality international journals in the coming months. 5. The key genetic resources (replicated plantings or progenies) and tools (genetic linkage maps) generated in this project will be used to underpin future research into improving apple fruit quality attributes.
First Year Of Impact 2017
Sector Agriculture, Food and Drink,Digital/Communication/Information Technologies (including Software),Education,Electronics,Environment
Impact Types Economic

 
Description Innovate UK Open Competition Round 2
Amount £628,666 (GBP)
Funding ID IUK 103842 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 03/2018 
End 02/2021
 
Description PomeVision IUK 101626 
Organisation Fruition PO Ltd
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution NIAB EMR is the leading tree fruit research organisation in the UK has key expertise in tree fruit physiology, production efficiency and pomology (Dr Mark Else) and linkage mapping (Ms Felicidad Fernandez). NIAB EMR's work covered the definition of appropriate descriptors and linkage mapping, QTL identification, manipulation of trees for trials and pre-harvest assessments of outcomes. NIAB EMR gained new insight into the commercial requirements of their genomic and cultivar work and developed an important resource for future research that will be utilised in new IUK projects to be funded under the ISCF programme.
Collaborator Contribution Worldwide Fruit Ltd (WFL) is the UKs largest, grader, packer, storer, marketer and distributor of UK pome fruits. Tony Harding, Technical Director, will lead the project. Dr Martin Luton and Caroline Ashdown provided technical support to the growers through field trials, and measured crop attributes and volume post-harvest. WFL packhouses provided cost modelling scenarios to show the benefits of higher commercial yields. The on-going collaboration with NIAB EMR and the TRC is bringing new insight to the science behind WFL's operations. Fruition PO Ltd implemented and managed the trials carried out on commercial grower farms and gained a deeper understanding of crop performance and the ability to lead in using digital imaging platforms to forecast harvestable yields. Technology Research Centre (TRC) has a group of technologists & engineers with extensive industrial R&D experience, in device design & prototyping for image processing and robotics technology. Resources include CAD systems, electronics workshop; data log equipment; fabrication & testing workshops. TRC have designed, built tested the image capture vehicle, ran trials, designed the database architecture and GUI, managed the project & exploitation in a JV with Stemmer. Stemmer Imaging develops and provides imaging solutions, notably smart cameras, image acquisition technology and smart vision solutions. This project was an opportunity for Stemmer to increase their participation in the fruit and vegetable sector; Stemmer allocated smart cameras and ancillary vision equipment for the system.
Impact Project outputs and outcomes include: 1) A cost effective image capture and processing techniques for complex unstructured environments; 2) New knowledge on phenotype mapping for commercially relevant traits, stimulating breeder activities; 3) A management decision making toolkit for improved yield and quality for growers; 4) This project enabled NIAB EMR to carry out the necessary phenotyping work to improve our understanding of the genetic control of apple russetting and bitter pit disorder; two scientific papers will be submitted to peer-reviewed, high quality international journals in the coming months. 5). The key genetic resources (replicated plantings or progenies) and tools (genetic linkage maps) generated in this project will be used to underpin future research into improving apple fruit quality attributes, such as the two new ISCF IUK projects to begin in 2017.
Start Year 2013
 
Description PomeVision IUK 101626 
Organisation Stemmer Imaging Ltd
Country United Kingdom 
Sector Private 
PI Contribution NIAB EMR is the leading tree fruit research organisation in the UK has key expertise in tree fruit physiology, production efficiency and pomology (Dr Mark Else) and linkage mapping (Ms Felicidad Fernandez). NIAB EMR's work covered the definition of appropriate descriptors and linkage mapping, QTL identification, manipulation of trees for trials and pre-harvest assessments of outcomes. NIAB EMR gained new insight into the commercial requirements of their genomic and cultivar work and developed an important resource for future research that will be utilised in new IUK projects to be funded under the ISCF programme.
Collaborator Contribution Worldwide Fruit Ltd (WFL) is the UKs largest, grader, packer, storer, marketer and distributor of UK pome fruits. Tony Harding, Technical Director, will lead the project. Dr Martin Luton and Caroline Ashdown provided technical support to the growers through field trials, and measured crop attributes and volume post-harvest. WFL packhouses provided cost modelling scenarios to show the benefits of higher commercial yields. The on-going collaboration with NIAB EMR and the TRC is bringing new insight to the science behind WFL's operations. Fruition PO Ltd implemented and managed the trials carried out on commercial grower farms and gained a deeper understanding of crop performance and the ability to lead in using digital imaging platforms to forecast harvestable yields. Technology Research Centre (TRC) has a group of technologists & engineers with extensive industrial R&D experience, in device design & prototyping for image processing and robotics technology. Resources include CAD systems, electronics workshop; data log equipment; fabrication & testing workshops. TRC have designed, built tested the image capture vehicle, ran trials, designed the database architecture and GUI, managed the project & exploitation in a JV with Stemmer. Stemmer Imaging develops and provides imaging solutions, notably smart cameras, image acquisition technology and smart vision solutions. This project was an opportunity for Stemmer to increase their participation in the fruit and vegetable sector; Stemmer allocated smart cameras and ancillary vision equipment for the system.
Impact Project outputs and outcomes include: 1) A cost effective image capture and processing techniques for complex unstructured environments; 2) New knowledge on phenotype mapping for commercially relevant traits, stimulating breeder activities; 3) A management decision making toolkit for improved yield and quality for growers; 4) This project enabled NIAB EMR to carry out the necessary phenotyping work to improve our understanding of the genetic control of apple russetting and bitter pit disorder; two scientific papers will be submitted to peer-reviewed, high quality international journals in the coming months. 5). The key genetic resources (replicated plantings or progenies) and tools (genetic linkage maps) generated in this project will be used to underpin future research into improving apple fruit quality attributes, such as the two new ISCF IUK projects to begin in 2017.
Start Year 2013
 
Description PomeVision IUK 101626 
Organisation The Technology Research Centre Ltd
Country United Kingdom 
Sector Private 
PI Contribution NIAB EMR is the leading tree fruit research organisation in the UK has key expertise in tree fruit physiology, production efficiency and pomology (Dr Mark Else) and linkage mapping (Ms Felicidad Fernandez). NIAB EMR's work covered the definition of appropriate descriptors and linkage mapping, QTL identification, manipulation of trees for trials and pre-harvest assessments of outcomes. NIAB EMR gained new insight into the commercial requirements of their genomic and cultivar work and developed an important resource for future research that will be utilised in new IUK projects to be funded under the ISCF programme.
Collaborator Contribution Worldwide Fruit Ltd (WFL) is the UKs largest, grader, packer, storer, marketer and distributor of UK pome fruits. Tony Harding, Technical Director, will lead the project. Dr Martin Luton and Caroline Ashdown provided technical support to the growers through field trials, and measured crop attributes and volume post-harvest. WFL packhouses provided cost modelling scenarios to show the benefits of higher commercial yields. The on-going collaboration with NIAB EMR and the TRC is bringing new insight to the science behind WFL's operations. Fruition PO Ltd implemented and managed the trials carried out on commercial grower farms and gained a deeper understanding of crop performance and the ability to lead in using digital imaging platforms to forecast harvestable yields. Technology Research Centre (TRC) has a group of technologists & engineers with extensive industrial R&D experience, in device design & prototyping for image processing and robotics technology. Resources include CAD systems, electronics workshop; data log equipment; fabrication & testing workshops. TRC have designed, built tested the image capture vehicle, ran trials, designed the database architecture and GUI, managed the project & exploitation in a JV with Stemmer. Stemmer Imaging develops and provides imaging solutions, notably smart cameras, image acquisition technology and smart vision solutions. This project was an opportunity for Stemmer to increase their participation in the fruit and vegetable sector; Stemmer allocated smart cameras and ancillary vision equipment for the system.
Impact Project outputs and outcomes include: 1) A cost effective image capture and processing techniques for complex unstructured environments; 2) New knowledge on phenotype mapping for commercially relevant traits, stimulating breeder activities; 3) A management decision making toolkit for improved yield and quality for growers; 4) This project enabled NIAB EMR to carry out the necessary phenotyping work to improve our understanding of the genetic control of apple russetting and bitter pit disorder; two scientific papers will be submitted to peer-reviewed, high quality international journals in the coming months. 5). The key genetic resources (replicated plantings or progenies) and tools (genetic linkage maps) generated in this project will be used to underpin future research into improving apple fruit quality attributes, such as the two new ISCF IUK projects to begin in 2017.
Start Year 2013
 
Description PomeVision IUK 101626 
Organisation Worldwide Fruit plc
Country United Kingdom 
Sector Private 
PI Contribution NIAB EMR is the leading tree fruit research organisation in the UK has key expertise in tree fruit physiology, production efficiency and pomology (Dr Mark Else) and linkage mapping (Ms Felicidad Fernandez). NIAB EMR's work covered the definition of appropriate descriptors and linkage mapping, QTL identification, manipulation of trees for trials and pre-harvest assessments of outcomes. NIAB EMR gained new insight into the commercial requirements of their genomic and cultivar work and developed an important resource for future research that will be utilised in new IUK projects to be funded under the ISCF programme.
Collaborator Contribution Worldwide Fruit Ltd (WFL) is the UKs largest, grader, packer, storer, marketer and distributor of UK pome fruits. Tony Harding, Technical Director, will lead the project. Dr Martin Luton and Caroline Ashdown provided technical support to the growers through field trials, and measured crop attributes and volume post-harvest. WFL packhouses provided cost modelling scenarios to show the benefits of higher commercial yields. The on-going collaboration with NIAB EMR and the TRC is bringing new insight to the science behind WFL's operations. Fruition PO Ltd implemented and managed the trials carried out on commercial grower farms and gained a deeper understanding of crop performance and the ability to lead in using digital imaging platforms to forecast harvestable yields. Technology Research Centre (TRC) has a group of technologists & engineers with extensive industrial R&D experience, in device design & prototyping for image processing and robotics technology. Resources include CAD systems, electronics workshop; data log equipment; fabrication & testing workshops. TRC have designed, built tested the image capture vehicle, ran trials, designed the database architecture and GUI, managed the project & exploitation in a JV with Stemmer. Stemmer Imaging develops and provides imaging solutions, notably smart cameras, image acquisition technology and smart vision solutions. This project was an opportunity for Stemmer to increase their participation in the fruit and vegetable sector; Stemmer allocated smart cameras and ancillary vision equipment for the system.
Impact Project outputs and outcomes include: 1) A cost effective image capture and processing techniques for complex unstructured environments; 2) New knowledge on phenotype mapping for commercially relevant traits, stimulating breeder activities; 3) A management decision making toolkit for improved yield and quality for growers; 4) This project enabled NIAB EMR to carry out the necessary phenotyping work to improve our understanding of the genetic control of apple russetting and bitter pit disorder; two scientific papers will be submitted to peer-reviewed, high quality international journals in the coming months. 5). The key genetic resources (replicated plantings or progenies) and tools (genetic linkage maps) generated in this project will be used to underpin future research into improving apple fruit quality attributes, such as the two new ISCF IUK projects to begin in 2017.
Start Year 2013
 
Description Fruit Focus 2017, 2018, 2019 and 2020 
Form Of Engagement Activity Participation in an open day or visit at my research institution
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact NIAB EMR's research work in precision growing of soft fruit crops was showcased in the Water Efficient Technologies (WET) Centre during Fruit Focus 2017. The WET Centre was officially opened by the Rt Hon. George Eustice, Minister of State at DEFRA. Four formal tours were held throughout the day, and three interviews were given by NIAB EMR staff for local television.
Project outputs from IUK 101623, 102144 and 102640 were presented at the fruit Focus Forum in July 2018, and demonstrated in the WET Centre during x4 sessions during Fruit Focus 2019. Project outputs from IUK 101623, 102144 and 102640 were demonstrated in the WET Centre during x4 sessions during Fruit Focus 2019. Outputs from these research projects were also presented at the on-line Fruit Focus event in 2020.
Year(s) Of Engagement Activity 2017,2018,2019,2020
URL https://www.thewetcentre.com
 
Description PomeVision IUK 101626 Consortium Meetings 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact PomeVision 101626 Kick-off meeting, EMR, 24 January 2014; Q1 meeting, TRC, 14 March 2014; Q2 meeting, EMR, 5 June 2014; Q3 meeting, EMR, 7 September 2014; Q4 meeting, EMR, 4 December 2014; Q5 meeting, EMR, 10 March 2015; Q6 meeting, TRC, 11 June 2015; Q7 meeting, EMR, 7 September 2015; Q8 meeting, TRC, 10 December 2015; Q9 meeting, EMR, 10 March 2015; Q10 meeting, TRC, 15 June 2015; Q11 meeting, EMR, 14 September 2015 Stemmer Imaging; Q12 meeting, 14 December 2015, TRC

In addition, ad-hoc meetings between consortium members are arranged to discuss and resolve any issues identified in the quarterly project meetings.
Year(s) Of Engagement Activity 2013,2014,2015,2016
 
Description PomeVision IUK 101626 Presentations made at National and Internnational conferences and meetings 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Discussion of project aims and objectives with academia / industry and demonstration of the technological approaches being used at:
The EMR Concept Pear orchard. Fruit Focus, EMR, 23 July 2014.
The HAPI Innovation Event 'Unlocking Innovation through Science and Collaboration, Prince Philip House, London, 26 February 2015.
The Institute of Agricultural Engineers' Annual Conference (May 2015).
The AAB Knowledge Exchange Conference 20 June 2015, Lancaster University.
Fruit Focus 2015, 22 July 2015, EMR
The "Innovation in Agri-Tech" event, 12 October 2015, Royal Academy of Engineering, London.
The Delta-T "SPAC" conference, 13 October 2015, Rothamsted Research, Harpenden.
The Plant and Animal Genome Conference (PAGXXIV) - San Diego (Jan 2016)
The 8th International Rosaceous Genomics Conference - Angers (June 2016)
Fruit Focus - NIAB EMR (July 2016)
The ISHS XI International Symposium on Integrating Canopy, Rootstock and Environmental Physiology in Orchard Systems - Bologna (August 2016).
The 3rd international scientific conference, Sustainable Fruit Growing: From Plant to Product" - Riga, Latvia (August 2016)
The Under 40's Grower Conference - South Africa (February 2017)
Year(s) Of Engagement Activity 2014,2015,2016