📣 Help Shape the Future of UKRI's Gateway to Research (GtR)

We're improving UKRI's Gateway to Research and are seeking your input! If you would be interested in being interviewed about the improvements we're making and to have your say about how we can make GtR more user-friendly, impactful, and effective for the Research and Innovation community, please email gateway@ukri.org.

Towards net zero: development of AI enabled biological observing

Lead Research Organisation: University of Plymouth
Department Name: Sch of Biological and Marine Sciences

Abstract

Sustainable management of the marine environment is a global concern, perhaps best highlighted by the UN declaration of 2021-2030 as the Decade of Ocean Science for Sustainable Development. Ecological understanding of the non-coastal marine environment has lagged behind the physical, chemical and geological understanding due to challenges in observing and monitoring marine life at depth and in open ocean environments. Modern seafloor survey and monitoring platforms, including Autonomous Underwater Vehicles (AUVs), Remotely Operated Vehicles (ROVs), and Autonomous Landers, are able to collect an array of spatially and temporally explicit, multi-sensor data, including vast video and / or image datasets, offering either high or large spatially and temporally resolved datasets. While use of these platforms, and their ability to make concurrent visual and environmental observation have already transformed our understanding of marine ecosystems, particularly hard substrate systems like seamounts and hydrothermal vents, the full potential of these autonomous and robotic systems has not yet been realised. One of the greatest challenges to realising that potential lies in overcoming the bottleneck created by the need for manual (human) interpretation of images and video in order to extract quantitative biological data. Recently developments in artificial intelligence and computer vision have offered a potential mechanism to overcome that bottleneck, offering a faster, more consistent, cost effective and shareable alternative to manual annotation. We have established that deep learning (a branch of artificial intelligence) can be used to reliably and quickly count specific species in the right conditions. This capacity needs to be expanded to a wider selection of taxa and pipelines developed that can be applied in-situ, moving us toward a future of AI enabled biological observing. Realising this future is important to reducing the carbon footprint of marine biological research, and helping us achieve our climate change targets. In this project we will investigate the best methods to translate the large volume of data collected by autonomous and robotic systems, into ecological knowledge to then feed into models enabling us to make predictions on how biodiversity is distributed and may change over time. This will drastically improve our perception of the oceans ecology and better inform conservation and management measures.

Publications

10 25 50
 
Description Our primary achievements from this award are that we have advanced our understanding of how to use AI in benthic ecological observing. Specifically we have learned that variation in annotations groupings (animal, phylum or morphology) has little impact on AI model performance, despite large differences in class numbers. Consequently, the decision for annotators should hinge on whether to invest effort in detailed annotation at the beginning of a project or to perform finer sorting of model predictions at the end. Further we have learned that little is gained in terms of AI model performance by using polygons vs tight boxes in annotation. As polygons take a considerable time to draw, annotators should consider carefully how best to invest their time.

Our award objectives were largely met with some changes to our proposed approach. Due to a change in personnel on the project we did not achieve objective 1 - to develop novel methods to determine the best way to group images into classes that provide the best performance of automated classifier. However, objectives 2 and 3 were fully met. We used different levels of animal classification, ranging from one class of "biota" to 38 classes of morphological types, to train a model able to accurately annotate a large dataset of images of benthic ecosystem. We had originally proposed to use AUV footage but used ROV not AUV footage.
3. Test the performances of this new classifier with different model architecture, particularly the lightest alternatives to study the potential for in-situ (autonomous) use - met


How might the findings be taken forward and by whom?


1. Develop novel methods to determine the best way to group images into classes that provide the best performance of automated classifier.
2. Use these new classes to train a model able to accurately annotate a large dataset of ~60K images of benthic ecosystem collected by an AUV as a proof of concept.
3. Test the performances of this new classifier with different model architecture, particularly the lightest alternatives to study the potential for in-situ (autonomous) use.
Exploitation Route We have already taken this work forward into new DEFRA and NERC funded projects, however further work is needed in development of training libraries and ways to improve model performance.
Sectors Aerospace

Defence and Marine

Agriculture

Food and Drink

Construction

Digital/Communication/Information Technologies (including Software)

Education

Energy

Environment

Government

Democracy and Justice

 
Description Application of Deep Learning Artificial Intelligence to Automate Seabed Imagery Processing
Amount £107,280 (GBP)
Funding ID RDE553 
Organisation Department For Environment, Food And Rural Affairs (DEFRA) 
Sector Public
Country United Kingdom
Start 03/2024 
End 02/2025
 
Description DEAL - DEcentrAlised Learning for automated image analysis and biodiversity monitoring
Amount £745,324 (GBP)
Funding ID UKRI041 
Organisation Natural Environment Research Council 
Sector Public
Country United Kingdom
Start 02/2024 
End 01/2027
 
Description Collaborative AI research with Plymouth Marine Laboratory 
Organisation Plymouth Marine Laboratory
Department NERC Earth Observation Data Acquisition and Analysis Service (NEODAAS)
Country United Kingdom 
Sector Academic/University 
PI Contribution We provided a training dataset for deep-learning AI in correct format.
Collaborator Contribution NEODAS AI researchers used our data to train an AI model for detecting benthic animals
Impact new joint funding proposals for AI work between University of Plymouth and PML
Start Year 2023