Towards net zero: development of AI enabled biological observing
Lead Research Organisation:
University of Plymouth
Department Name: Sch of Biological and Marine Sciences
Abstract
Sustainable management of the marine environment is a global concern, perhaps best highlighted by the UN declaration of 2021-2030 as the Decade of Ocean Science for Sustainable Development. Ecological understanding of the non-coastal marine environment has lagged behind the physical, chemical and geological understanding due to challenges in observing and monitoring marine life at depth and in open ocean environments. Modern seafloor survey and monitoring platforms, including Autonomous Underwater Vehicles (AUVs), Remotely Operated Vehicles (ROVs), and Autonomous Landers, are able to collect an array of spatially and temporally explicit, multi-sensor data, including vast video and / or image datasets, offering either high or large spatially and temporally resolved datasets. While use of these platforms, and their ability to make concurrent visual and environmental observation have already transformed our understanding of marine ecosystems, particularly hard substrate systems like seamounts and hydrothermal vents, the full potential of these autonomous and robotic systems has not yet been realised. One of the greatest challenges to realising that potential lies in overcoming the bottleneck created by the need for manual (human) interpretation of images and video in order to extract quantitative biological data. Recently developments in artificial intelligence and computer vision have offered a potential mechanism to overcome that bottleneck, offering a faster, more consistent, cost effective and shareable alternative to manual annotation. We have established that deep learning (a branch of artificial intelligence) can be used to reliably and quickly count specific species in the right conditions. This capacity needs to be expanded to a wider selection of taxa and pipelines developed that can be applied in-situ, moving us toward a future of AI enabled biological observing. Realising this future is important to reducing the carbon footprint of marine biological research, and helping us achieve our climate change targets. In this project we will investigate the best methods to translate the large volume of data collected by autonomous and robotic systems, into ecological knowledge to then feed into models enabling us to make predictions on how biodiversity is distributed and may change over time. This will drastically improve our perception of the oceans ecology and better inform conservation and management measures.
| Description | Our primary achievements from this award are that we have advanced our understanding of how to use AI in benthic ecological observing. Specifically we have learned that variation in annotations groupings (animal, phylum or morphology) has little impact on AI model performance, despite large differences in class numbers. Consequently, the decision for annotators should hinge on whether to invest effort in detailed annotation at the beginning of a project or to perform finer sorting of model predictions at the end. Further we have learned that little is gained in terms of AI model performance by using polygons vs tight boxes in annotation. As polygons take a considerable time to draw, annotators should consider carefully how best to invest their time. Our award objectives were largely met with some changes to our proposed approach. Due to a change in personnel on the project we did not achieve objective 1 - to develop novel methods to determine the best way to group images into classes that provide the best performance of automated classifier. However, objectives 2 and 3 were fully met. We used different levels of animal classification, ranging from one class of "biota" to 38 classes of morphological types, to train a model able to accurately annotate a large dataset of images of benthic ecosystem. We had originally proposed to use AUV footage but used ROV not AUV footage. 3. Test the performances of this new classifier with different model architecture, particularly the lightest alternatives to study the potential for in-situ (autonomous) use - met How might the findings be taken forward and by whom? 1. Develop novel methods to determine the best way to group images into classes that provide the best performance of automated classifier. 2. Use these new classes to train a model able to accurately annotate a large dataset of ~60K images of benthic ecosystem collected by an AUV as a proof of concept. 3. Test the performances of this new classifier with different model architecture, particularly the lightest alternatives to study the potential for in-situ (autonomous) use. |
| Exploitation Route | We have already taken this work forward into new DEFRA and NERC funded projects, however further work is needed in development of training libraries and ways to improve model performance. |
| Sectors | Aerospace Defence and Marine Agriculture Food and Drink Construction Digital/Communication/Information Technologies (including Software) Education Energy Environment Government Democracy and Justice |
| Description | Application of Deep Learning Artificial Intelligence to Automate Seabed Imagery Processing |
| Amount | £107,280 (GBP) |
| Funding ID | RDE553 |
| Organisation | Department For Environment, Food And Rural Affairs (DEFRA) |
| Sector | Public |
| Country | United Kingdom |
| Start | 03/2024 |
| End | 02/2025 |
| Description | DEAL - DEcentrAlised Learning for automated image analysis and biodiversity monitoring |
| Amount | £745,324 (GBP) |
| Funding ID | UKRI041 |
| Organisation | Natural Environment Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 02/2024 |
| End | 01/2027 |
| Description | Collaborative AI research with Plymouth Marine Laboratory |
| Organisation | Plymouth Marine Laboratory |
| Department | NERC Earth Observation Data Acquisition and Analysis Service (NEODAAS) |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | We provided a training dataset for deep-learning AI in correct format. |
| Collaborator Contribution | NEODAS AI researchers used our data to train an AI model for detecting benthic animals |
| Impact | new joint funding proposals for AI work between University of Plymouth and PML |
| Start Year | 2023 |
