BioCam - Mapping of Benthic Biology, Geology and Ecology with Essential Ocean Variables

Lead Research Organisation: University of Southampton
Department Name: Faculty of Engineering & the Environment

Abstract

The UK and international governments need to understand the effects of human activities on deep-sea ecosystems in order to make well informed decisions on how to protect and preserve them for future generations. In particular, deep-water coral reefs can be thought of a hotspot for biological activity, providing a rich habitat for diverse communities of marine life at depths of up to ~2000 m. At the same time deep-sea corals grow slowly, taking several thousands of years to form these reefs, are fragile and extremely sensitive to changes in the environment. As such, failure to protect them from trawling and industrial activities may have irreversible effects not only on the distribution of coral, but also on the distribution of the marine communities they support.

However, monitoring of live coral distribution is expensive and time consuming. The main reason for this is that while deep-water corals are sparsely distributed over areas the size of a large city, the features that need to be observed to identify them reliably are on the scale of centimetres. Furthermore, colour is often used to tell apart healthy live coral from dead coral, which means that direct visual observation using underwater cameras is often necessary. Unfortunately, taking images underwater requires vehicles equipped with cameras and powerful lighting systems to operate within one or two meters of the seafloor. Since vehicles operating this close to the seafloor need to avoid obstacles, they can only travel at about a fifth of the speed people walk. In addition to these limitations, while an image can tell us if live coral is present or not, more advanced 3D imaging methods are needed to also tell us how much coral there is, which is important to know when monitoring changes in their abundance.

In this project, we will develop a 3D underwater camera system that is capable of measuring the distribution of live coral on the seafloor over areas that are more than 50 times larger than is currently possible. The system will use a pair of highly sensitivity cameras, a powerful flashed lighting system and a pulsed laser to obtain full colour images and high resolution 3D shape information at a range of almost 10 m (~5 times higher than is typical) from the seafloor. Being able to take high-resolution images from further away will increase the area that can be observed in a single frame and also allow underwater vehicles to operate at safer altitudes and so travel significantly faster than previously possible. Furthermore, images obtained by the 3D camera system will be processed to generate large 3D image landscapes that cover areas of several hundreds of hectares (1 hectare = 10,000m2). These landscapes will each consist of several hundreds of thousands of images of the seafloor obtained over several days using an underwater vehicle. The landscapes will provide scientists with a rich, explorable computer generated reconstruction of deep-sea environments that they can use to visualise and study patterns in the distribution of live coral that would not be immediately obvious in a folder consisting of several hundreds of thousands of raw image frames. Furthermore, the reconstructions will be made compatible with existing web-based interfaces that will potentially allow scientists from all over the world to directly identify live coral and make measurements of their size and distribution over the internet.

The technology developed in this project will allow live coral distributions to be studied over spatial scales that were not previously possible. By revisiting sites over several years, the high-resolution data obtained by the proposed sensor will help facilitate a better understand of the changes that take place on the seafloor, and allow our governments to make better informed decisions regarding the best strategies to preserve and protect these habitats without unnecessarily compromising commercial and industrial activities in the ocean.

Planned Impact

This research will develop a compact, low power 3D imaging system that is capable of generating high-resolution digital reconstructions of underwater scenes. The representation of underwater scenes offered by commercial underwater camera systems is limited to the footprint of a single image frame. However, the extent visible in a single image frame is often not well matched to the task of observing or inspecting natural scenes or artificial structures since it yields little insight into features and patterns that exist on broader scales. The advantage of the system proposed in this work is that it will deliver digital 3D reconstructions of underwater scenes composed of several thousands of images or more, as a packaged data product. The use of standard data and metadata formats will also allow organisations to share data when necessary, and will allow features covering the full range of spatial scales observed to be seamlessly visualised, explored and interrogated by its users using existing software infrastructures. The capabilities offered will allow users to optimise the way in which observation and inspection tasks are performed as well as provide an opportunity to consider the most effective way to archive and maintain data.

The sensor hardware will have integrated lighting control and data storage, and will use standard communication protocols that will make it possible to integrate the device into existing platforms with minimal disruption. Furthermore, the capabilities offered by the device can fulfil the role played by standard underwater cameras, which means that it has a potential market of several thousands of underwater platforms, including low-cost, compact (human deployable) remotely operated vehicles (ROVs). The application areas for such systems span natural science, mineral resource prospecting, environmental monitoring, mapping of debris and plastics, and industrial inspection. The main non-academic beneficiaries of such a system are commercial and government marine conservation groups and industrial groups involved with the installation, maintenance and decommissioning of artificial structures in freshwater and oceanic environments.

With regards to seafloor monitoring, long range autonomous underwater vehicles (AUVs) equipped with the 3D imaging system developed in this work can provide a cost effective solution for sustained monitoring of the seafloor. In particular, the high-level of autonomy can offer a significant reduction in cost through reduced reliance on crewed research vessels. The availability of high-resolution imagery poses advantages from an operational perspective since it can improve navigational estimates and so improve the localisation accuracy of specific features of significance. This capability can be extended to improve position estimates in real-time to increase the repeatability of seafloor observations by allowing specific areas to be revisited for more efficient temporal monitoring.

With regard to inspection of simple structures (e.g. pipelines, damns), pre-inspection planning surveys and the inspection of decommissioned sites, these are operationally comparable to monitoring surveys. These can benefit from the 3D reconstruction capabilities developed in this proposal. However, the economic pressures and risks associated with inspecting more complex structures (e.g. rigging, plants, hulls) demands low-cost, highly manoeuvrable platforms that can operate near their targets, such as compact, human deployable ROVs. For such systems, the relative localisation (i.e. knowing where the platform is with respect to the target) is a major operational bottleneck that makes complete inspection a time consuming task. Extension of the mapping capabilities described in this proposal to enable real-time generation of 3D reconstructions would allow ROV operators with an approximate knowledge of the targeted structure's shape to perform more efficient and complete inspections.
 
Description Invitation to present at buisiness tradeshows such as Oceanology International. Discussions with ChuraUmi aquarium about developing interactive displays in an aquarium. Formed the basis of technology deployed during the Adaptive Robotics expedition, which has led to widespread media coverage and engagement.
Sector Other
Impact Types Societal

 
Description Automated benthic understanding with multimodal observations - International Co Investigator
Amount $0 (AUD)
Organisation Australian Research Council 
Sector Public
Country Australia
Start  
 
Description In-situ Vent Analysis Divebot for Exobiology Research (InVADER), NASA - International Collaborator
Amount $0 (USD)
Funding ID 80NSSC18K1651 
Organisation National Aeronautics and Space Administration (NASA) 
Sector Public
Country United States
Start 07/2018 
End 06/2022
 
Description Mapping in the Background: Scalable capabilities using low-cost passive robotic systems for seafloor imaging
Amount £622,071 (GBP)
Funding ID EP/S001182/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 06/2018 
End 06/2021
 
Description Southampton and Tokyo collaboration in Marine Robotics and Coral Mapping
Amount £7,000 (GBP)
Funding ID 11871/12664 
Organisation Daiwa Anglo-Japanese Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 08/2017 
End 07/2018
 
Title Data collected from marine surveys uploaded to Squidle+ 
Description 3 datasets were uploaded to Squidle+, a seafloor imaging and annotation public database 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? Yes  
Impact The dataset was demonstrated at an event held in Tokyo, Japan by a collaborative group to an audience of 100+ people, including policy makers and marine ecologists. 
URL http://130.211.155.120/geodata/explore#map
 
Description National Oceanography Centre 
Organisation National Oceanography Centre
Country United Kingdom 
Sector Academic/University 
PI Contribution The project involves engineers from the University of Southampton working with marine ecologists from the National Oceanography Centre. The University of Southampton is developing the technology that will be applied to meet the goals of the team from the National Oceanography Centre, who will use the data to draw new insights into the ecology of UK Marine Protected Areas.
Collaborator Contribution The National Oceanography Centre will set out the objectives of missions performed by Autonomous Underwater Vehicles using the technology developed in this research. The performance metrology will be defined and analyses will be performed on optimal strategies for data collection.
Impact Documents have been produced to define the observational strategy for ocean-expeditions that will be carried out during this project.
Start Year 2017
 
Description The University of Sydney 
Organisation University of Sydney
Department Australian Centre for Field Robotics
PI Contribution The University of Southampton is developing a long-range 3D imaging system for use on Autonomous Underwater Vehicles. The University of Sydney has a long history of successful research in 3D imaging. The goals of the project and skill sets of the 2 groups are mutually complementary. Data has been collected from coral reefs off Sesoko, which both groups have contributed to data collection and data processing.
Collaborator Contribution The University of Sydney has a long history of successful research in 3D imaging. The goals of the project and skill sets of the 2 groups are mutually complementary. Data has been collected from coral reefs off Sesoko, which both groups have contributed to data collection and data processing. In particular, the University of Sydney has provided access to data processing pipelines and know-how regarding robust software for real-time data collection.
Impact Data has been collected for coral reefs off Sesoko which has produced high resolution 3D visual reconstructions of vast underwater scenes
Start Year 2017
 
Description The University of Tokyo 
Organisation University of Tokyo
Department Institute of Industrial Science
Country Japan 
Sector Academic/University 
PI Contribution The University of Southampton is developing a long-range 3D imaging system for use on Autonomous Underwater Vehicles. The University of Tokyo has a long history of successful research in visual mapping using underwater robots.. The goals of the project and skill sets of the 2 groups are mutually complementary. Data has been collected from coral reefs off Sesoko, which both groups have contributed to data collection and data processing. This has also lead to a collaboration grant between these groups funded by the Daiwa Anglo-Japanese foundation.
Collaborator Contribution The University of Tokyo provided platforms on which to mount 3D imaging systems and time on board research vessels to monitor coral reefs off Sesoko.
Impact Data has been collected and high-resolution 3D maps of coral reefs have been generated. Seminars have been given both in the UK and Japan to promote the respective projects.
Start Year 2017
 
Description 62nd Underwater and Seafloor Technology Forum 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited talk given at the Universit of Tokyo, Japan to an audience of ~200, including 2 philanthropic organisations.
Year(s) Of Engagement Activity 2018
URL https://www.iis.u-tokyo.ac.jp/ja/news/2984/
 
Description ABC news 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.crossroadstoday.com/story/38991421/artificial-intelligence-guides-rapid-data-driven-expl...
 
Description Article to publicise Seatechnology cover article 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article to publicise the cover article of SeaTechnology magazine
Year(s) Of Engagement Activity 2019
URL https://www.southampton.ac.uk/smmi/news/2019/01/17-adaptive-robotics-expedition-features-on-cover-of...
 
Description Azo Robotics Article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.azorobotics.com/News.aspx?newsID=10037
 
Description Beyond Challenger: a new age of deep-sea science and exploration 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited presentation was delivered at the Royal Society, London entitled "Whole site multi-resolution photogrammetric surveys of deep-sea vents and cold seeps".
Year(s) Of Engagement Activity 2018
URL https://royalsociety.org/science-events-and-lectures/2018/11/beyond-challenger/
 
Description Cover article of Seatechnology Magazine 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Cover article describing the Adaptive Robotics expedition associated with this grant.
Year(s) Of Engagement Activity 2018
URL https://sea-technology.com/sea-technology-vol-59-no-12-december-2018
 
Description EPSRC RAS article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.ukras.org/news/new-opportunities-for-phd-and-postdoc-level-available
 
Description Eco Magazine Alert 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.ecomagazine.com/news/science/artificial-intelligence-guides-rapid-data-driven-exploratio...
 
Description Eco Magazine Article 2: Underwater Paradigm Shift 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.mydigitalpublication.com/publication/?i=543413&ver=html5&p=1
 
Description Eureka Alert! Article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.eurekalert.org/pub_releases/2018-08/soi-aig082918.php
 
Description IIS UTokyo media article 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact International article to publicise the press release relating to the Adaptive Robotics expedition associated with this grant.
Year(s) Of Engagement Activity 2018
URL https://www.iis.u-tokyo.ac.jp/en/news/2954/
 
Description Invited Talk at Gardline Ltd. "Science Cafe" 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Invited talk given at a marine survey company engaged in seafloor survey activities
Year(s) Of Engagement Activity 2019
 
Description Laboratory Equipment Article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.laboratoryequipment.com/news/2018/08/ai-guides-data-driven-exploration-underwater-habita...
 
Description Marine technology news 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering the Adaptive Robotics expedition based on press release
Year(s) Of Engagement Activity 2018
URL https://www.marinetechnologynews.com/news/guides-rapid-driven-exploration-564127
 
Description NBC magazine article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering the Adaptive Robotics expedition based on press release
Year(s) Of Engagement Activity 2018
URL https://www.nbc-2.com/story/38991421/artificial-intelligence-guides-rapid-data-driven-exploration-of...
 
Description News on 6 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL http://www.newson6.com/story/38991421/artificial-intelligence-guides-rapid-data-driven-exploration-o...
 
Description Ocean News Article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL http://cdn.coverstand.com/9767/531875/d006a05195e1a05b4a029d46e397de0cd7b1f4cb.4.pdf
 
Description Oceanology International 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk about the next 50 years of marine robotics given at the "Celebrating 50 years of Oi" held in San Diego, USA. Talk was attended by people from Google, NASA JPL and NASA Lyndon B. Johnson Space Center. Sparked discussion and led to visits to Scripps Ocean Institute and Honeybee Robotics.
Year(s) Of Engagement Activity 2019
URL https://www.oceanologyinternationalamericas.com/Conference/50-years-celebration/
 
Description Oceanology International 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited talk at a major ocean trade show to introduce latest technology in seafloor mapping
Year(s) Of Engagement Activity 2018
URL http://www.oceanologyinternational.com/en/Sessions/51851/Catch-the-Next-Wave-Frontiers-of-Exploratio...
 
Description Phys Org Article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://phys.org/news/2018-08-artificial-intelligence-rapid-data-driven-exploration.html
 
Description Portland Business Journal 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.bizjournals.com/portland/cotm
 
Description Press release - Adaptive Robotics 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Press release about an expedition related to this project that resulted in the generation of the largest known 3d visual reconstruction of the seafloor.
Year(s) Of Engagement Activity 2018
URL https://schmidtocean.org/artificial-intelligence-guides-rapid-data-driven-exploration/
 
Description Scaling capabilities in maritime robotics 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Article "Scaling capabilities in maritime robotics" describes activities of this project.
Year(s) Of Engagement Activity 2019
 
Description Schmidt Ocean Institute Event at the Exploratorium 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Supporters
Results and Impact Invited talk and demo stand at a private function organised by the Schmidt Ocean Institute
Year(s) Of Engagement Activity 2018
 
Description Science Daily Article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.sciencedaily.com/releases/2018/08/180830095358.htm
 
Description Ship to Shore August 15th 2018 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Media (as a channel to the public)
Results and Impact Live broadcast to the Smithsonian museum, USA from the RV Falkor on the Adaptive Robotics expedition related to this project.
Year(s) Of Engagement Activity 2018
URL https://www.si.edu/Events/Calendar/?trumbaEmbed=date%3D20180815#/?i=1
 
Description Ship to shore 9th August 2018 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Live broadcast to the Smithsonian museum, USA from the RV Falkor on the Adaptive Robotics expedition related to this project.
Year(s) Of Engagement Activity 2018
URL https://www.si.edu/Events/Calendar/?trumbaEmbed=date%3D20180809#/?i=1
 
Description Small Piece 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Live broadcast to the University of Southampton hosted Small Piece summer school for secondary school students from the RV Falkor on the Adaptive Robotics expedition related to this project.
Year(s) Of Engagement Activity 2018
 
Description Subsea World Article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://subseaworldnews.com/2018/09/04/ai-leads-data-driven-survey-of-changing-underwater-habitats/
 
Description Technology networks 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL https://www.technologynetworks.com/informatics/news/ai-helps-create-one-of-the-largest-maps-of-the-s...
 
Description Tunisiesoir Article 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Article covering activities of the Adaptive Robotics expedition associated with this project.
Year(s) Of Engagement Activity 2018
URL http://www.tunisiesoir.com/science/research-mapped-onto-one-of-the-worlds-largest-multiresolution-3d...
 
Description UK and Japan Collaboration in Marine Robotics and Coral Mapping 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Article on UK and Japan Collaboration in Marine Robotics and Coral Mapping describes activities of the NERC Oceanids program and this project.
Year(s) Of Engagement Activity 2017
URL http://ieeeoes.org/userfiles/files/September-2017-OES-Beacon.pdf
 
Description Youtube videos on Adaptive Robotics 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Youtube videos uploaded during the Adaptive Robotics expedition, searchable via #Adaptive Robotics
Year(s) Of Engagement Activity 2018
URL https://www.youtube.com/watch?v=uQVnabh45u4