Mapping in the Background: Scalable capabilities using low-cost passive robotic systems for seafloor imaging

Lead Research Organisation: University of Southampton
Department Name: Faculty of Engineering & the Environment

Abstract

This fellowship will develop a simple approach to collect large volumes of seafloor visual imagery using fleets of low-cost, passively-drifting underwater robots. Based on the concept of the Lagrangian imaging float outlined in Roman et al., 2011 [1], this fellowship will investigate how seafloor visual observations can scale to enable several orders of magnitude larger regions of the seafloor to be surveyed at significantly lower costs and reduced effort than is possible with today's RAI systems. To achieve this, novel methods will be developed to increase the endurance of these platforms from several hours to several weeks, and enable the location of each platform in the fleet to be determined in a globally consistent way. The images obtained will be packaged into 3D visual seafloor reconstructions along the trajectories taken by the platforms as they drift on near bottom currents. This will provide valuable data for a range of scientific studies and statutory monitoring applications. While the trajectories cannot be precisely targeted, sacrificing the ability to follow predefined track lines allows for a significant reduction in power consumption and can eliminate the need for the expensive sensors and acoustic tracking needed for real-time localisation of RAI systems in the GPS deprived underwater environment. Furthermore, the increased mission duration eliminates the need for daily deployment and recovery using expensive crewed research vessels that put humans in high-risk situations and rapidly exceed the cost of even the most expensive robotic platforms.

To demonstrate this concept, this fellowship will develop a fleet of low-cost platforms that can maintain a desired altitude while obtaining 3D images of the seafloor. Each platform will cost < £5k and have an endurance of several weeks. Novel methods to reduce energy consumption will be developed. This includes a buoyancy control system that can achieve neutral buoyancy at any depth even in regions where the density of seawater is not precisely known. An intelligent, low-power 3D imaging system will be developed together with a hybrid altitude and illumination control system that minimises energy consumption. This will adapt both the frequency at which images are acquired and the strength of illumination used in order maintain a high-quality of data while minimising the amount of energy needed to control the platform's altitude. In order to determine the location of the seafloor images, a novel approach that combines; matched features between sequential images to estimate the motion between acquisitions; seafloor depth observations of all members in the fleet; information available about the terrain and the physics of regional scale flow, will be used to determine the most likely trajectories of each platform. The concepts developed in this research will be demonstrated through a series of experiments carried out under controlled conditions and during field trials using a small fleet of passive drifters that will be deployed at sea.

[1] C Roman et al., Lagrangian floats as seafloor imaging platforms, Continental Shelf Research 31, 1592-1598 (2011)

Planned Impact

The challenges addressed by this proposal have a direct impact on the cost and availability of RAI systems to make detailed observations in extreme and hazardous underwater environments. This contributes to the UK's industrial strategy, which identifies RAI technologies in extreme environments as a challenge area for future sectors, and the goals of the EPSRC for building the resilience needed for a prosperous nation. The stakeholders that will directly and indirectly benefit beyond academia are; 1) industry e.g. developers, manufacturers, distributer and operators through economic gain and end users through the advance of knowledge, 2) regulators in government and private sectors through benefits to society, 3) wider public through benefit to society and the skills of people.

Wider public: On decadal timescales, sustaining seafloor environments and their ecosystems contributes to the economy and social well-being of the general public through job creation (e.g. sustaining industrial activity such as offshore resource development) and security (e.g. understanding the risks posed by global hazards such as climate change and pollution). On shorter timescales of < 5 years, the activities in this fellowship will developed experts with the necessary skill set to deliver impact to UK industry in the RAI sector through collaboration and technology transfer. This combined with outreach and public engagement will raise awareness of RAI systems and inspire future generations of RAI researchers.

Regulators: The growing recognition of human impacts on the environment (e.g. global warming, marine plastics and mining as discussed in recent intergovernmental G7 meetings) drives governments and industry to consider the best practices for sustained environmental monitoring. Building knowledge and understanding through monitoring the impact of industrial activities (e.g. offshore oil and gas extraction, fishing, mining, development of infrastructure and decommissioning) on seafloor ecosystems can help ground estimates of the socioeconomic costs associated with these activities, which in turn can help governments formulate informed and effective conservation and resource management policies over timescales of < 10 years. The cost of meeting the UK government's commitment to the conservation of marine protected areas and responsible decommissioning of offshore infrastructure is dependent on the technologies available for adoption to make observations over the necessary spatial and temporal scales. The application of the low-cost methods for long-term, regional-scale seafloor monitoring developed in this fellowship can contribute to this goal by enabling responsible monitoring at orders of magnitude lower cost than what is possible using today's technologies.

Industry and end-users: The requirement for regional scale monitoring drives industry and end-user investment in the development of technologies and best practises that can be widely adopted. End-users can benefit from these methods by advancing our knowledge and understanding of the impact of human activities on the environment. Industries in the supply chain stand to make economic gains by seeing the technologies they have invested in become widely adopted, both in the UK and globally. The investment in skills needed within the UK to deliver the UK's industrial strategy and the transfer of these skills to industry will lead to future innovations in the RAI sector and continued industrial impact.
 
Description We have gathered environmental conservation data from marine protected areas in Norway and Spain using the platform developed in this research. The low cost platform has been deployed at sea and gathered information about cold water coral and seagrass distribution
Exploitation Route Driftcam platforms are currently written in to an EU grant to survey various marine environments for the HORIZON-CL6-2022-BIODIV-01-01 call
Sectors Aerospace

Defence and Marine

URL https://ocean.soton.ac.uk/driftcam
 
Description The camera systems being developed have led to investment at Sonardyne into developing seabed imaging technology for their own internal research purposes.
First Year Of Impact 2019
Sector Aerospace, Defence and Marine
Impact Types Economic

 
Description (TechOceanS) - Technologies for Ocean Sensing
Amount € 8,975,662 (EUR)
Funding ID 101000858 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 09/2020 
End 09/2024
 
Description EPSRC Core Equipment at Southampton 2020
Amount £704,499 (GBP)
Funding ID EP/V035975/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 11/2020 
End 05/2022
 
Description Scalable seagrass mapping using deep learning
Amount € 3,060 (EUR)
Funding ID SEA02_014 
Organisation EUROFLEETS 
Sector Private
Country France
Start 08/2021 
End 10/2021
 
Title Development of Driftcam prototye 
Description A low cost passive drifting deep-sea imaging system has been developed. The platform can collect images of the seafloor by maintaining a fixed altitude off the seabed while drifting laterally on underwater currents. 
Type Of Material Improvements to research infrastructure 
Year Produced 2020 
Provided To Others? Yes  
Impact None yet. The system will be deployed in March 2020 under EUMR Autonomous habitat mapping in fjords. 
URL https://ocean.soton.ac.uk/driftcam
 
Title Data and underwater imagery from Trondheim Fjord 
Description Driftcam was deployed under the EU MR network to survey cold water coral. Data gathered is being prepared for uploaded onto the SOI Squidle image database platform. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact Led to collaborative work with NTNU in Norway 
URL https://console.cloud.google.com/storage/browser/soi-uos-data/dy109/autosub6000/20190916_090456_as6_...
 
Title GRASS maps data from Balearic Islands 
Description Data of seagrass (posedonia) distribution was gathered from a Marine Protected Area in the Balearic Islands 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact The dataset is released and currently being prepared for ingest into soi squidle 
URL https://www.eurofleets.eu/2021/09/14/grassmap-to-set-sail-today/
 
Description EUMR - Autonomous habitat mapping in fjords 
Organisation Norwegian University of Science and Technology (NTNU)
Country Norway 
Sector Academic/University 
PI Contribution A successful application under the EUMarine Robotics Research Infrastructure Network securing ship time and access to robotic platforms belonging to the NTNU. A joint research expedition to be conducted in Trondheim to monitor cold-water coral reefs in Trondheim Fjord. We will be deploying our passive robotic drifters to collect data.
Collaborator Contribution Universit of Porto and NTNU provide access and operational services for ships and robotic systems. University of Sydney is our research partner, and will deploy their own mapping drifters to collect data. Data processing algorithms developed at the University of Southampton and Sydney will be used to process data and make outputs publically available.
Impact The expedition will be in March 2020 so there are no outputs yet..
Start Year 2020
 
Description EUMR - Autonomous habitat mapping in fjords 
Organisation University of Porto
Country Portugal 
Sector Academic/University 
PI Contribution A successful application under the EUMarine Robotics Research Infrastructure Network securing ship time and access to robotic platforms belonging to the NTNU. A joint research expedition to be conducted in Trondheim to monitor cold-water coral reefs in Trondheim Fjord. We will be deploying our passive robotic drifters to collect data.
Collaborator Contribution Universit of Porto and NTNU provide access and operational services for ships and robotic systems. University of Sydney is our research partner, and will deploy their own mapping drifters to collect data. Data processing algorithms developed at the University of Southampton and Sydney will be used to process data and make outputs publically available.
Impact The expedition will be in March 2020 so there are no outputs yet..
Start Year 2020
 
Description EUMR - Autonomous habitat mapping in fjords 
Organisation University of Sydney
Department Australian Centre for Field Robotics
Country Australia 
Sector Academic/University 
PI Contribution A successful application under the EUMarine Robotics Research Infrastructure Network securing ship time and access to robotic platforms belonging to the NTNU. A joint research expedition to be conducted in Trondheim to monitor cold-water coral reefs in Trondheim Fjord. We will be deploying our passive robotic drifters to collect data.
Collaborator Contribution Universit of Porto and NTNU provide access and operational services for ships and robotic systems. University of Sydney is our research partner, and will deploy their own mapping drifters to collect data. Data processing algorithms developed at the University of Southampton and Sydney will be used to process data and make outputs publically available.
Impact The expedition will be in March 2020 so there are no outputs yet..
Start Year 2020
 
Description EUMR - Visual Autonomous Habitat Mapping 
Organisation National Oceanography Centre
Country United Kingdom 
Sector Academic/University 
PI Contribution This is a competitively won access to european infrastructure to collect seafloor mapping data using platforms and ships belonging to the university of girona. It is covered by the EUMR facilities access Trans National Access program.
Collaborator Contribution We were originally going to deploy UoS robotic platforms and imaging systems to conduct a joint survey with Uni Girona, National Oceanography Centre, Australian Centre for Field Robotics. However, due to COVID travel will not be possible, and so data collection will be conducted by the local team (ie. Uni Girona) and this will be shared between the groups for data processing and analysis.
Impact Multidisciplinery between marine science and engineering. Outputs to be generated following a ocean expedition.
Start Year 2020
 
Description EUMR - Visual Autonomous Habitat Mapping 
Organisation University of Girona
Country Spain 
Sector Academic/University 
PI Contribution This is a competitively won access to european infrastructure to collect seafloor mapping data using platforms and ships belonging to the university of girona. It is covered by the EUMR facilities access Trans National Access program.
Collaborator Contribution We were originally going to deploy UoS robotic platforms and imaging systems to conduct a joint survey with Uni Girona, National Oceanography Centre, Australian Centre for Field Robotics. However, due to COVID travel will not be possible, and so data collection will be conducted by the local team (ie. Uni Girona) and this will be shared between the groups for data processing and analysis.
Impact Multidisciplinery between marine science and engineering. Outputs to be generated following a ocean expedition.
Start Year 2020
 
Description EUMR - Visual Autonomous Habitat Mapping 
Organisation University of Sydney
Country Australia 
Sector Academic/University 
PI Contribution This is a competitively won access to european infrastructure to collect seafloor mapping data using platforms and ships belonging to the university of girona. It is covered by the EUMR facilities access Trans National Access program.
Collaborator Contribution We were originally going to deploy UoS robotic platforms and imaging systems to conduct a joint survey with Uni Girona, National Oceanography Centre, Australian Centre for Field Robotics. However, due to COVID travel will not be possible, and so data collection will be conducted by the local team (ie. Uni Girona) and this will be shared between the groups for data processing and analysis.
Impact Multidisciplinery between marine science and engineering. Outputs to be generated following a ocean expedition.
Start Year 2020
 
Description Eurofleets+ SEA Regional Call Proposal GRASSMAP 
Organisation University of the Balearic Islands
Country Spain 
Sector Academic/University 
PI Contribution This is a competitively won opportunity to map seagrass off the balearic islands using platforms of our partner organisations. Our team lead the proposal and planning of the field expedition.
Collaborator Contribution The expedition will take place in September / October 2021 to map seagrass, and essential ocean variable off the coast of the baleric island. Platforms and sensors developed by UoS will be used, and data processing methods will also be used for the gathered data.
Impact none
Start Year 2020
 
Description Sonardyne - Driftcam 
Organisation Sonardyne International Ltd
Country United Kingdom 
Sector Private 
PI Contribution Development of a low-cost seafloor imaging drifter, advise and technology transfer for expertise in subsea imaging
Collaborator Contribution Support of manufacture, technology development, access to company facilities and mentoring, and phd industry sponsorship (90k GBP) for an international student
Impact Design of a mapping system and underwater lighting.
Start Year 2018
 
Title oplab_pipeline 
Description oplab_pipeline is a python toolchain to process AUV dives from raw data into navigation and imaging products. The software is capable of: 1) Process navigation: fuses AUV or ROV sensor data using state of the art filters and geolocalises recorded imagery. 2) Camera and laser calibration: performs automatic calibration pattern detection to calibrate monocular or stereo cameras. Also calibrates laser sheets with respect to the cameras. 3) Image correction: performs pixel-wise image corrections to enhance colour and contrast in underwater images. 
Type Of Technology Software 
Year Produced 2020 
Open Source License? Yes  
Impact Open-sourcing this software has increased the interest of companies in us. They have contacted us directly and we are working on a licensing scheme. 
URL https://github.com/ocean-perception/oplab_pipeline
 
Description BBC feature robotic survey of seagrass at studland bay 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Survey of Seagrass at Studland bay were carried out - the BBC featured the survey work.
Year(s) Of Engagement Activity 2022
URL https://www.bbc.co.uk/news/uk-england-dorset-61320330
 
Description Bournemouth Echo coverage of Studland Bay Robotic Seagrass Monitoring 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Bournemouth Echo covered Studland bay robotic seagrass survey activities
Year(s) Of Engagement Activity 2022
URL https://www.bournemouthecho.co.uk/news/20148158.seagrass-studland-bay-studied-scientists/
 
Description Business booth at Oceans 2022 with Sonardyne International 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact A booth at an international conference with seafloor mapping robots on display. A youtube interview was produced which has ~300 views
Year(s) Of Engagement Activity 2022
URL https://www.unmannedsystemstechnology.com/video/sonardyne-discusses-smarty200-auv-payloads/
 
Description Christchurch Sailing Club coverage of Studland Bay Robotic Seagrass Survey 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Supporters
Results and Impact Christchurch Sailing Club covered robotic survey of seagrass at Studland Bay
Year(s) Of Engagement Activity 2022
URL https://www.christchurchsailingclub.co.uk/article/studland-mcz-and-vnaz-
 
Description Grass maps in local news 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Media (as a channel to the public)
Results and Impact An article about the marine protected area monitoring survey using driftcam was featured in the Ultima Hora spanish newspaper.

https://www.ultimahora.es/noticias/local/2021/09/14/1300295/mar-balear-campo-investigaciones-alto-nivel.html
Year(s) Of Engagement Activity 2021
URL https://www.ultimahora.es/noticias/local/2021/09/14/1300295/mar-balear-campo-investigaciones-alto-ni...
 
Description Grass maps ship to shore 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact A live Ship to Shore broadcast during the Driftcam survey of seagrass in a Marine Protected Area in the Balearic Islands.
Year(s) Of Engagement Activity 2021
URL https://www.facebook.com/watch/live/?ref=watch_permalink&v=713911612855653
 
Description IMaREST Our Oceans Our Future 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Research relating to subsea imaging was featured on the ITN IMaREST Our Oceans Our Future
https://www.imarest.org/policy-news/institute-news/item/5199-ouroceans
https://www.youtube.com/watch?v=lnojznA1BQo&feature=emb_title
Year(s) Of Engagement Activity 2019
URL https://www.imarest.org/policy-news/institute-news/item/5199-ouroceans
 
Description Invited seminar at Marum 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact 40 researchers from Uni Bremen attended the seminar
Year(s) Of Engagement Activity 2020
URL https://www.marum.de/Termine/Blair-Thornton.html
 
Description Invited seminar at Marum, germany 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact PI was invited to present research on underwater imaging (Driftcam, Biocam) to experts at the University of Bremen.
Year(s) Of Engagement Activity 2020
URL https://www.marum.de/Blair-Thornton.html
 
Description Invited talk at DOOS (Deep Ocean Observing Strategy) panel 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Blair Thornton was invited to discuss advances in seafloor visual survey and automated data interpretation technologies at a DOOS panel that focuses on observations in the deep ocean to address the grand societal issues of climate change prediction and adaptation, ecosystem conservation, and sustainable management in the deep ocean. The goal of the DOOS project is the creation of a common statement of requirements and initial strategy for sustained global deep ocean observations; considering all Essential Ocean Variables, regions, and technologies so as to extract high impact, feasibile, and fit-for-purpose actions for the next 5-10 years.
Year(s) Of Engagement Activity 2022
 
Description Invited talk at Oceanology International 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Invited talk to discuss future of marine autonomy and present results of recent expeditions
Year(s) Of Engagement Activity 2019
URL https://www.oceanologyinternationalamericas.com/en-gb/whats-on.html
 
Description Invited talk at the EOOS Technology Forum 2022 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk entitled under the theme "Thinking ahead: The technology of the science we will need for the ocean we want"
Year(s) Of Engagement Activity 2022
URL https://www.eoos-ocean.eu/events/eoos-technology-forum-2022-workshop/
 
Description Invited talk at the European Subsea Cables Association (ESCA) Plenary 2022 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Blair Thornton gave a plenary on 'Re-imagining Subsea imaging' at the ESCA's 2022 meeting. ESCA is an organisation of submarine cable owners, operators and suppliers and is primarily aimed at promoting marine safety and protecting cable installations in European and surrounding waters
Year(s) Of Engagement Activity 2022
URL https://www.escaeu.org/
 
Description Keynote at Underwater Technology 2021 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Keynote presentation by Blair Thornton on advanced in seabed imaging.
Year(s) Of Engagement Activity 2020
URL http://www.ut2021.org/
 
Description Presentation at MATS 2019 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The following presentation was selected by an organising committe for oral presentation at the industry focused MATS (marine autonomy technology showcase)

Driftcam: A lagrangian float for scalable and low cost wide area seafloor imaging

This was delivered by Dr. Miquel Massot Campos, who is a PDRA on this project.
Year(s) Of Engagement Activity 2019
URL https://noc-events.co.uk/mats-2019-day-2-presentations
 
Description Presentation at MATS2022 on seagrass monitoring 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact Presentation by Miquel Massot on "Automated interpretation of AUV imagery for monitoring seagrass distribution in Studland bay"
Year(s) Of Engagement Activity 2022
URL https://noc-events.co.uk/mats-2022
 
Description Press release about robotic mapping techniques developed to monitor seagrass distribution in Studland Bay 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Media (as a channel to the public)
Results and Impact Article about the robotic mapping technology used to monitor seagrass distribution at Studland Bay.
Year(s) Of Engagement Activity 2022
URL https://www.southampton.ac.uk/smmi/news/2022/09/05-auv-smarty200.page
 
Description Press release regarding Studland Bay Robotic Monitoring of Seagrass 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Media (as a channel to the public)
Results and Impact A press release describing survey activities to robotically monitor seagrass distribution at Studland Bay (Marine Conservation Zone). A youtube video was created and has had ~700 views.
Year(s) Of Engagement Activity 2022
URL https://www.southampton.ac.uk/news/2022/08/studland-bay-seagrass-survey.page
 
Description Reimagining Seabed Imaging 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact Invited seminar talk at the winchester school of arts on marine survey
Year(s) Of Engagement Activity 2021
 
Description Stand at MATS2022 (Marine Autonomy Technology Showcase) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact University of Southampton stand at the MATS2022 event. BioCam and other mapping systems were on display, with information about the AT-SEA project.
Year(s) Of Engagement Activity 2022
URL https://noc-events.co.uk/mats-2022
 
Description Swanage news coverage of Studland Bay Seagrass Robotic Survey 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact Swanage news wrote an article on seafloor robotic survey activities
Year(s) Of Engagement Activity 2022
URL https://www.swanage.news/researchers-to-map-studlands-seagrass-using-robots/