Sub-THz Radar sensing of the Environment for future Autonomous Marine platforms - STREAM

Lead Research Organisation: University of Birmingham
Department Name: Electronic, Electrical and Computer Eng

Abstract

The impending era of autonomous shipping will create a revolution in maritime navigation and mission planning which has many parallels with the revolution which is already underway in the automotive world and will impact all aspects of vessel design, manufacture and operation, reducing costs and environmental impacts. The key enabler will be the role of advanced electronics to provide the platform with full intelligence to facilitate autonomous operation with sensing and processing capabilities superior to those of a human.
Merchant shipping companies announced autonomous vessels in 2018 [http://spectrum.ieee.org/transportation/marine/forget-autonomous-cars-autonomous-ships-are-almost-here]. However, in addition to large autonomous ships, there is a growing, potentially huge, market for small/medium agile vessels, demanding new sensing capabilities to provide situational awareness of the proximate environment. Their requirements differ significantly from those of large ships: (i) for the safety of the boat, humans or sea animals in/on the water, robust all-weather day/night detection and classification of small objects is required at ranges of up to ~300 m - too close for large ships to manoeuvre, (ii) large waves are more hazardous for smaller boats so wave profiling is critical for adaptation to the dynamic environment and safe path planning.
We assert that the key sensor modality to satisfy these requirements is novel sub-THz radar operating in the 140-340 GHz frequency spectrum. By its nature, radar is robust to the conditions that limit electro-optical (EO) sensors, but the proposed short wavelength and wide bandwidth can bring key capabilities unavailable in traditional marine radar: (i) imagery that is closer to familiar video, able to exploit the vast legacy of image processing algorithms; (ii) greatly improved cross-range resolution from a small sensor, leading to significant improvements in detection and classification of small objects and compatibility with small vessels; (iii) 3D imagery that can highlight objects; (iv) sensitivity to surface texture which will facilitate image segmentation and, ultimately, enable detection of anomalies within the mapped scene; (v) adaptability of the waveform for enhanced scene assessment in the spatio-temporal domain.
This proposal falls under EPSRC's Sensors and Instrumentation theme, addressing the challenge of providing new and essential capabilities for situational awareness for small marine craft, ensuring safe and efficient operation in dynamic sea conditions. This will be achieved by creating a sub-THz intelligent radar, delivering superior imagery, precise measurements and cognitive scene assessment by adapting radar parameters and using novel data processing including: (i) two-stage data assessment to detect and classify objects as anomalies in the '5D descriptor space' of range, cross-range, elevation, Doppler and micro-Doppler and (ii) mapping the dynamic 3D sea surface in real time. This will enable reliable detection of hazards including small surface or semi-submerged objects, hazardous seas at ranges up to ~300 m with high spatial and temporal fidelity.
The proposal comprises a wide scope of essential research stages: phenomenological studies of sub-THz scattering and propagation above the sea surface, comprehensive analysis of radar signatures and imagery of marine objects and environment and development of cognitive sensing strategy and signal processing to provide situational awareness.

Planned Impact

The major impacts of this work will arise from the scientific work on propagation and scattering of THz waves in a marine environment, image resolution enhancement and 3D mapping, and development of cognitive strategies for small object detection. This will provide situational awareness on small vessels to yield improved safety and protection of sea life. We will publish scientific results in open access publications and create a publicly available repository with datasets of labelled radar signatures and imagery to aid development of classifiers for marine objects, both by industry and by other academic organizations around the world. Technological innovation protection by patents will enable a financial return via licensing and, ultimately, by commercialisation through partnership with industry.
A principal target for the impact is the potentially huge emerging sector of autonomous operation and support for the helmsman, for smaller professional and pleasure boats, which will be an important market for the UK and global shipbuilding industries in coming years.
Through the project's Stakeholder Advisory Group the project will ensure that its work remains well aligned to the needs of industry, and that industry itself is aware of the work. These are both major factors for ensuring that the work will be taken through to production. This will form its industrial impact and then the deployment of these systems will increase safety at sea for humans, and also enhance the safety of sea mammals, thus in turn reducing the damage to the environment which is a consequence of such accidents/collisions..
The proof that sub-THz radars can provide the solution to meet the autonomous marine sensing requirement will also spur the advance of the emerging technology of sub-THz components and sub-systems and support further work in this area. In particular, the system designs which emerge from this project will inform the component designers of the performance which they will need to achieve.
The close industrial connections of the work will also inject a particular note of the practical relevance of the research, which will increase the impact of the academic publications.
The project will lead to a significant impact in the areas of regulation of maritime operations and radio regulations, in order to enable autonomous vessels to operate effectively. Industry operates under a framework of international regulation, supervised by the International Maritime Organisation and the International Telecommunications Union respectively. The project will influence these bodies indirectly, through the Stakeholder Advisory Group whose industrial members have inputs to the Maritime and Coastguard Agency and Ofcom who in turn have inputs to the international bodies.
A different but another important impact arising from the measurements on sea mammals will be the development of the principles of novel techniques for monitoring sea mammals, in particular to check on their presence in areas where man-made structures, such as oil rigs or wind farms, might affect their environment. The project's publications and the measurement data base will provide the basis for the commercial and the bespoke academic development of radar systems to perform this task. The progression of this impact will be supported by the presence of representation of the SMRU Consulting Ltd. on the Stakeholder Advisory Group.
The PhD students and Research Fellows working alongside the senior staff and recognised scientists will gain valuable experience, knowledge, communication and technical skills to develop capabilities that they can transfer to their subsequent industrial or further academic careers. Working with industrial partners will increase their awareness of system integration aspects and industrial needs and will give broader vision which will result in high quality expertise in both academic and industrial research paving ways to leadership in academy and industry.
 
Description In spite of significantly disrupted experimental part of the project due to COVID closures, the radar data were collected in maritime environment (coastal area) at frequencies 79 GHz and 300 GHz in 2021 and, similar trials were made at Birmingham Edgbaston Reservoir in 2022, which were made with the upgraded experimental hardware and software to increase capabilities. They consituent first publicly available radar imagery of marine scenes at such frequencies. Initial data is put in repository.
New approach for nomaly detection in radar data has been developed, where spatial FFT is used to trasform and filter out normative vs compact features, with the inverse FFT output data which will predominantly consist only anomalous (spatially compact) form.
Image segmentation approach is also being developed to detect regions within the scene which may potentially include targets of interests.
Exploitation Route Strong interest from BAE systems, ICASE funded by EPSRC/ BAE systems was confirmed on the subject of anomaly detection.
Other partners on the projects will benefit from the project outcomes: L3 Harris and Raymarine.
Sectors Aerospace, Defence and Marine,Transport

 
Description EPSRC/BAE Systems ICASE " Maritime target detection at mm-wave frequency range through novel anomaly detection approach"
Amount £40,000 (GBP)
Funding ID Vaucher 210172 
Organisation BAE Systems 
Sector Academic/University
Country United Kingdom
Start 10/2021 
End 09/2025
 
Description Multi-Dimensional ISAR Imagery From Space To Space
Amount £393,000 (GBP)
Funding ID DSTLX1000163770 
Organisation Ministry of Defence (MOD) 
Sector Public
Country United Kingdom
Start 10/2021 
End 04/2023
 
Title 76-81 GHz Millimeter Wave Radar Sea Surface and Maritime Target Measurements 
Description The purpose of the described maritime radar trials was to conduct imaging radar measurements of sea surface clutter/reflections and targets of opportunity in the 76-81 GHz frequency band. This is the first part of the University of Birmingham data collection for the EPSRC funded STREAM project, EP/S033238/1 Sub-THz Radar sensing of the Environment for future Autonomous Marine platforms 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact Based on this dataset we were able to develop novel anomaly detection algorithm based on FT, which allow compact representation of normative form, which is a sea clutter and therefore it allows hightlight of outlier within, representing the target . 
URL http://edata.bham.ac.uk/626/
 
Title Amplitude Characteristics of Littoral Sea Clutter Data at K-band and W-band (dataset) 
Description Files correspond to data used for the generation of the figures in this manuscript. In each dataset, each column contains the log magnitude values of a single range profile and successive columns correspond to the subsequent range profiles. Dataset_1 and Dataset_2 are 24 GHz radar data looking towards the shore and looking at the open sea respectively. Dataset_3 and Dataset_4 are the corresponding 94 GHz data. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
URL https://risweb.st-andrews.ac.uk/portal/en/datasets/amplitude-characteristics-of-littoral-sea-clutter...
 
Title Doppler Beam Sharpening and MIMO for improved imagery 
Description 77 GHz MIMO automotive radar have been used from the moving platform and DBS technique has been applyied to obtain high resolution imagery for the first time. The results are to be reported at IRS conference 2023 (conference 6 pages paper) and journal paper is in progress. DBS has also been used for the first time to produce imagery with 150 GHz from the moving platform. The results are to be reported at IRS conference 2023 (conference 6 pages paper) and journal paper is in progress. MIMO and DBS technique is used for the first time for sea surface imagery. 
Type Of Material Data analysis technique 
Year Produced 2023 
Provided To Others? Yes  
Impact The researchers and marine sensor developers will be aware about the new technique to obtain high resolution imagery of the sea surface, suitable for marine SA for, specifically, small platforms. With this resolution it will be possible to resolve and detect small targets (smwimmer head) on the sea. The technique is expected to provide an adequate input into the anomaly detection. 
 
Title Multi-sensor dataset for maritime situational awareness. 
Description The dataset was recorded with Microwave Integrated Systems Laboratory sensor suite which includes bespoke high resolution 79 GHz, 150 GHz and 300 GHz as well as INRAS Radarlog data, video (stereo) and Lidar data. Multi-sensory and multidimentional data can be used for development and testing algorithms for target detection, image segmentation and classification of detected targets. The repository include report which describes the measurement scenarios and the structure of the data of the radar sensors used at the Edgbaston reservoir in Birmingham as part of an ongoing data-collection campaign for the STREAM project. Data was collected on 05/11/2021 and 04/02/2022. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact The trials were conducted to develop novel DBS/SAR radar approach for sea surface imagery. Hence for DBS/SAR, side looking moving radar measurements and stationary azimuth scans of the reservoir environment are taken. 
URL https://doi.org/10.25500/edata.bham.00000807
 
Title Physical Model of sea surface with targets for Simulation in HFSS 
Description In order to develop and test algorithm concepts, especially for hard to measure experimental scenarios, a methodology of electromagnetically simulating imagery and Doppler measurements of the sea surface and anomalies has been developed. The use of commercial electromagnetic simulation software has been pursued due to previous experience and their use of efficiently developed computational methods. ANSYS Electronics Desktop using HFSS was chosen due to its apparent adaptability to our specific scenario. The Electronics Desktop scripting language has been learnt and incorporated into simulations to allow them to be run on the high performance computing cluster at UoB (BlueBEAR) which drastically reduces simulation times. The simulator is based around the use of the multi-bounce shooting bouncing rays' methodology (fundamentally ray tracing, using Huygens point sources at reflection points). Idealised (main lobe) antennas may be defined in the simulator as both transmit sources and receive elements. A sea surface is simulated in MATLAB from empirical sea surface height spectra (for example Pierson-Moskowitz spectra) and a random phase scheme. The generated surface points are then faceted through triangulation and converted to an STL CAD file that can be imported into the HFSS simulation. STL files of anomalies/targets are imported onto the surface, the surfaces can be progressed over time according to the water surface wave dispersion relation and targets progressed according to their defined motion. Rays are launched towards the scene/sea surface conforming to the defined antenna patterns. The simulation is performed over defined frequency bands of interest (76 - 81 GHz, 145 - 150 GHz and 280 - 300 GHz); the output is a series of S-parameters for the defined transmit and receive antenna combinations - this is inverse FFT'ed to provide radar range profiles on which analysis can be performed. Simulations have been made for scanned imaging, height finding, Doppler and azimuth resolution enhancement algorithm application. 
Type Of Material Computer model/algorithm 
Year Produced 2022 
Provided To Others? No  
Impact This tool will allow modelling of any anomaly within the sea clutter and therefore will enable algorithms development with fidelity inavalable if only based on experiemntal (limited) data. 
 
Description BAE Systems 
Organisation BAE Systems
Country United Kingdom 
Sector Academic/University 
PI Contribution UoB and UStA will share data collected to BAE, results of findings are diisseminated to BAE through annual summary reports. UoB supervise a PhD student funded by BAE/EPSRC through associated ICASE.
Collaborator Contribution BAE is a partner which provided ICASE with cash contribution of £40k to fund a PhD srtudent on the topic, instead of initially planned 50k by the request from the consortium. BAE is helping to organize joint trials in Gosport Marina, pay for boat hire and provide safety boat and targets BAE team is on steering board of the project.
Impact Consortium partners and BAE have had valuable discussion at the stage of writing the proposal, and at later stages when proposal started to steer our research The main outcome is - joint PhD supervision so far, project reports shared with BAE.
Start Year 2020
 
Description University of St-Andrews is a collaborative partner on the project 
Organisation University of St Andrews
Department School of Physics and Astronomy
Country United Kingdom 
Sector Academic/University 
PI Contribution The project STREAM is a collaborative project of the University of Birmingham (lead) and the University (UoB) of St-Andrews (UStA). Related grant is EP/S032851/1 the core of STREAM is a collaboration between two highly experienced radar groups at UoB and UStA who bring expertise in multi-sensor suites for autonomous ground vehicles, sub-THz radar system development, and knowledge of sub-THz targets, clutter and propagation phenomenology. This is complemented by biologists at UStA with specific expertise relating to the monitoring of sea mammals.
Collaborator Contribution 1. Collaboratively developed the proposal 2. Have monthly progress meetings discussing plans and results 3. Took part in large joint trials 4. Produced 2 yearly summaries which were sent to all project partners (Raymarine, JLR, BAE Systems etc)
Impact Data collected together, shared and being jointly processed Results of discusssion are fed into publications Additional contacts with marine sensors developers are being established
Start Year 2020
 
Title 150 GHz radar modification to increase chirp rate and enable Doppler capability for SAR imagery and wave profile reconstruction 
Description In order to increase the measurement capability of our 150 GHz radar system, significant upgrades were made to the FMCW chirp generation hardware. The original hardware allowed for a maximum chirp repetition rate of around 80 Hz with a minimum chirp duration of 1ms. This is far too slow to allow any informative (un-aliased) Doppler processing. The subsequent upgrades involved the development of new clock and reference distribution hardware, programming of direct digital synthesis chirp and local oscillator frequency synthesiser boards, as well as designing and integrating appropriate filtering, mixing and amplification stages. Faster data acquisition hardware and software was also designed and developed. The new system now allows chirp repetition rates of up to 10 kHz, with sub 100 us chirp durations, which makes a radar a first of a kind sub-THz radar with capability to use both spatioal resolution and Doppler to detect anomaly. It was designed as such, to enable the system to adequately sample any expected Doppler's that may be encountered from the sea surface motion and therefore provide an extra degree of information to add to our imaging analysis. 
Type Of Technology Systems, Materials & Instrumental Engineering 
Year Produced 2022 
Impact With this capabilities information content is significantly enriched to detect anomaly in Doppler domain. 
 
Description Advisory board meetings with BAE systems, L3 Harris, JLR 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Dialogue and technical discussion on requirements and collaboration between University of Birmingham and main industrial players in marine autonomy
Year(s) Of Engagement Activity 2020
 
Description Prof M. Gashinova is a char of MODEST group of UK Radar Community (EMSIG) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The scope of this focus group within the EMSIG network is to bring together academics, industrialists, and potential end-users working on development of radar technologies intended to provide sensing and ultimately situational awareness at short ranges, from tens of cm to hundreds of meters.
Key players in global race for autonomy are actively seeking technologies that can deliver situational awareness and mission planning through imaging, detection, tracking, positioning, and classification by fusing diverse technologies, including machine learning and adaptive computing platforms to monitor and to respond intelligently to the changing environment.
The group will focus on identifying current challenges and gaps, defining perspective directions of research and successful strategies for collaboration, complementarity of the expertise within the group, knowledge exchange. One of the intentions of the group activities is to form a recognized expert structure which can be involved in policymaking and regulatory activities within the UK and internationally.
Year(s) Of Engagement Activity 2018