Machine Learning of Behavioural Models for Improved Multi-Sensor Fusion

Sensor fusion is of increasing importance in the defence and security sector, presenting the most relevant information combined from multiple sources to operators to enable them to make the correct decision quickly. It also enables compact sensors on small platforms such as UAVs to be combined to provide the same information as more sophisticated sensors at lower cost and with greater resilience.

The ability to benefit from combining data from multiple sensors is a function of the models for the behaviour of the targets under surveillance: put simply, the better the behavioural models, the greater the benefit of sensor fusion. Existing models are simple, and typically assume that the target moves according to a nearly-constant velocity model, ie integrated Brownian motion. Any improved behavioural models need to be flexible enough to describe the range of behaviours and target-generated phenomenology (eg related to the use of deception) that are encountered but also deterministic enough both to distinguish target-like trajectories from sequences of false alarms and to extract useful meta-data from the trajectory data (eg when any deception was being deployed).

The availability of, albeit largely asynchronous, historic datasets from each of multiple sensors and the power of machine learning should, in theory, make it possible to learn such models from the data. However, this has not been explored extensively in the past. This is because applying machine learning to such datasets is challenging: the core challenge is to learn the dynamic models efficiently while also capitalising on the well-understood statistics associated with modelling non-linear measurements, missed detections and false alarms. Thankfully, techniques (involving extensions to Particle-MCMC that calculate and then capitalise upon gradient information and can also make use of modern compute resources typically used for Deep Learning) have recently been developed that are applicable to this specific kind of machine learning problem.

This research involved in this PhD project will help Leonardo to seek and adapt these novel techniques to the learning of behavioural models from historic multi-sensor surveillance data and will evaluate the utility of the learned behavioural models in the context of generic multi-sensor fusion contexts and specific contexts relevant to Leonardo and anticipated to include the fusion of electronic surveillance, radar and EO/IR data.

Become an expert in multi-fusion data research and help Leonardo to take their research to the next level and become a future leader in data science working with an important and innovative company, whose essential work helps to keep the UK safe from threats.