Deep Learning of Surveillance Video (EPSRC iCASE BAE)

Deep Belief Networks are popular methods for a variety of pattern recognition/machine learning tasks often outperforming conventional computer analysis methods and humans. There is an emerging literature in static image analysis (e.g. http://image-net.org/about-publication). However, analysis of video is only recently beginning to be studied, the most prominent being the Facebook C3D work, which shows that posing the problem as a 3D (2D imagery over time) convolutional problem outperforms more traditional methods, and also augmented the raw pixel data with optical flow data. One issue that remains a fixed width window is used to capture and model the temporal dynamics. Whilst this might be appropriate for the tasks addressed in C3D (e.g. sports video classification) this may have limitations in many other domains. A recent paper at BMVC2015 also addresses the related area of anomaly detection in video.
The project will build on recent investigation of how to detect fights in surveillance video, where activity occurs at varying duration and multi-scale (over time) have been developed to account for such features. Standard image feature recognition methods did not work well under challenge (night-time illumination, variable resolution, crowded scenes) and imaging conditions and texture based temporal features were developed to address the problem. It would interesting to compare a deep belief network approach.
This project will also look more generally at detecting anomalous events, principally to develop alert systems assisting human operators to deal with the challenge of many hundreds of surveillance camera feeds in modern surveillance rooms. One extension would be to investigate methods to catalogue or summarise video - logging only important events or key frames (sequences) where events change or have been identified as anomalous or contain potential interesting activities.
The main research challenge is to define how best to process temporal data of varying duration within a deep learning approach.