Deep Learning of Surveillance Video (EPSRC iCASE BAE)
Lead Research Organisation:
CARDIFF UNIVERSITY
Department Name: Computer Science
Abstract
Deep Belief Networks are popular methods for a variety of pattern recognition/machine learning tasks often outperforming conventional computer analysis methods and humans. There is an emerging literature in static image analysis (e.g. http://image-net.org/about-publication). However, analysis of video is only recently beginning to be studied, the most prominent being the Facebook C3D work, which shows that posing the problem as a 3D (2D imagery over time) convolutional problem outperforms more traditional methods, and also augmented the raw pixel data with optical flow data. One issue that remains a fixed width window is used to capture and model the temporal dynamics. Whilst this might be appropriate for the tasks addressed in C3D (e.g. sports video classification) this may have limitations in many other domains. A recent paper at BMVC2015 also addresses the related area of anomaly detection in video.
The project will build on recent investigation of how to detect fights in surveillance video, where activity occurs at varying duration and multi-scale (over time) have been developed to account for such features. Standard image feature recognition methods did not work well under challenge (night-time illumination, variable resolution, crowded scenes) and imaging conditions and texture based temporal features were developed to address the problem. It would interesting to compare a deep belief network approach.
This project will also look more generally at detecting anomalous events, principally to develop alert systems assisting human operators to deal with the challenge of many hundreds of surveillance camera feeds in modern surveillance rooms. One extension would be to investigate methods to catalogue or summarise video - logging only important events or key frames (sequences) where events change or have been identified as anomalous or contain potential interesting activities.
The main research challenge is to define how best to process temporal data of varying duration within a deep learning approach.
The project will build on recent investigation of how to detect fights in surveillance video, where activity occurs at varying duration and multi-scale (over time) have been developed to account for such features. Standard image feature recognition methods did not work well under challenge (night-time illumination, variable resolution, crowded scenes) and imaging conditions and texture based temporal features were developed to address the problem. It would interesting to compare a deep belief network approach.
This project will also look more generally at detecting anomalous events, principally to develop alert systems assisting human operators to deal with the challenge of many hundreds of surveillance camera feeds in modern surveillance rooms. One extension would be to investigate methods to catalogue or summarise video - logging only important events or key frames (sequences) where events change or have been identified as anomalous or contain potential interesting activities.
The main research challenge is to define how best to process temporal data of varying duration within a deep learning approach.
People |
ORCID iD |
Andrew David Marshall (Primary Supervisor) | |
Thomas Hartley (Student) |
Publications
Studentship Projects
Project Reference | Relationship | Related To | Start | End | Student Name |
---|---|---|---|---|---|
EP/P510452/1 | 30/09/2016 | 29/09/2021 | |||
1852482 | Studentship | EP/P510452/1 | 30/09/2016 | 29/03/2021 | Thomas Hartley |
Description | BMVC 2019 XAI Workshop |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Postgraduate students |
Results and Impact | I gave a talk at the Explainable AI (XAI) workshop at the British Machine Vision Conference. The audience was international as were the presenters. A number of industry representatives were also present. |
Year(s) Of Engagement Activity | 2019 |
URL | https://sites.google.com/view/ixmv2019/ |
Description | School Poster Day |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Local |
Primary Audience | Postgraduate students |
Results and Impact | Internal school poster day, allowed us to discuss our research with other PhDs and staff. |
Year(s) Of Engagement Activity | 2019 |
Description | Vision Researchers Colloquium 2018 |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Postgraduate students |
Results and Impact | GW4, a consortium of 4 South West Research Universities meet to discuss and gives talks on vision related topics. I gave a 20 minute talk on my work: Surprise and Expectation. It was well received and generated a number of interesting questions. |
Year(s) Of Engagement Activity | 2019 |
Description | Vision Researchers Colloquium 2018 |
Form Of Engagement Activity | Participation in an activity, workshop or similar |
Part Of Official Scheme? | No |
Geographic Reach | Regional |
Primary Audience | Postgraduate students |
Results and Impact | Presented poster at Bristol University on network pruning. Some useful feedback given. |
Year(s) Of Engagement Activity | 2018 |