Cognitive Systems Foresight: Human Attention and Machine Learning

Human observers move their eyes in order to direct their attention to important aspects of a visual scene. There are models called salience maps; they predict where the eyes will move to when looking at a scene. At present, these models do not deal with video input, nor do they predict how an observer's task will affect where they look. In other words, there are no models for real-life viewing situations, where an observer has a specific task.We are proposing a new approach to this problem. We have access to video information from cameras used in urban surveillance, and to the operators whose job it is to spot abnormal behaviour in such video inputs. We shall obtain (previously unseen) video recordings of events in UK urban streets, and display them in a simulated control room to operators familiar with the town in question. We shall monitor where they look on the bank of video screens, and also when they decide that an event is abnormal and/or requires some form of intervention, e.g. calling the police. We shall use the record of eye fixations to teach a computer system to distinguish between normal and abnormal events. In this way, we shall be able to learn what is important for humans to do such surveillance by observing their eye fixation behaviour, for a realistic (and difficult) task and set of real-life video sequences. The project is important for four reasons. First, this will be the first attempt to develop a model of human attention/eye movements which will be firmly based on realistic video input and a real task. Second, this will be the first time that a computer system is able to learn from human behaviour in this way. Third, we will learn much about the ability of trained observers to cope with a demanding task as the number of TV monitors increases. Finally, we will develop an automated system which will be able to analyse the input from any urban CCTV camera in order to alert operators to look at that video stream - at present, most CCTV video streams are not observed by anyone since there are too many cameras for the number of human observers. Therefore, an automated alerting system is greatly neeeded and this project constitutes the best attempt to date to produce one.