Recognition and Localisation of Human Actions in Image Sequences

The explosion in the amount of generated and distributed digital visual data that we nowadays witness can only be paralleled to the similar explosion in the amount of textual data that has been witnessed the decade before. However, while retrieval based on textual information made great progress and resulted in commercially usable search engines (e.g. Google, Yahoo), vision-based retrieval of multimedia material remains an open research question. As the amount of produced and distributed videos increases at an unprecedented pace, the significance of having efficient methods for content-based indexing in terms of the depicted actions can hardly be overestimated. In particular in the domain of analysis of human motion progress is expected to boost applications in human computer interaction, health care, surveillance, computer animation and games, and multimedia retrieval. However, mapping low level visual descriptors to high level action/object models is open problem and the analysis faces major challenges to the degree that the analysed image sequence exhibits large variability in appearance and the spatiotemporal structure of the actions, occlusions, cluttered backgrounds and large motions. In addition learning structure and appearance models is hindered by the fact that segmentation and annotation for the creation of training datasets are onerous tasks. For these reasons, there is a great incentive for the development of recognition and localisation methods that can either learn from few annotated examples or in a way that minimizes the amount of required manual segmentation and annotation.This project will build on recent development in Computer Vision and Pattern Recognition in order to develop methods for recognition and localisation of human and animal action categories in image sequences. Once trained, the methods should be able to detect and localise in a previously unknown image sequence, all the actions that belong to one of the known categories. The methods will allow learning the models in an incremental way starting from few examples and will allow computer assisted manual interaction using appropriate interfaces in order to facilitate model refinement. The methodologies will allow training the models in image sequences in which there is significant background clutter, that is in the presence of multiple objects/actions in the scene and moving cameras. No prior knowledge of the anatomy of the human body is a-priori considered, and therefore the models will be able to identify a large class of action categories, including facial/hand/body actions, animal motion, as well as interaction between humans and objects in their environment (such as drinking a glass of water).