Learning about Activities from Video

Imagine a system that could search the web for video clips containing an activity similar to one already highlighted (e.g. a car parking or two people having a conversation); that could participate in a card game after observing others do the same; and that could detect someone involved in an unfamiliar activity in a car park. Furthermore, suppose that it could do all of these things with no prior knowledge about the specific objects and activities depicted. All of these capabilities can be couched in terms of looking for similar or analogous activities in video clips.A lot of work has been done over the past forty years on devising methods for finding objects and activities in pictures and video clips by hand-crafting computer-models of what they are expected to look like. Ways have now been found to fully automate the creation of such models for objects (e.g. pedestrians) and simple movements (e.g. running) by learning from large sets of pictures and video clips. This should therefore make it possible to search for similar objects and movements with no prior knowledge of those things.Some progress has been made recently on extending this level of automation to handle a limited range of more complex activities in very simple scenes. This has been achieved by firstly learning about the appearance of objects and then learning about the activities in which they are involved using logical induction. Unfortunately there isn't yet an easy way to ensure the object categories produced are appropriate for the activities to be learnt. Our main aim is to resolve this problem by steering the search for object categories towards those that lead to the most coherent set of activities. A consequence of this could be to change the way we think about age-old problems in computer vision and logical reasoning.