Tensorial modeling of dynamical systems for gait and activity recognition

Biometrics such as face, iris, or fingerprint recognition have received growing attention in the last decade, as automatic identification systems for surveillance and security have started to enjoy widespread diffusion. They suffer, however, from two major limitations: they cannot be used at a distance, and require user cooperation, assumptions impractical in real-world scenarios. Interestingly, psychological studies show that people are capable of recognizing their friends just from the way they walk, even when their gait is poorly represented by point light display. Gait has several advantages over other biometrics, as it can be measured at a distance, is difficult to disguise or occlude, can be identified even in low-resolution images, and is non-cooperative in nature. Furthermore, gait and face biometrics can be easily integrated for human identity recognition.Despite its attractive features, though, gait identification is still far from being ready to be deployed in practice. What limits its adoption in real-world scenarios is the influence of a large number of nuisance factors which affect appearance and dynamics of the gait. These include, for instance: walking surface, lighting, camera setup (viewpoint), but also footwear and clothing, objects carried, time of execution, walking speed. Similar issues are shared by other applications of motion classification, such as action and activity recognition. Multilinear or tensorial models, in which a number of (nuisance) factors linearly mix to generate what we observe (in our case the walking gait), have been proven in the recent past to be able to describe the influence of such factors, for instance in the context of face recognition. However, video sequences are more complex objects than single images. We first need to represent video footages in a compact way.Encoding the dynamics of videos by means of some sort of dynamical model has been proven effective in both action recognition and gait identification, in situations in which the dynamics is critically discriminative. Besides, the actions of interest have to be temporally segmented from a video sequence, while actions of sometimes very different lengths might have to be compared. Dynamical representations are very effective in coping with temporal detection and compression, and indeed several researchers have explored the idea of encoding motions via linear, nonlinear, stochastic or chaotic dynamical systems.In this project, therefore, we propose to develop a novel, general framework for the classification of video sequences (with a focus on the walking gait), based on the application of tensorial decomposition techniques to image sequences represented as realizations of suitable dynamical models.The proposed framework will allow us to deal with the issue of the nuisance factors which greatly affect identification from gait and activity recognition in a principled way. The main goal is to push towards a more widespread diffusion of gait ID, as a concrete contribution to enhancing the security levels in the country in the current, uncertain scenarios. With their implications for crime prevention and security, biometrics and surveillance are fast growing business areas, a fact reflected by the increasing number of government-sponsored initiatives in the area in most advanced economies. In addition, the techniques devised in this proposal are extendable to action and identity recognition with immense commercial exploitation potential, ranging from content-based video retrieval from repositories such as YouTube, to HMI, to interactive video games, etcetera.

Within the private sector companies whose core business is in semi-automatic surveillance or biometrics would be the primary beneficiaries of a robust gait identification system. Most of them focus at the moment on cooperative biometrics such as face or iris recognition: investing in behavioral techniques ahead of the rest of the market could provide them with a significant competitive edge. As large-scale tensor modeling can be useful in many applications, however, companies active in vision or medical imaging, physicians specialized in biometric analysis and sport teams labs could also commercially benefit from this research. We expect policy-makers and government agencies to be attracted by the idea of novel surveillance and biometric systems able to improve the general level of security in the country, and that of sensitive areas in particular. Examples are airport management authorities such as BAA, railway companies, underground and public transport authorities (e.g., Transport for London). The wider public will of course be the ultimate beneficiary of any improvement in the security level in public places and transport. A successful outcome of the project will likely boost the competitiveness of the above mentioned private sector actors, to the economic competitive advantage of the United Kingdom. Companies active in biometrics and surveillance will benefit from privileged access to cutting edge technology in behavioral biometrics. The public sector and the government could potentially see the security level of the country dramatically improved by the adoption of some of the outcomes of this research. The infrastructure is basically already there, as millions of active CCTV cameras make the United Kingdom one of the most surveyed western nations. The UK citizen's quality of life will arguably benefit too, in the longer term, by the deployment of behavioral biometric systems as anti-terrorist measures. Among the indirect benefits to the wider public it is worth to mention the impact on people's health of providing physicians with automatic support in diagnosis based on sophisticated data (e.g., 3D MRI scans) as an application of tensor models. Realistic timescales for such benefits vary with the specific target. To start seeing commercial applications in an initially limited context (e.g. biometric access), an additional three or four years of R&D after the end of the proposed research would be needed, possibly in partnership with a company. It is safer to assume a longer time scale when targeting a wider application to security in public areas. Government or agency support will in this case be crucial. The research assistant working full time on the project will develop a variety of skills he/she could exploit in many possible future employment scenarios. These include: project and website management, Matlab coding and use of sophisticated statistical toolboxes, report and paper writing, professional networking. The proposer belongs to the Oxford Brookes Computer Vision Group, which has very well established channels for technology transfer from research to product and a track record of achieving this. Intellectual Property Rights management and exploitation will be managed by the Research and Business Development Office (RBDO) at Oxford Brookes University. To disseminate our results we plan to arrange seminar days with relevant companies active in the Oxford/London area and beyond. Towards the end of the project, public agencies such as the Transport for London or BAA could be contacted with concrete results to show in order for a proof of concept arrangement to be set up. The Vision Group already enjoys links with HMGCC, the government centre of excellence. More details are given in the impact plan.