Adaptive Facial Deformable Models for Tracking (ADAManT)

We propose to develop methodologies for automatic construction of person-specific facial deformable models for robust tracking of facial motion in unconstrained videos (recorded 'in-the-wild'). The tools are expected to work well for data recorded by a device as cheap as a web-cam and in almost arbitrary recording conditions. The technology that will be developed in the project is expected to have a huge impact in many different applications including but not limited to, biometrics (face recognition), Human Computer Interaction (HCI) systems, as well as, analysis and indexing of videos using facial information (e.g., YouTube), capturing of facial motion in games and film industry, creating virtual avatars, just to name a few.

The novelty of the ADAMant technology is multi-faceted. We propose the very first, robust, discriminative deformable facial models that can be customized, in an incremental fashion, so that they can automatically tailor themselves to the person's face using image sequences under uncontrolled recording conditions (both indoors and outdoors). Also, we propose to build and publicly release the first annotated, with regards to facial landmarks, database of facial videos made 'in-the-wild'. Finally, we aim to use the database as the base of the first competition for facial landmark tracking 'in-the-wild', which will run as a satellite workshop of a top vision venue (such as ICCV 2015).

As a proof of concept, and with a focus on a novel application, the ADAMant technology will be applied for (1) facial landmark tracking for machine analysis of behaviour in response to product adverts watched by people at comfort of their home (indoors) and (2) facial landmark tracking for automatic face verification using videos recorded by mobile devices (outdoors). In an increasingly global economy and ever-ubiquitous digital age, the market can change rapidly. As stipulated by the UK Researcher Councils' Digital Economy Theme, realising substantial transformational impact on how new business models are being created and taking advantage of the digital world is one of the main challenges. As human face is at the heart of many scientific disciplines and business models, the ADAManT project provides technology that can reshape established business models to become more efficient but also create new ones. Within EPSRC's ICT priorities our research is extremely relevant to autonomous systems and robotics, since it enables the development of robots capable of understanding human behaviour in unconstrained environments (i.e., design of robot companions, robots as tourist guide, etc.).

Technologies that can robustly and accurately track facial motion as observed by omnipresent webcams in digital devices would have profound impact on both basic sciences and industrial sector. They would open up important new applications of ICT in basic research, medicine, healthcare, digital economy and business.
(a) basic research - application of computer vision tools for automatic tracking of facial signals in the wild could open up tremendous potential to measure behaviour indicators (in psychology, psychiatry, security sector, etc.) that heretofore resisted measurement because they were too subtle or fleeting to be measured by the human eye.
(b) computer science - affective interfaces, interactive games, implicit-tagging of multimedia content, and various online services would all be enabled or enhanced by this novel technology, effectively leading to the development of the next generation of human-centered computing; furthermore the developed technologies could have a huge impact in movies (special effects) and games industry which still perform facial motion capture with tedious manual procedures or require from the actors to wear special equipment which affect their acting performance; finally the developed tracking methodologies will facilitate the development of robust mobile biometric systems.
(b) robotics: the development of automated tools for tracking of facial motion in unconstrained conditions would enable the development of robots capable of understanding human behavior in both indoor and outdoor environments (i.e., design of robot companions, robots as tourist guide, etc.).
(d) medicine and health care - many disorders in neurology and psychiatry (schizophrenia, suicidal depression, Parkinson's disease) involve aberrations in display and interpretation of facial behaviour. ADAManT technology could provide tracked facial motion with increased reliability, sensitivity, and precision needed to explore the relationship between facial behaviour and mental disorder and lead to new insights and remote diagnostic methods. Also, remote monitoring of conditions like pain and depression, remote assessment of drug effectiveness, etc., would be possible, leading to more advanced personal wellness technologies.
(e) digital economy and commercial applications - Automatic measurement of consumers' liking in response to product ads will have profound impact in automatic market research analysis, mainly because this will offer the possibility of conducting massive market-research studies. This is in striking contrast to standard market research methods for collecting liking ratings that are based on self-reporting techniques, which are known to be notoriously slow, error prone, and tedious. The ADAManT technology will be of great value not only for the companies working in market research analysis (such as RealEyes) but it will extremely valuable to their clients too. This is exactly in the line with challenges listed in the UK Researcher Councils' Digital Economy Theme -- realising substantial transformational impact on how new business models are being created and taking advantage of the digital age. The ADAManT technology will also make possible automatic assessment of the driver's stress level, detection of micro sleeps, spotting driver's puzzlement, and effectively enabling next generation of in-vehicle assistive technology. It will facilitate automatic assessment of student's interest level, puzzlement, and enjoyment in online- and E-education, enabling the development of truly intelligent tutoring systems.