3D Intrinsic Shape Recognition Under Deformation and View Changes

In the proposal, we tackle the novel visual recognition problem of 3D (three-dimensional) deformable object shape identities or categories. Images of 3D objects undergo large appearance changes due to different object poses (articulation or deformation) as well as camera view-points. We attempt to recognise objects from single images by their 3D shape identities (or intrinsic shapes) regardless of their present poses and camera view-points. Humans can perceive 3D shapes of objects from single images, provided that they have previously seen 3D shapes of similar other objects. The knowledge formerly learnt on 3D shapes is called 3D shape prior. A key idea for fulfilling the proposed task is to learn and exploit the shape priors for object recognition.

The proposed research is well-lined with and goes beyond important topics of computer vision. Whereas much work for view-point invariant object recognition is limited to rigid object classes with bountiful textures, we consider deformable object shapes. In a series of work in the field of single view reconstruction, promising results have been shown for human body shape reconstruction under pose variations. There has also been a notable latest success in 3D human pose recognition. On the top of these results, we go beyond to capture 3D intrinsic shape variations for object recognition. The intended outcomes would benefit the relevant academic fields and their existing markets, and would also lead to potential new applications such as automatic monitoring of public obesity and animal tracking.

As the relevant academic topics have been long-explored and matured to have yielded commercial application markets, our research potentially benefits those existing markets: e.g. content based image retrieval, photo album management and surveillance systems, by recognising intrinsic object shapes. Human shape modeling in the proposed work itself is a useful output for man-machine interface such as avatar, when it explicitly yields shape reconstruction. It also helps human pose recognition, which was recently deployed for a commercial console game with a huge success, in an invariant manner to human shape changes.

The proposed research potentially spins out new applications, especially interesting for public health cares. Public heath, particularly obesity has doubled since 1980 and has been a top social issue in UK and the world. Systematic ways of monitoring and caring individual and public degree of overweight are highly demanding. An interesting new sub-topic of computer vision study, food recognition [49,57], is relevant to this issue. Owing to the widespread use of smart phones equipped with a camera, users can take photos of daily food to consume and the software automatically recognises food types and provides users with useful dietary information according to a pre-defined food calorie table. Various mobile health projects (called m- or e-health) have also been carried out for health services and information. The proposed system can automatically categorise human shapes in real-time and without users' attention or notice (no particular pose and camera view-point needed). The estimated shapes can lead to more direct measurements for obesity i.e. waist circumferences and weights. We intend to carry on our research in the context of vision-based health-care, by human shape recognition, food recognition and also their combination in future.

Our solution aims at recognising human individual shapes or shape categories. This further strengths existing biometric solutions (face, gait recognition) in an appropriate environment. A X-ray type scanner that produces "naked" images of passengers has been adopted for speeding up security checks at Manchester Airport. With the aid of the sensor, clothing effects are removed in the deployment of the proposed silhouette-based algorithm. The proposed algorithm can be combined with existing face or gait recognisers to improve the accuracy. The real-time solution does not need to save images for further use and quickly destroys images to lessen privacy issues.

Our proposed research is for generic object categories, not limited to human bodies. Animal tracking and monitoring especially for horse or big fish has been long studied. They need an non-invasive manner of monitoring without human observers and present satellite navigation tracking system is costly and requires specific sensors. The proposed vision-based solution would be useful to help and improve existing solutions by consideration of large shape and pose variations, and to detect animal sub-species by their shape categories in a reasonably constrained environment.

Our research and successful outcomes would positively influence security and health related policy-makers and government agencies. With adverse impacts of government funding cuts on UK national health services, technological development in mobile health-cares can help achieve the productivity and cost-efficiency savings necessary in both public and private healthcare sectors.

From the intended project activities, the P.I. would learn skills for project management and professional networking, and the R.A. the skills to carry out main research activities including efficient coding, management of different program versions and code sharing. Both P.I. and R.A. would be involved in and learn skills of interacting with a wide spectrum of audiences, and giving a clear description of the project goals and achievements for maximum dissemination of the results.