Video-based animation of people

The goal of this research project is to solve the fundamental problem of re-using multiple-view video capture of people to support interactive animation with the quality of the source video. Research will investigate the resampling of a multiple-view video database for interactive animation of people to enable user control of movement and viewpoint with video-quality rendering. The proposed research will address the underlying problem of representation of articulated and highly dynamic non-rigid structures in video to allow indexing, reuse and manipulation whilst maintaining the visual quality. Ultimately the challenge is to enable video-quality rending of real people by reuse of captured video sequences allowing user control of movement as in conventional animation. Recent research has demonstrated photo-realistic animation of faces and simple objects by resampling video sequences.This proposal aims to take video-based animation to complex articulated objects such as people without full 3D reconstruction which has been shown to result in loss of visual quality in previous work. The new challenge is to synthesise video sequences of novel movements from captured video of different motion. Animating people from video requires several key advances, including: a representation of human posture and movement which supports the generation of sequences of previously unseen movements; a representation of multiple-view video of articulated objects allowing efficient storage and indexing; algorithms to efficiently selecting the most useful examples for the synthesis of a new sequences from a large set of example sequences; and new video synthesis algorithms for articulated objects in previously unseen configurations. Animation of people directly from captured video has the potential to provide enabling technology for next-generation, video-quality content production in film, television and games. Video-based animation of people will allow powerful re-use and manipulation of captured video (e.g., generating novel body movements for an actor), together with seamless compositing within photo realistic scenes.