Video-based animation of people
Lead Research Organisation:
University of Dundee
Department Name: School of Computing
Abstract
The goal of this research project is to solve the fundamental problem of re-using multiple-view video capture of people to support interactive animation with the quality of the source video. Research will investigate the resampling of a multiple-view video database for interactive animation of people to enable user control of movement and viewpoint with video-quality rendering. The proposed research will address the underlying problem of representation of articulated and highly dynamic non-rigid structures in video to allow indexing, reuse and manipulation whilst maintaining the visual quality. Ultimately the challenge is to enable video-quality rending of real people by reuse of captured video sequences allowing user control of movement as in conventional animation. Recent research has demonstrated photo-realistic animation of faces and simple objects by resampling video sequences.This proposal aims to take video-based animation to complex articulated objects such as people without full 3D reconstruction which has been shown to result in loss of visual quality in previous work. The new challenge is to synthesise video sequences of novel movements from captured video of different motion. Animating people from video requires several key advances, including: a representation of human posture and movement which supports the generation of sequences of previously unseen movements; a representation of multiple-view video of articulated objects allowing efficient storage and indexing; algorithms to efficiently selecting the most useful examples for the synthesis of a new sequences from a large set of example sequences; and new video synthesis algorithms for articulated objects in previously unseen configurations. Animation of people directly from captured video has the potential to provide enabling technology for next-generation, video-quality content production in film, television and games. Video-based animation of people will allow powerful re-use and manipulation of captured video (e.g., generating novel body movements for an actor), together with seamless compositing within photo realistic scenes.
Organisations
Publications
Ivekovic S
(2008)
Human body pose estimation with particle swarm optimisation.
in Evolutionary computation
John V
(2010)
Markerless human articulated tracking using hierarchical particle swarm optimisation
in Image and Vision Computing
V John
(2009)
Articulated Human Motion Tracking with HPSO
in Conference paper see below
Vijay John
(2010)
Articulated human motion tracking using charting and particle swarm optimization
in Conference paper see below
| Description | We developed novel algorithms for recognizing whole-body actions (e.g., walking, running, jumping) from multi-camera videos. Multi-camera set-ups are common in animation and video production studios, but also in surveillance environments. We also developed novel algorithms to obtain a 3-D articulated stick figures tracking a moving person in a multi-camera video, without markers on the person's body (markerless mocap). |
| Exploitation Route | The intended main application was markerless 3-D human motion tracking. At the time the grant was awarded, such systems were cutting-edge research, and the standard was marker-based tracking, which needed time-consuming and invasive preparation of actors being filmed by a mocap system. |
| Sectors | Creative Economy,Digital/Communication/Information Technologies (including Software),Leisure Activities, including Sports, Recreation and Tourism |
| Description | The prototype system we developed was trialled, on a pilot basis, at the Duncan of Jordanstone College of Art, University of Dundee, to explore its suitability for students of the MSc in Animation. The system was not adopted by the MSc as too preliminary, and also because by the end of the project some low-cost commercial systems suitable for artists' use were emerging. |
| First Year Of Impact | 2010 |