Development of a Prototype High Definition Facial Performance Capture and Animation System

Dimensional Imaging has developed a highly novel and proprietary facial performance
capture solution (“DI4D™”) that requires no special markers, make-up or structured light. The
current DI4D™ PRO system uses a bank of high resolution video cameras to capture the
facial performance of a single performer who sits in front of the system. By capturing dense
“3D scan” data at 60fps and then tracking a dense, fixed topology mesh through the resulting
3D scan sequence, the system provides many times the fidelity of traditional facial motion
capture systems that track discrete dots. However, automatic pre-processing of the data
currently requires approx. 20 minutes per second of captured data and manual post processing
currently requires approx. 30 minutes per second of captured data.
Under a previous InnovateUK funded Launchpad project, Dimensional Imaging has
demonstrated the feasibility of using a head mounted camera (HMC) system to capture 4D
facial performance capture data. This offers the benefits of more freedom of movement for the
actors and the ability to allow multiple performers to be captured simultaneously.
The aims of this project are 1) to develop a pre-production prototype HMC system, 2) test the
prototype HMC in a “production” environment, 3) improve the efficiency of data processing
stage (i.e. reduce significantly the time required to pre-process and post process 1 second of
data), and 4) research the the most effective use of the captured performance data in facial
animation pipelines. We believe that this will result in a world leading system that is capable
of meeting the requirements of next generation video game development as well as VFX for
television and movies