133525Visual understanding of faces in motionClosedFeasibility StudiesInnovate UKRecent days have seen an explosion of markerless facial performance capture methods that can reconstruct the dense geometry of the face from multi-view or even a single video. Unfortunately, even the best of these methods fail to capture the extensive range of shapes and deformations displayed by lips in motion, which results in a lack of expression in video synthesis of faces in speech. Lips exhibit agile and highly deformable motions which are extremely hard to capture -- at the same time, subtle lip motions are often the main vehicle for humans to convey emotions. **Capturing 3D lip motion with high accuracy** is therefore crucial if we are to design systems that can **synthesise realistic human speech** to convey human-like emotions. **The key goal and main innovation in VISIM will be to construct a low-cost, class-leading, three-dimensional (3D) morphable model (3DMM) of the face that focuses on representing the detailed dynamics of the mouth in speech to achieve unprecedented levels of realism in video synthesis of human speech.** We will achieve this by constructing a lightweight, multi-view, high frame-rate, video capture system to reconstruct the motion of the mouth and perform further lip shape refinement using photometric consistency and exploiting the redundancy in multiple views. We will use this to capture a range of subjects and train a new, richer 3DMM that encodes the detailed 3D dynamics of different people in speech. The **business value for Synthesia** is clear: this new, highly expressive 3DMM model will be instrumental in allowing us to **automate** the accurate capture of detailed 3D lip motion just from **monocular video** using an analysis-by-synthesis approach. We will then apply this to high-end facial video synthesis for media and entertainment industries. VISIM will create impact by bringing the high-levels of realism at the affordable price of a low-cost, video-based, automated capture system. This will pave the way to democratising high-end facial performance capture, allowing it to be adopted in new markets such as language dubbing for the TV, movie and advertising industries, where photorealistic results are critical but production budgets have so far restricted access to high-end content creation. This technology will enable Synthesia, a newly formed UK SME, to establish a new market in 'natural' language dubbing with photorealistic synthesis of the face. Keywords: Facial Performance Capture, Video-Based Tracking, Digital Content Creation.