73674Highly expressive voices for machine video content localisationClosedStudyInnovate UKImagine any video available in any language, with both the unique qualities of the original actors' voices, and the unique way in which they delivered their lines, preserved in the new language. This is the ambitious vision that Papercup will make reality by harnessing the latest developments in the world of machine learning. In 2016, Google's Deepmind created the WaveNet vocoder. This was a revolution in speech synthesis. Prior to this, speech synthesis models were either concatenative (meaning that they work by glueing together short audio samples of recorded speech) or modelled methods, which generate speech "from scratch" using a model of how the human speech production system works. Concatenative synthesis typically resulted in more natural sounding voices, but with unnatural flow because the audio samples come from unrelated sections of speech. Modelled methods tended to produce better flow, but the voices sounded robotic. WaveNet is a deep-learning method, trained directly on audio samples, and combines the natural variation of modelled methods with the natural sound of concatenative methods. This development means that speech synthesis could become essentially indistinguishable from human speech. A vocoder (such as WaveNet), however, is not even half the story. You still have to tell it what to say, and how to say it. For a computer to achieve this, we must first recognise what was said in the original video, by whom, and in what way. Papercup exploits the latest developments in deep learning and has developed a patent-pending method for analysing the unique acoustic features of each speaker, and the way in which they delivered their lines. This is encoded by our algorithms using an internal learned representation, which enables the stresses, intonation, and emotion to be transferred across languages, in a manner analogous to the way translation tools translate text from one language to another. In this way, Papercup's approach replicates the unique vocal characteristics of the actors, and replicates their delivery. This has the potential to revolutionise the voiceover translation industry by creating faithful voiceover translations that accurately convey the original content in additional languages, and do so at scale with significantly lower costs than using traditional voiceover translation services with voice-actors.