Natural Speech Technology

Humans are highly adaptable, and speech is our natural medium for informal communication. When communicating, we continuously adjust to other people, to the situation, and to the environment, using previously acquired knowledge to make this adaptation seem almost instantaneous. Humans generalise, enabling efficient communication in unfamiliar situations and rapid adaptation to new speakers or listeners. Current speech technology works well for certain controlled tasks and domains, but is far from natural, a consequence of its limited ability to acquire knowledge about people or situations, to adapt, and to generalise. This accounts for the uneasy public reaction to speech-driven systems. For example, text-to-speech synthesis can be as intelligible as human speech, but lacks expression and is not perceived as natural. Similarly, the accuracy of speech recognition systems can collapse if the acoustic environment or task domain changes, conditions which a human listener would handle easily. Research approaches to these problems have hitherto been piecemeal and as a result progress has been patchy. In contrast NST will focus on the integrated theoretical development of new joint models for speech recognition and synthesis. These models will allow us to incorporate knowledge about the speakers, the environment, the communication context and awareness of the task, and will learn and adapt from real world data in an online, unsupervised manner. This theoretical unification is already underway within the NST labs and, combined with our record of turning theory into practical state-of-the-art applications, will enable us to bring a naturalness to speech technology that is not currently attainable.The NST programme will yield technology which (1) approaches human adaptability to new communication situations, (2) is capable of personalised communication, and (3) takes account of speaker intention and expressiveness in speech recognition and synthesis. This is an ambitious vision. Its success will be measured in terms of how the theoretical development reshapes the field over the next decade, the takeup of the software systems that we shall develop, and through the impact of our exemplar interactive applications.We shall establish a strong User Group to maximise the impact of the project, with a members concerned with clinical applications, as well as more general speech technology. Members of the User Group include Toshiba, EADS Innovation Works, Cisco, Barnsley Hospital NHS Foundation Trust, and the Euan MacDonald Centre for MND Research. An important interaction with the User Group will be validating our systems on their data and tasks, discussed at an annual user workshop.

Leading market analysts predict that revenues from speech technology in North America alone will reach $1 billion by 2011. The reality has lagged behind such predictions in the past since the technology is not refined enough even at this time, but paradigms are shifting. The revolutionary change in connectivity and mobile computing in recent years gives rise to a number of compelling application drivers for the proposed research programme: (1) Rapid developments in mobile computing - decreasing power consumption, high network bandwidth and cloud computing - are stimulating demand for new interfaces. (2) Demographic and economic pressures mean that home care and support systems will become commonplace; such systems will benefit from personalised spoken interaction. (3) Remote meetings are becoming standard, stimulated by the economic conditions and climate change; natural speech technology will enable much richer interactions. (4) As data access becomes more open, the volume of available audio data will increase exponentially; natural speech transcription will result in such data oceans become searchable and structured. (5) There is a potentially huge market (entertainment, consumer apps, robotics) that would be opened up by the availability of adaptive, controllable, expressive speech synthesis. (6) Clinical applications of speech technology will be substantially enriched by the personalised systems proposed in NST. As these drivers have reached a critical level, the NST team has made a number of crucial breakthroughs in adaptive speech synthesis, in conversational speech transcription and in new algorithms to robustly handle changing environments. The research potential is thus poised to meet the application drivers. Beneficiaries of the research can be found in the commercial sector (e.g., remote meeting technology; speech synthesis for computer games; speech archive search; etc), the public sector (e.g., voice reconstruction services for the National Health Service), the third sector (e.g., charities providing support for sufferers of neurodegenerative diseases), art and design, policy makers (e.g., investment in the use of spoken language technology can reduce travel and therefore carbon emissions; it can also enable people to live longer in their own homes, thus reducing the need for residential care services), and the general public (e.g., prospective voice banking and donation could become as commonplace and as widely known as blood donation). The programme's direct training and development impact will be large, through the PhD students and researchers who will work on the project and through researchers on projects that are drawn in on the side of NST; indirectly the training impact will be even larger through other students, researchers and visitors at the three universities, as well as programme workshops.