Babble: domain-general methods for learning natural spoken dialogue systems

The demand for future conversational speech technologies is estimated to reach a market value of $3 billion by 2020 (Grand View Research, 2014). Our proposed technology will provide vital foundations and impetus for the rapid development of a next-generation of naturally interactive conversational interfaces with deep language understanding, in areas as diverse as healthcare, human-robot interaction, wearables, home automation, education, games, and assistive technologies.

Future conversational speech interfaces should allow users to interact with machines using everyday spontaneous language to achieve everyday needs. A commercial example with quite basic capabilities is Apple's Siri. However, even today's limited speech interfaces are very difficult and time-consuming to develop for new applications: their key components currently need to be tailor-made by experts for specific application domains, relying either on hand-written rules or statistical methods that depend on large amounts of expensive, domain-specific, human-annotated dialogue data. The components thus produced are of little or no use for any new application domain, resulting in expensive and time-consuming development cycles.

One key underlying reason for this status quo is that for spoken dialogue, general, scalable methods for natural language understanding (NLU), dialogue management (DM), and language generation (NLG) are not yet available. Current domain-general methods for language processing are sentence-based and so perform fairly well for processing written text, but they quickly run into difficulties in the case of spoken dialogue, because ordinary conversation is highly fragmentary and incremental: it naturally happens word-by-word, rather than sentence-by-sentence. Real conversation happens bit by bit, using half-starts, suggested add-ons, pauses, interruptions, and corrections -- without respecting the boundaries of sentences. And it is precisely these properties that contribute to the feeling of being engaged in a normal, natural conversation, which current state-of-the-art speech interfaces fail to produce.

We propose to solve these two problems together, by for the first time:

(1) combining domain-general, incremental, and scalable approaches to NLU, DM, and NLG;

(2) developing machine learning algorithms to automatically create working speech interfaces from data, using (1).

We propose a new method "BABBLE" in which speech systems can be trained to interact naturally with humans, much like a child who experiments with new combinations of words to discover their usefulness (though doing this offline to avoid annoying real users while doing so!).

BABBLE will be deployed as a developer kit and as mobile speech Apps for public use and engagement, and will also generate large dialogue data sets for scientific and industry use.

This new method will not require expensive data annotation or expert developers, leading to easy creation of new speech interfaces that advance the state-of-the-art in interacting more naturally, and therefore more successfully and engagingly with users.

New advances have been made in key areas relevant to this proposal: incremental grammars, formal semantic models of dialogue, and sample-efficient machine learning methods. The opportunity to combine and develop these approaches has arisen only recently, and now makes major advances in spoken dialogue technology possible.

There are 3 main arenas of impact for this work: commercial, societal, and academic.
The major societal impact of this work is in widening access to information technology, and making it more efficient and engaging for users through the use of spoken conversation.
The core technology for natural conversational speech interfaces developed here has the potential to impact on several important groups of users:

* Internet users, mobile App users, and wearable technology users via improved conversational interfaces for control and search;
* Visually impaired individuals via better speech interfaces;
* The elderly and disabled, via automated assisted independent living using dialogue-based interaction;
* Users of immersive virtual reality interfaces (using speech for control and search);
* Computer Game players, e.g. via improved dialogue-based interaction with virtual characters;
* Users of educational technologies, using conversational virtual characters for learning;
* People interacting with robots (spoken Human-Robot Interaction).

In all of the above areas, the important economic impact of the work is also in lowering costs for industry through the automation of speech interface development -- a central objective of this proposal. Companies will benefit through the lowering or removal of costs associated with hiring expert system developers and data annotators. These advances will make speech interfaces a more affordable technology, deployable more rapidly in wider contexts.
Academic and industrial researchers will also benefit via the provision of new, open resources for conversational system development.

To realise the full impact of the research, we will therefore focus on the following pathways:

* BABBLE mobile Apps: public engagement with the developed systems released as speech Apps on smartphones, mobile devices, and wearables (like Siri and Google Now);

* Open data releases: the large anonymised spoken dialogue data-sets generated by the project will be of interest to academic researchers and industry;

* BABBLE Toolkit: software and tools developed will be released for academic and industrial use;

* Interdisciplinary publications and research visits: combining research from two previously disconnected communities: wide-coverage grammars of Natural Language and statistical approaches to automated dialogue systems;

* Demonstrations: showcasing the BABBLE technology at events such as the Scottish Informatics and Computer Science Alliance (SICSA) demofests and EC ICT events, which bring together academia with industrial developers;

* Robotarium demos: showcasing BABBLE systems for human-robot interaction applications at the Robotarium (EPSRC Infrastructure Grant, 2013);

* Impact workshop: organised at the end of the project to amplify these avenues to impact, inviting researchers from both academia and industry;

* Spin-out company: the BABBLE technology will be integrated in to a spin-out speech technology company which is currently being developed at Heriot-Watt by the PI, under an "Impact Acceleration" grant.

To reach the general public as well as interested academic and industry researchers, we will also use a range of social media such as Twitter and YouTube, as we have done in the past with success for a number of our projects, as well as using traditional websites. We will also further develop our contacts with press and media, which have lead to a number of newspaper, radio, and television outputs describing the PI's research. Our advisory board member Dr. Matthew Purver, founder and Chief Data Scientist of Chatterbox, will advise us on technology transfer and commercialisation of dialogue and language technology (see letters of support).