Robust Incremental Semantic Resources for Dialogue

When humans process language, they do so incrementally, understanding and producing sentences on a word-by-word basis. In conversation, we easily switch roles between speaker and hearer mid-sentence, taking turns speaking and listening to show attention, clarify information or add detail when needed, interactively contributing to a shared, emerging picture of what we mean. If we want human-computer dialogue systems to be natural, efficient and easy to use, they must behave as incrementally as humans do: understanding and reacting interactively on a word-by-word basis rather than insisting on fully-formed sentences. We would prefer a system which behaves as in (1) below to the more familiar but annoying (2), or even the more patient but less interactive (3):

(1)
Usr: I'd like er [pause] . . .
Sys: Yes?
Usr: a ticket to Paris from, hang on . . .
Sys: Paris, France?
Usr: right, from London please.
Sys: OK, checking for Paris to London.

(2)
Usr: I'd like er [pause] . . .
Sys: I'm sorry, I don't understand. Please state your destination.

(3)
Usr: I'd like er [pause] . . .
Usr: a ticket to Paris from, hang on . . .
Usr: from London please.
Sys: OK. Do you mean Paris, France?

Previous research has developed computational models of dialogue which can behave incrementally, allowing the kind of interaction shown in (1); but they currently rely on hand-written rules or statistical models to relate words to actions and concepts. These lack the ability to express the complex meanings that human language is so good at conveying, and are time-consuming to create for any new system, domain or task. Instead, they need incremental models which deal with semantics, updating some representation of meaning as each word is heard or spoken, and which can be automatically learned from data; but general methods for doing this are currently lacking. This project will bridge this gap, providing a linguistically-based, learnable framework for incremental semantic interpretation and generation, which can be used to improve and extend existing dialogue systems.

The project will start from recent work in theoretical linguistics and dialogue modelling which has produced the incremental semantic processing framework Dynamic Syntax (Kempson et al., 2001). This shows promise in modelling complex incremental dialogue, but is currently under-developed from a practical point of view, needing time-consuming expert hand-crafting, and missing a link between action planning and language generation. This project will address these issues. First, we will develop methods for automatically learning Dynamic Syntax grammars from data, allowing other researchers to easily produce and use their own versions in their own systems. Second, we will develop its methods for generating language so that it can be integrated with the way dialogue systems plan their actions on the fly. These new capabilities will be implemented computationally and evaluated on real data. Together, they will then be used to build a demonstration dialogue system which can behave incrementally, and will be packaged into a publicly available toolkit for researchers to develop their own incremental, semantic dialogue systems.

Benefits for users of human-computer dialogue systems:

Language-based or speech-enabled human-computer interfaces are common, and are becoming more common as mobile phone use and technology improves, and as computer games become more sophisticated. Users of these interfaces now include general internet users (via language-based interfaces to remote databases and networked applications); mobile phone/device users (via speech-based interfaces to local and remote devices); computer games players (via dialogue-based interaction with virtual characters); visually impaired individuals (via screen-free dialogue interaction). The quality of life of these users would be enhanced by the increased naturalness and interactivity of the incremental dialogue systems that will be enabled by this research. Realising this impact will involve transfer of the tools and toolkit resulting from the project to other academic and commercial research groups, with subsequent commercial development requiring timescales of 2-5 years - see below.

Benefits for the language technology industry:

Dialogue systems are used by many commercial applications (telephone call-handling, customer response, ticket-booking etc.), and the size of the language technology industry in the EU was 8.4 billion euros in 2008, with a projected growth rate of 10% annually [1]. The resources and toolkit to be developed by this project can lead to commercialisable dialogue system applications, and will therefore benefit this industry in future, contributing to global economic performance, and specifically the economic competitiveness of the UK. Commercialisation will require significant further development, but the compatibility of the resources with existing approaches to dialogue system implementation mean timescales could be short (e.g. 2-5 years, as witnessed by similar developments of e.g the CHAT or LetsGo! systems).

Benefits for school and university students:

The tools and toolkit produced will provide resources for use in university coursework within QMUL and beyond (resources will be released publicly under an open-source license), and for use in school teaching on relevant technology and language-related courses. We will facilitate realisation of this impact by publishing a schools-targetted article in the cs4fn magazine and website, by presenting at events such as the Big Bang and National Science and Engineering Week, and by developing a web-based demonstration of the technology.

Benefits for academic researchers:

Many dialogue researchers are currently engaged in research into incrementality in dialogue modelling and dialogue system implementation, but this is a young and emerging sub-field. The project results, and resulting toolkit and resources, will directly provide resources for continued development of such systems, providing general incremental semantic processing capabilities for interpretation and generation and easy deployment via learning from data and integration with general agent planning methods. Through continued research into incremental systems, we expect commercialisation options to begin over the next few years, as was the case with non-incremental dialogue system research in the preceding decades.

[1] http://www.langtech.co.uk/us/knowledge-center/doc_download/10-study-on-the-size-of-the-language-industry-in-the-eu.html