ACE-LP: Augmenting Communication using Environmental Data to drive Language Prediction.

Communication is the essence of life. We communicate in many ways, but it is our ability to speak which enables us to chat in every-day situations. An estimated quarter of a million people in the UK alone are unable to speak and are at risk of isolation. They depend on Voice Output Communication Aids (VOCAs) to compensate for their disability. However, the current state of the art VOCAs are only able to produce computerised speech at an insufficient rate of 8 to 10 words per minute (wpm). For some users who are unable to use a keyboard, rates are even slower. For example, Professor Stephen Hawking recently doubled his spoken communication rate to 2 wpm by incorporating a more efficient word prediction system and common shortcuts into his VOCA software. Despite three decades of developing VOCAs, face-to-face communication rates remain prohibitively slow. Users seldom go beyond basic needs based utterances as rates remain, at best, 10 times slower than natural speech. Compared to the average of 150-190 wpm for typical speech, aided communication rates make conversation almost impossible.

ACE-LP brings together research expertise in Augmentative and Alternative Communication (AAC) (University of Dundee), Intelligent Interactive Systems (University of Cambridge), and Computer Vision and Image Processing (University of Dundee) to develop a predictive AAC system that will address these prohibitively slow communication rates by introducing the use of multimodal sensor data to inform state of the art language prediction. For the first time a VOCA system will not only predict words and phrases; we aim to provide access to extended conversation by predicting narrative text elements tailored to an ongoing conversation.

In current systems users sometimes pre-store monologue 'talks', but sharing personal experiences (stories) interactively using VOCAs is rare. Being able to relate experience enables us to engage with others and allows us to participate in society. In fact, the bulk of our interaction with others is through the medium of conversational narrative, i.e. sharing personal stories. Several research projects have prototyped ways in which automatically gathered data and language processing can support disabled users to communicate easily and at higher rates. However, none have succeeded in harnessing the potential of such technology to design an integrated communication system which automatically extracts meaningful data from different sources, transforms this into conversational text elements and presents results in such a way that people with severe physical disabilities can manipulate and select conversational items for output through a speech synthesiser quickly and with minimal physical and cognitive effort.

This project will develop technology which will leverage contextual data (e.g. information about location, conversational partners and past conversations) to support language prediction within an onscreen user interface which will adapt depending on the conversational topic, the conversational partner, the conversational setting and the physical ability of the nonspeaking person. Our aim is to improve the communication experience of nonspeaking people by enabling them to tell their stories easily, at more acceptable speeds.

Communication is fundamental to quality of life. Having a voice enables disabled individuals to direct their lives; it impacts on their mental health and helps in finding employment and remaining in work for longer. Slow communication rates mean that many disabled people are lonely which results in poor quality of life. "The Right to Speak", a 2012 report by the Scottish Government warns that poor communication affects delivery of care and puts people's lives at risk. Although estimates suggest that 0.05% of the UK population could benefit from current VOCAs, recent research by Communication Matters (CM), the largest charity for people who use AAC, has recognised that with changing demographics and improved technology, future generations of VOCAs could be used by an order of magnitude more people (0.5% of the UK population).

This project will target nonspeaking literate people including people with cerebral palsy; locked-in syndrome, motor neuron disease (MND), head and neck cancer, and Parkinson's disease (PD). According to the CM research, the target groups would account for around 25% of AAC users, approximately 315,000 people in the UK. The severity of disability and the progressive nature of some diseases impacts on families, carers, healthcare professionals, friends and all who interact with the AAC user, increasing the beneficiaries significantly.
This is a participatory research project and as such, at least 20 end users, their families, friends and caregivers will have direct benefit from the project. From experience, we know that participants benefit socially and their self-esteem and communication skills improve when involved in a research project.

We will extend our existing AAC Research Group online community to engage interested parties throughout the project to reduce the chasm between innovation and adoption. In addition, we will present workshops at Communication Matters Symposia throughout the project to present our results to nonspeaking people, their families, speech and language therapists and other professionals interested in AAC.

We have a strong track record working with industry to transfer research into commercial products. We will carry out a professionally-run industrial workshop in the form of a technology roadmapping session in year 3 to identify routes to impact. We have identified two possible routes (an open source non-profit organisation and a technology start-up) which will be the starting point for our discussions with University advisors, our Advisory Group and our Industrial partners. Our three industrial partners anticipate that they will benefit from the outcomes of the research, both in terms of AAC technology and within mainstream surveillance data analysis.

Publicity features in newspapers, radio and television will be used to raise the public awareness of the abilities and needs of people who use AAC; building on our success in obtaining media publicity for our previous projects. We will also collaborate with the Scottish National Museum to create an installation as part of their new Communications exhibition.