Ultrax: Real-time tongue tracking for speech therapy using ultrasound

Speech Sound Disorders (SSDs) are the most common communication impairment in childhood, affecting 6.5% of all UK children, or 2 children in every classroom. SSDs make it difficult for people to communicate with peers and integrate with society, yet the efficacy of interventions for most types of SSDs is weak. Speech, Language and Communication Disorders (SLCD) are a key UK government priority at present, with 2011 designated the national year of speech, language and communication . A recent government report (Bercow, 2008) highlighted the need for a programme of research to enhance the evidence base for children and young people with SLCD. Our programme of research aims to fulfill this need by developing technology which will aid the assessment, diagnosis and treatment of SSDs. Currently in Speech and Language Therapy, technological support is sparse. Technologies that do exist have been expensive to run or complicated to operate and hence not adopted in clinical practice. This project will develop technology (Ultrax) to turn ultrasound into a tongue imaging device specifically designed to provide real-time visual feedback of tongue movements. Most interventions for SSDs rely heavily on auditory skills; clients must listen to their own productions and modify them. However, with Ultrax people with SSDs will actually be able to see the movements of their own tongues and use this information to modify their speech. It is already possible to capture tongue movements by placing a standard medical ultrasound probe under the chin. Ultrasound has the potential to provide powerful information about atypical speech and to enable speakers to modify their own incorrect articulations. However, the image is grainy, information (especially about the tongue tip) is often lost and the image is difficult to interpret. We will improve this image by exploiting prior knowledge about the range of possible tongue shapes and movements in order to provide valuable constraints in tracking tongue contours in sequences of ultrasound images. We will apply a tongue model to this problem, making use of explicit sequence-based optimization for dynamic tracking and smoothing through time. We will use this technology to enhance the ultrasound images, transforming them into a dynamic, real-time 2D video of the tongue's movements which we hypothesize will be A) more easily understood by children B) extend the range of visible tongue shapes from only vowels and /r/ to include /t/,/k/,/ch/ and other consonants which are often targets for therapy Ultrax will be used to provide bio-feedback therapy for people with SSDs and to provide a means for objectively assessing progress by comparing tongue shapes before and after therapy. We will collect a large database of ultrasound and MRI images of tongue movements from 12 adults (ultrasound and MRI) and 90 primary school children (ultrasound) on which to base the model of tongue contours and to test its performance. At the same time, we will split the 90 children into 3 groups and record each group's response to one of 3 types of ultrasound display: 1. Raw, unenhanced, ultrasound 2. Unenhanced ultrasound with added anatomical context (e.g. the position of the teeth and roof of the mouth) 3. Fully enhanced ultrasound, developed in this project. The ability of the children to imitate tongue shapes and movements will be evaluated to determine whether they find the enhanced images easier to interpret than unenhanced images, leading to an improved ability use ultrasound for bio-visual feedback. We will trial ultrasound therapy with 9 children with SSDs (3 children for each type of display) enabling us to evaluate practical issues arising during therapy and pave the way for a future clinical trial. At the conclusion of our research project we will have developed the basis for a new visual-feedback tool (Ultrax) for Speech and Language Therapists to use in the diagnosis and treatment of SSDs.

Speech, Language and Communication difficulties (SLCD) are common in childhood. They pose a major challenge to society, affecting educational attainment and the social and emotional health of over one million children and young adults in the UK. The most common type of SLCD is a speech sound disorder (SSD), in which children's speech is difficult to understand. Few technologies exist which aim to help children with SSDs. In our research project we will develop a technology, Ultrax which will enhance ultrasound images (from a standard medical ultrasound machine) to visualise the tongue's movements during speech. The ultimate goal of our research is that Ultrax will be used by Speech and Language Therapists to assess, diagnose and treat SSDs by allowing children and adults with SSDs to actually see the movements of their own tongues and hence modify them. There are three large groups targeted to benefit from our research: 1. Children and adults with speech sound disorders. 2. Speech and language therapists learning about typical and disordered articulation. 3. Students of SLT or phonetics learning about typical and atypical articulation For children and adults with SSDs, Ultrax has the potential to be used for more accurate diagnosis of SSDs and ultimately better treatment with faster results, leading to a decrease in the cost of supplying therapy to children with SSDs. Within the life of our project we will trial various types of ultrasound image with 9 children with SSDs. A Speech and Language Therapist with extensive experience of similar types of therapy will design and implement individualised therapy programmes for these children with the goal of improving their speech and ultimately their quality of life. We will also run workshops demonstrating normal and disordered speech with ultrasound for Speech and Language Therapists. At Queen Margaret University, we have experienced success running similar workshops in the past and we have the technology to video-conference seminars to a wider audience. A website will be set up for both professionals and parents/user groups interested in speech and via this website we will communicate our research findings in an accessible manner. As part of our project we will collect one of the largest databases of ultrasound images of tongue movements (90 children and 12 adults). This will be made available to researchers all over the World who are interested in normal and disordered speech production and speech recognition technology. Locally, we will use this database in our teaching of both undergraduate and post-graduate students. Student Speech and Language Therapists can find learning about the dynamics of speech production challenging, our database of ultrasound (and MRI) tongue movements and shapes will be a valuable teaching tool in training the UK's future clinicians. Ultrax has strong commercialisation prospects. Articulate Instruments Ltd is one of only a handful of companies worldwide that supplies instrumentation to the speech therapy market. They have been involved from the outset of this project and are firmly committed to taking Ultrax forward, developing it further into a product for the assessment, diagnosis and treatment of speech disorders in both children and adults in collaboration with research Speech and Language Therapists at Queen Margaret University.