Gesture Recognition in Aphasia Therapy

Aphasia is a language disorder, usually caused by stroke. It affects a substantial number of people; there are about 250,000 people living with aphasia in the UK and approximately 45,000 new cases each year. People with aphasia have difficulty with all aspects of communication: speaking, reading and writing. The negative effects of aphasia can be immense and are not confined to the affected person but are also felt by the immediate family and social circle.One approach advocated by therapists is for the aphasic person to communicate using gestures, but this also poses challenges as the ability to produce and understand gestures may be impaired. In recent research we have investigated whether gesture production can be enhanced by therapy and the initial evidence from face-to-face therapy is encouraging. However gesture therapy is resource-intensive and this has motivated our proposal.Our goal is to develop and pilot an innovative gesture training tool to be used in aphasia therapy at home. This will build on our existing work on non-computer based therapy and there is no previous published research in this area. Our approach to achieving this goal will be to refine and adapt existing gesture recognition technology, incorporating it within a training package that we will develop. In so doing, we will refine and exploit existing research into gesture recognition and gesture-based interaction, utilising mainstream interaction devices such as the Nintendo Wii remote and open source software. We will investigate appropriate forms of gesture recognition and feedback for this user community.The development of the tool will address several fundamental issues: it needs to recognise a substantial set of gestures that are relevant to the aphasic user and can be trained by the therapist; it must not be over-sensitive as gesture production is likely to be variable and inaccurate; we need to understand and incorporate appropriate forms of training and feedback in the tool.Research in the wild is integral to this proposal. Close engagement with the user community will be facilitated by our project partner, the Stroke Association. Aphasic people will be employed as consultants in exploring and evaluating gesture recognition technologies as the design and development of the therapy tool progresses. In the later stages of the project, a pilot study will be conducted with 10 people who have severe aphasia to investigate the therapeutic efficacy of the tool as well as its broader acceptability.The potential transformational impact of using gesture-recognition technology in therapy is significant. Given the numbers of people affected by aphasia, the provision of a therapeutically effective, low-cost, gesture training tool would be of substantial benefit.

Who will benefit from this research? The main beneficiaries will be people with aphasia. Aphasia affects about 250,000 people in the UK and profoundly impairs quality of life. Secondary beneficiaries will be caregivers of people with aphasia, a group at risk of negative health outcomes [1], and rehabilitation staff and services, particularly speech and language therapists. Other disabled groups, e.g. people with intellectual disorders, may benefit from similar creative uses of interaction technologies. HCI and LCS researchers will benefit from exposure of ideas to the user community. IT companies may benefit from commercial development and exploitation of the tool. How will they benefit? Aphasic people will benefit by acquiring a new therapeutic tool that will help them learn a vocabulary of gestures and hence communicate with family members and friends. Wii applications with other disabled groups suggest that there may be further benefits for mood and wellbeing [2]. Family members will also benefit from the communicative gains. We will select gestures that relate to caregiving (e.g. 'drink' and 'toilet') to help alleviate carers' burden. Rehabilitation staff will acquire a new therapeutic tool to use with their aphasic clients, based on cheap and widely available technology. As the tool is self administered it should be highly cost effective. This is important, given the scarcity of face-to-face therapy: many clinical settings offer only minimal provision [3]. Timescales: The gesture training tool will be available 12 months into the project, when it will be piloted with 10 aphasic people. If the results are positive, we will make the tool widely available to the stroke and clinical communities. Thus there will be immediate benefits. We will also seek funding for a large follow up trial with at least 50 users. This will provide class 1 evidence about the efficacy of the tool. The current and follow up study will be informative for rehabilitation staff working with other client groups, and may promote further adaptations. Such subsidiary benefits are long term (3-8 years). What will be done to ensure that they have the opportunity to benefit? Engagement with aphasic people is integral to this research. Aphasic consultants will advise on the development of the technology, and different pilot participants will test its learning potential and acceptability. We will consult and inform participants' main carers. Results will be disseminated through the 30 London stroke groups affiliated to the Stroke Association and a travelling road show will give group members taster opportunities with the technology. Results of the study will be published in outlets designed for people with aphasia, such as 'Aphasia Now' (website). Additional activities will engage clinical staff. We will stage a CPD training event, to report on the project findings and give clinicians access to the technology. Practising therapists will be invited to trial the therapy tool with their clients, and give feedback. We will target both clinical and academic audiences when writing for publication and making conference presentations. Any Intellectual Property arising from the project will be protected through patent applications by the University's dedicated Research and Enterprise (CREU) team. We anticipate further development of the tool following feedback from the pilot and will work with CREU to achieve this through technology transfer opportunities. The project team is very experienced in engaging user communities and carrying out high impact research, e.g. exploring computer applications in healthcare (HCID) and outcomes from novel therapies (LCS). Thus the team offers an ideal synergy of skills for this project. 1. Bakas, T. et al (2006) Rehabilitation Nursing, 31 (1) 33 - 42 2. Science Daily (2009) Nintendo Wii may enhance Parkinson's Treatment. 3. Enderby, P. et al (2002) Clinical Rehabilitation, 16, 604-608.