SMART - Shaping Multilingual Access through Respeaking Technology

Our day and age is characterised by a proliferation of live multimedia and multilingual content, such as news and TV programmes. Such content is not accessible to everyone. In many countries, live subtitles in the same language are required by law to enable deaf and hard-of-hearing audiences to enjoy access to information, culture and entertainment. In the UK and other countries (e.g. Canada, Spain, Switzerland), intralingual respeaking is the most well-established technique to produce these subtitles: it relies on human-machine interaction whereby a respeaker listens to the original sound of a live programme or event and simultaneously dictates it to a speech recognition software that turns the each sentence into subtitles displayed on screen.

Respeaking so far has been used to produce intralingual subtitles, i.e. in the same language. Given the current multilingual content boom, the SMART project aims to investigate whether respeaking can be used to produce interlingual subtitles, i.e. in a different language. This entails adding translation to an already challenging task which includes listening, speaking, adding oral punctuation, controlling prosody to minimise speech recognition errors, and accounting for space constraints to produce meaningful subtitles. Automatic translation alone is unable to provide live subtitles in a different language of sufficient quality. Interlingual respeaking (IRSP) represents therefore an innovative technique that builds on its monolingual predecessor to achieve this goal. This technique, however promising, has not been systematically tested yet. Therefore there is a need for empirical data to assess its viability. To this end, SMART sets out to achieve three main goals:

1. Investigate the feasibility of IRSP by analysing its challenges. Participants' performance during IRSP tasks will be studied in detail to inform optimal strategies to overcome these challenges. Two pilot projects on students have yielded promising results in this respect. We will need to explore these further in a sample of professionals;

2. Measure the quality of the live subtitles thus produced using an existing, recently-developed measurement tool (NTR model) which could represent a first step towards establishing a quality benchmark for IRSP;

3. Identify the key competencies needed by professionals already working in the language industry (e.g. interpreting, subtitling) to support timely and efficient acquisition of IRSP skills.

IRSP has the potential to provide interlingual subtitles of virtually any live events, such as lectures, conferences, theatre shows, business meetings, live TV interviews or programmes involving speakers of different languages. IRSP could also enhance traditional subtitling methods for pre-recorded programmes, improving productivity. By doing so, IRSP would make such events and programmes accessible to everyone, irrespective of any sensory or language barriers.

SMART research findings will inform the design of bespoke training courses to equip language professionals with optimal skills to ensure high-quality live interlingual subtitling. We will test these materials in a Summer School during the project; after its end, we will continue to offer training that will provide language professionals with a new career path and the possibility to diversify the services they offer. A public engagement event and informative video will raise awareness of IRSP among the wider public and any other stakeholders interested in providing or benefiting from this service.

The UK pioneered the development of intralingual respeaking (today used to subtitle 85% of live TV programmes). SMART will provide the data needed for the UK to become a world leader in IRSP. IRSP is a radical evolution in respeaking methodology and a smart move towards meeting current communication needs, whilst contributing to 'barrier-free' access to different domains of public and private life for all members of society.

SMART is a project with a high socio-economic impact potential, namely by:

1. Demonstrating the viability of interlingual respeaking (IRSP) as a technique. This will allow access service providers to increase productivity while ensuring quality;

2. Demonstrating how IRSP can facilitate access to, and reuse of, multilingual content, thus fulfilling a social mission, broadening the concept of accessibility and contributing to a 'barrier-free' society;

3. Developing a sustainable and scalable model for IRSP skills acquisition that can contribute to capacity building for access service providers and career development for language professionals;

4. Recommending guidelines for effective and efficient IRSP use that account for different contexts and stakeholders' needs.

SMART aims to consolidate the UK's reputation as a pioneer in respeaking. SMART outcomes will benefit different stakeholder groups, including: access service providers, language industry professionals and professional bodies, members of the general public (especially hearing-impaired individuals and people who cannot access audiovisual content in its original language), third sector, governmental and international organisations. All these groups will contribute to and benefit from the project as follows.

Three access service providers will be involved from the outset as non-academic Advisory Board members and co-creators of knowledge. This will ensure that they are fully consulted throughout every project phase so that SMART goals are consistent with industry needs. This group will also actively contribute to the dissemination of findings and to debates about the scope and implementation of IRSP in different domains of public and private life, for instance through participation in the end-of-project symposium alongside other stakeholder groups (see below). In turn, they will benefit from insights into IRSP skills and how training can be optimised to ensure high quality live subtitling.

Language industry professionals will play an integral role in the project by participating in its experiments and impact activities, i.e. in the Summer School as main recipients of IRSP training, and in a public engagement event to provide a live IRSP demo. In turn, they will benefit from the potential of IRSP to open up a new career path. SMART can build on existing ties with professional bodies to reach this stakeholder group, and on support from access service providers, who are committed to identifying relevant profiles among their staff members (see Letters of support).

At a broader level, SMART will generate significant new insights that will inform and shape the debate about how IRSP can be integrated within current policies, services and processes concerned with the nature, cost and efficiency of accessibility to information, entertainment and culture. To this end, SMART will create opportunities for dialogue with professional bodies, third sector and governmental organisations, for instance through the end-of-project symposium. Direct contact with these groups will be possible through existing investigators' links based on previous collaboration (e.g. with the British Film Institute, charities about hearing loss and migrant rights, Ofcom, United Nations International Telecommunication Union). New links will be sought during the project life. Members of the general public will have the opportunity to experience IRSP-produced subtitles live during a public engagement event at a cultural venue in London and will provide feedback from their user perspective.

Other means for generating impact are included in a carefully planned set of media engagement activities, for instance a project website with integrated social media feed, radio interviews/podcasts on FRED Film Radio and presence on key online platforms (e.g. Media Accessibility Platform). For further details, please see the Pathways to Impact.