Making accessibility accessible: maximising the impact of the integrated immersive inclusiveness project

Lead Research Organisation: University of Nottingham
Department Name: School of English

Abstract

This follow on funding project builds on an existing collaboration between Red Earth Theatre, a small-scale touring company with an established track record and commitment to research in inclusive integrated communication for young audiences (with a focus on deaf audiences) and an interdisciplinary team of researchers across arts and computer sciences at the University of Nottingham.

Red Earth's established 'total communication' approach to making theatre aims to integrate modes of communication - including metaphor, symbol, costume, set, lighting, auditory, signed, oral, written, and music - immersively within the theatrical aesthetics of the performance. British Sign Language and/or Sign Supported English has been successfully embedded within the action through the skilled work of their actors, but the challenge of achieving integrated embedded captions within the stage action to achieve immersiveness rather than the attention splitting caused by the placement of captions whether at the side, above or via phone screens (as with Talking Birds' The Difference Engine) or smart glasses (National Theatre) is expensive and demanding for small- and medium-scale theatre companies.

Through a series of iterative workshops and events with key stakeholders from the deaf community and the seven theatre companies that make up the Arts Council funded Ramps on the Moon network, funded by the AHRC/EPSRC's 2018 Research and Partnership Development call - Next Generation of Immersive Experiences, our team has worked to explore the potential for the use of cheap or freely available immersive technologies - using virtual reality tracking and projection - to support further development of integrated inclusiveness for deaf audiences in small scale touring productions. Having successfully developed, trialled and shared prototype approaches in short extracts of performance, we now seek to embed our suggested solutions in the context of a full touring production. This will enable the integration of our immersive captions as a full element of the performance and production from the beginning of the design process, in contrast to the existing practice of captioning that is added at the end of the production process and situated above or to the side of the stage.

In autumn 2019, Red Earth will tour Soonchild, based on Russell Hoban's book and incorporating specially commissioned music from the folk ensemble Threaded. Set in the frozen North, Soonchild tells the story of the shaman Sixteen Face John as he undertakes an epic journey to capture the world's songs for his unborn child, who cannot be born until the songs have been found. This production, aimed at children aged 8+ and their families, will enable us to showcase the technologies in action, creatively embedding our previous research with the aim of:

- sharing the potential of this innovative technology with audiences. Red Earth planned tour of a minimum of 15 performances in the autumn of 2019 will reach an anticipated audience of up to 1000 young children and family members (the show will be revived in 2020 touring to a planned minimum of 20 medium sized-venues). Feedback from audience members will inform future marketing and information on the system to support venues seeking to engage with often hard to reach audiences in the D/deaf community, ensuring that our integrated immersive inclusive technology has maximum impact with theatre audiences.
- proving the stability and robustness of the system, which will be showcased to new communities of theatre venues and production companies via a hands-on technical workshop and supported via specially created video tutorials and how to guides. On completion of the Soonchild tour, the equipment used will be made available for loan to other companies who wish to trial the technologies, ensuring effective and supported pathways to impact for prospective new users of our integrated immersive incusive technology.

Planned Impact

While this project continues to sustain the pathways to impact envisaged in the original 'Integrated Immersive Inclusiveness project, supporting Red Earth's capacity to deliver on the potential benefits of our prototype technology, our aim is to build new impact outcomes with a significant change in scale through activities centred on a full production of Soonchild which will creatively demonstrate the potential of our aesthetically embedded captioning technology to new audiences and user communities, as follows:

1. up to 1,000 audience members (children aged 8+ and their families) will experience integrated immersive captioning for the first time in the touring show that this project will enable, made feasible by our development of a stable, user friendly, low equipment cost, open source projection mapping system. Follow on funding takes this technology from the laboratory to the stage of venues around the country, where audiences on the spectrum of hearing loss from D/deaf to hard of hearing will benefit for the first time. Success will be measured by capture of audience feedback, and planned integration of the technology into subsequent Red Earth tours.

2. small scale theatre companies across the UK and beyond will benefit from access to that projection mapping system, supported by a set of best practice guidelines for delivering embedded captions, training materials for how to use the technology and how to embed it in a production process, and a focussed workshop/tutorial to disseminate the system, guidelines and knowledge. Both workshop and the availability of materials will be publicised by Ramps on the Moon and the Independent Theatre Council (ITC), and through publication channels such as Arts Professional and The Stage, ensuring maximum reach and impact. Availability of a physical, borrowable system that can be used by other production companies wishing to try out the system before committing to purchasing the necessary equipment makes the trialling of the technology by other small companies feasible and will ensure sustainability of the project beyond the funding period.

3. Red Earth's 'D/deaf Like Me' report, carried out with support from an AHRC Midlands3Cities DTP placement student in 2017, highlighted that 'the major stumbling block to participation of D/deaf people in theatre is communication access - at every stage of the process ... Information in an accessible form to those who need it, via multiple channels - the right language and the right medium' is vital. Through this project, venues across the UK will benefit from work led by CoI Mevel and the Inclusive Engagement Fellow that will develop best practice guidance and examples for marketing materials to ensure that audiences understand what the production will offer. Support for venues to develop sustainable relationships with audiences from the deaf community, many of whom currently fall within the hard to engage category, will be of direct benefit to the individual venues visited on tour, and for other venues visited by Red Earth and other inclusive theatre companies on subsequent tours beyond the funding period.
 
Title Promotional video to showcase creative captioning 
Description With input from Red Earth Theatre, we scripted, storyboarded, shot and edited a short promotional video showcasing creative captioning of the type used in Red Earth's performances. 
Type Of Art Film/Video/Animation 
Year Produced 2019 
Impact The video was promoted very widely by the researchers, Red Earth, the Nottinghamshire Deaf Society and the University of Nottingham, on their website and via social media. The video has since been shown at Netflix headquarters in the United States, and at the 60th American Translators Association conference. The video has also been integrated into academic programmes on audiovisual translation in the Netherlands and in the UK. 
URL https://www.youtube.com/watch?v=MjEo3UL_Zl4
 
Title Soonchild - Red Earth Theatre 
Description The production of Soonchild by Red Earth Theatre was supported by the Making Accessibility Accessible project. Based on Russell Hoban's book and incorporating specially commissioned music from the folk ensemble Threaded, Soonchild is set in the frozen North and tells the story of the shaman Sixteen Face John as he undertakes an epic journey to capture the world's songs for his unborn child, who cannot be born until the songs have been found. This production, which was aimed at children aged 8+ and their families, enabled us to showcase our immersive captioning in action. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2019 
Impact 1,885 audience members experienced Soonchild, including creative practitioners and programmers from organisations such as Manchester International Festival and Sinfonia Viva who have subsequently contacted us to explore the potential for the deploying our projection mapping system in their own productions. Reviews noted the inclusive access: 'Multi-layered and interesting, its core message is shouted even stronger with the thoughtful and effective inclusion of signing and subtitles. So it is a party that everybody is welcome to" (Folk Phenomena); The Stage reported Artistic Director Amanda Wilde as saying, "This research project with the University of Nottingham has encouraged us to evaluate our current access provision and to be more creative with how access is integrated into our future touring shows. As a small-scale touring company committed to providing access for D/deaf audiences, it has afforded us the space with our research colleagues, to be more ambitious and inventive in developing our access offer and to push the boundaries of captioning in theatre." 
URL https://www.youtube.com/watch?time_continue=98&v=kwU83YqDFHI&feature=emb_logo
 
Description Following on from our previous AHRC/EPSRC project 'Integrated immersive inclusiveness: trialling immersive technologies in the creation of inclusive and integrated theatre for deaf audiences' this project aimed to deploy our prototype technologies in the context of a full touring show, Red Earth Theatre's Soonchild (https://redearththeatre.com/shows/soonchild/), in order to: a) enhance and stabilise the technologies developed in the 'Integrated Immersive Inclusiveness' project; b) capture and evaluate the benefits and challenges of integrating immersive captioning into the creative process of show development in order to develop guidance for other companies seeking to deploy the system and c) develop a suite of training materials aimed at small-medium scale theatre production companies and materials to support venues in marketing/explaining our immersive technologies to potential audiences.

Our activities, based around repeated engagement with Red Earth Theatre and its creative and technical team from design meetings to final tour, enabled us to generate the following key findings:

1. Building on the prototype technologies we developed 'Caption-o-matic - Projection Mapping Software', a new projection mapping system with accompanying free to access software that provided a stable, robust and accessible means of providing immersive captioning within the set of any show, integrated within standard design and production processes (e.g. white card set models; Qlab cue controls). Our new system addresses the complexity of common projection mapping systems, simplifying the process by using digital twins of the scenery and an innovative, intuitive and crucially fast position mapping system which fits with the rapid "get-in" times of touring productions. As the system can be generalised to projection-mapping any content, not just captions, on stages the work has wider implications for doing any kind of projection mapped performances. The experience of touring Soonchild proved the stability of the system, with our immersive creative captions shared with audiences of nearly 2,000 across 13 venues in England and Wales; and the system has been shared with nearly 50 production and creative practitioners in a series of workshops (see Narrative Impact section for more details); we are also able to support productions with loan of the projection equipment purchased through this project.

2. Through regular capture of process and reflection from Red Earth, creative team and tech team, through the development and tour of Soonchild, we have been able to highlight the key challenges and benefits of incorporating captions immersively into production design from the outset. Through the workshops and presentations to individual companies as well as to invited audiences of producers, creatives and techs, we have been able to identify further challenges, questions and needs, which continue to inform the development of our suite of training activities and materials.

3. We developed a new and innovative notion of 'accessible paratext' - which highlights the need for the information and marketing around productions to also be accessible in order to provide a crucial point of access for D/deaf members of the public who often feel that theatrical performances are 'not for them'. The perception that the theatre is not for D/deaf audiences appears to be driven by several factors, including the fact that many members of the D/deaf community have neither heard of nor seen integrated theatre, also because access to accessible performances is not forthcoming: information about these performances, in the form of what we here define as paratext either does not exist and is not communicated in a way that is accessible, or that makes the accessible nature of the performances tangible to members of the D/deaf audience. In order to promote creative captions to D/deaf audiences and beyond, a video was scripted, storyboarded, shot and edited in collaboration with Red Earth Theatre: produced by Co-I Mével, 'Creative Captions' (https://www.youtube.com/watch?v=MjEo3UL_Zl4&feature=youtu.be) provides an example of such paratext and shows how captions can add to the storytelling and aesthetics of any show. This film was used as part of the marketing material of Red Earth's Soonchild but has also been widely shared and discussed across the wider industry; see information in Narrative Impact.
Exploitation Route As we have made clear above, the projection mapping system developed as a key outcome of this funding has already been the subject of enormous interest from creative practitioners, companies and venues, and the activities outlined in the Narrative Impact section, including leading training sessions at the Lowry's 'Artist Network Training Session: Making Accessible Work' event on 25 March 2020 to over 50 registered attendees, will help us to embed its use as part of a new approach to making small and medium scale theatre accessible.

Our Narrative Impact response also outlines potential interest in the creative captions work beyond theatre, with interest from the field of live music and screen production.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software)

URL https://immersivetheatrecaptions.wordpress.com/
 
Description Our aim in this project was to build new impact outcomes with a significant change in scale from our original 'Integrated Immersive Inclusiveness' project. Supporting a full production of Red Earth Theatre's Soonchild would enable us to creatively demonstrate the potential of our aesthetically embedded captioning technology to new audiences and user communities. Those aims have been achieved and indeed exceeded as follows: 1. New audiences: 1,885 audience members (children aged 8+ and their families) experienced integrated immersive captioning for the first time in the touring show that this project enabled, made feasible by our development of a stable, user friendly, low equipment cost, open source projection mapping system. Audience numbers significantly exceeded our projected estimate of 1,000. Red Earth aim to use the immersive creative captioning system in their next planned show, The Red Tree. 2. Small scale theatre companies and venues across the UK: Companies and venues have been introduced to the projection mapping system, supported by a series of workshops and sessions to disseminate the system, guidelines and knowledge: a. The project's planned 'Access made easy' technical showcase day hosted at Arena Theatre Wolverhampton on 7 October was attended by 24 delegates including theatre makers, producers, venue co-ordinators, technical managers and designers. Feedback from participants was very positive: 'It was really well delivered and super useful. To watch the tech in action and then have the tech explained worked so well and answered so many of my questions.' b. We were invited to lead a workshop and demo on 'Creative Captioning' in December at the Lowry, Salford to production teams from the Lowry, Royal Exchange Theatre, Manchester International Festival, Contact Theatre and associate artists (18 attendees). We continue to work with the Lowry to look at routes for funding and supporting a model of mediated training and support for deployment of the system and toolkit and have been asked to lead training sessions at their 'Artist Network Training Session: Making Accessible Work' on 25 March 2020. c. We have met with several smaller companies interested in using the system and developing its deployment in new ways, including Lip Service, Threaded Music, Sinfonia Viva and venues including Lakeside Arts and Nottingham Playhouse, and will be supporting projects throughout 2020. The potential for venues to make it available to touring and associate companies is something that we had not foreseen and we will explore routes to enable us to support this development. d. The system, workshop and availability of support was publicised via ITC and through an article in The Stage -https://www.thestage.co.uk/news/2019/immersive-captioning-technology-developed-to-enhance-d-deaf-audiences-experience/ - which led to some of the contacts listed above, as the article noted that the technology 'will be offered free of charge to other theatre companies, alongside training on how to use it, once the tour has finished'. e. As a result of such varied contacts and contexts for deployment we are developing a more comprehensive suite of training materials which will be publicised further once complete. 3. Venues visited by Red Earth and Soonchild also benefitted from work led by CoI Mevel to develop accessible paratexts - marketing materials that explain the inclusive access approach of the show to audiences who may feel that theatre is not usually 'for them'. The Creative Captions video (https://www.youtube.com/watch?v=MjEo3UL_Zl4&feature=youtu.be) and the Soonchild trailer (https://www.youtube.com/watch?v=kwU83YqDFHI) have had over 2000 views between them; both aim to demonstrate the potential of inclusive and creative access strategies to potential audiences and help to support venues in marketing the show. The Creative Captions video will be usable by Red Earth and indeed by other companies and venues beyond the Soonchild tour and has been picked up by other bodies interested in inclusive access. The video was promoted very widely by the researchers, Red Earth, the Nottinghamshire Deaf Society and the University of Nottingham, on their website and via social media. The video has since been shown at Netflix headquarters in the United States, and at the 60th American Translators Association conference, as well as in national fora. The video has also been integrated into academic programmes on audiovisual translation in the Netherlands (Antwerp) and in the UK (Roehampton). Panthea (https://panthea.com/), the company which is the European leader of surtitling solutions for the performing arts, has expressed interest in finding out more about our ideas, and would like to integrate tools/add-ons for creating creative captions into their current suite of tools (they do both live environments and more traditional screen captions); we hope to develop this relationship further in the coming months. Our research is thus continuing to build on the achievements of our 'Integrated Immersive Inclusiveness' project in terms of impacting the cultural and creative sector at a significantly greater scale. By doing so the work is achieving societal impact in enabling access to theatre to be part of an immersive aesthetic experience, rather than an 'add-on' at the side of or above the stage, via simple and cheap technologies that are open to small theatre companies and venues.
First Year Of Impact 2019
Sector Creative Economy
Impact Types Cultural

 
Description Nottingham French Studies journal small research grants
Amount £400 (GBP)
Organisation University of Nottingham 
Department Department of Modern Languages and Cultures
Sector Academic/University
Country United Kingdom
Start 05/2019 
End 07/2019
 
Description Collaboration with Nottinghamshire Deaf Society 
Organisation Nottinghamshire Deaf Society
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution From the outset of the Integrated Immersive Inclusiveness award we worked with Nottinghamshire Deaf Society to include them as key stakeholders in our research. We have hosted members of the society at all three of the events run through the funding period - scoping event, hackfest and final showcase - and have worked in collaboration to develop immersive captions that respond to the needs of different members of the deaf community.
Collaborator Contribution Nottinghamshire Deaf Society committed the time of staff and members to support our research, including helping us to identify and work with volunteer participants at the sessions listed above. They have been enthusiastic advocates of our work within the deaf community and their members have played a key part in the development of our research and approach via participation in discussion and feedback on different projection and captioning styles. Their support has been important in developing the case for the Follow-on Funding project also.
Impact The collaboration has fed into the design and delivery of the technologies and guidelines for immersive captioning reported elsewhere in this form, and Nottinghamshire Deaf Society has also helped with dissemination of information through the Making Accessibility Accessible project.
Start Year 2018
 
Description Extended Collaboration with Red Earth Theatre 
Organisation Red Earth Theatre
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Building directly on the partnership inherent in the Integrated Immersive Inclusiveness project, we have extended our relationship from one of shared exploration of themes, to a full production partnership for their 2019 show Soonchild - the subject of the Making Accessibility Accessible follow on funding. We have worked with them to gather requirements for and prototype tools for embedding immersive captions in a theatrical setting, and then supported them to use those tools across a full national touring production.
Collaborator Contribution Red earth have supported both the original Integrated Immersive Inclusiveness and Making Accessibility Accessible follow on project with access to their production team and actors throughout. Working with them on Soonchild on the development and production of their has enabled us continue our collaborative research, and scale up our findings.
Impact Public events as reported elsewhere. Soonchild touring production visited 13 venues and seen by 1885 audience members.
Start Year 2017
 
Title Caption-o-matic - Projection Mapping Software 
Description A new piece of software which supports the process of projection mapping in theatre productions. The software uses pre-made 3D models of any and all set pieces which, when combined with a simple "real world" clicking interface, create a mapping of all the objects on stage, which can then have various media displayed on them using a single (or multiple) front projectors correctly respecting occlusion. A system is provided to send text (captions) to any part of the set with appropriate formatting metadata. An OSC-based networking system allows these projections to be controlled in realtime using common theatrical production software such as QLab. 
Type Of Technology Software 
Year Produced 2019 
Open Source License? Yes  
Impact A large number of theatre companies have been contacting us hoping to use the software for their own productions. Similarly, several venues and theatrical production groups have similarly contacted us hoping to use or promote the software. 
 
Description Access made easy technical showcase 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The project's planned 'Access made easy' technical showcase day hosted at Arena Theatre Wolverhampton on 7 October was attended by 24 delegates including theatre makers, producers, venue co-ordinators, technical managers and designers. The day involved a showing of Soonchild followed by a live demo of the projection mapping system and Q&A plus discussion on the benefits and challenges of immersive captioning. Several delegates contacted the project team subsequently to seek further advice and input and we are supporting a number of projects as a result.
Year(s) Of Engagement Activity 2019
 
Description Lowry creative captions workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Industry/Business
Results and Impact The project team was invited to participate in a workshop on creative captions and access at the Lowry, Salford. We demonstrated the projection mapping system and discussed its potential for deployment in a range of theatre and performance situations with representatives from the Lowry and other theatre companies and venues in the North West.
Year(s) Of Engagement Activity 2019
 
Description Presentation at Media4all 9 conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Over 100 people attended the presentation I gave at the Media4all9 conference which took place online in January 2021. The presentation on accessible paratext sparked a vibrant discussion on the conference's forum, in particular with regards to its overlap with creative captioning. The presentation was recorded and is now available on Youtube, where it has 79 views (https://www.youtube.com/watch?v=etzUIsM815U)
Year(s) Of Engagement Activity 2021
URL https://jornades.uab.cat/media4all9/
 
Description Presentation at Media4all8 Conference (Stockholm) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Over 50 people attended the presentation I gave at the Media4all8 conference in Stockholm in June 2019. The presentation sparked a vibrant discussion on the technology involved to produce captions in live environment, its overlap with captioning for screens, and generally accessible practices in the media. As a result of this presentation, I have been invited to give talks at Manchester and Bristol universities.
Year(s) Of Engagement Activity 2019
URL https://www.tolk.su.se/polopoly_fs/1.427722.1560533745!/menu/standard/file/M4A8%20Programme.pdf
 
Description Presentation at Open University Translation Talk Series 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact I was invited to give a presentation for the Open University Translation Talk Series. The presentation focussed on Accessible Paratext and on actively engaging with D/deaf audiences in audiovisual media. The presentation was very well attended by the current cohort of translation postgraduate students at the Open University as well as by a number of members of staff. The event took place online because of current limitations related to Covid. It nonetheless gave rise to an interesting and lively discussion afterwards.
Year(s) Of Engagement Activity 2020
 
Description Presentation at University of Bristol 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact I gave a presentation at the University of Bristol in November 2019 which sparked a lively conversation about captioning in live environments, as well as future technical and epistemological developments. The presentation was live-tweeted which provided increased visibility.
Year(s) Of Engagement Activity 2019
URL https://www.bristol.ac.uk/sml/events/eventtranslationseminar.html
 
Description Presentation at University of Leicester LeCTIS seminar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Postgraduate students
Results and Impact I was invited to give a talk as part of of the University of Leicester Centre for Translation LeCTIS series. The talk took place on 3rd February on MSTeams and was attended by postgraduate students, colleagues who work in the area of Translation Studies, and professional members of the East Midlands Regional group of the Institute of Translators and Interpreters.
Year(s) Of Engagement Activity 2022
 
Description Symposium: Staging the Voice, Voicing the Stage 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Robinson led a workshop panel as part of a symposium hosted by Newcastle University and Opera North: Stage(text)s: integrating surtitles and captioning

Playwright and dramaturge Kaite O'Reilly characterises 'surtitling/captioning as an equal, parallel text, rather than side text'. In this conversation we are interested in exploring the relationships between that 'parallel' text, the performing/speaking/singing body and the dramaturgy and scenography of performance. How can integrated aesthetic and creative captioning offer new opportunities for those relationships and for the performance/artwork as a whole? How can captions be creative, inclusive and enabling for all? What are the challenges?

Contributors alongside Robinson were:
Stuart Leeks, Editor, Opera North
Alice Gilmour, Community Partnerships Manager, Opera North
Amy Payne, Mezzo Soprano, Opera North
Year(s) Of Engagement Activity 2021
 
Description Talk at ERDF Lead:NG Launch event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Discussing our work as an inspirational example to SMEs of creative use of immersive technology.
Year(s) Of Engagement Activity 2021
 
Description Talk for University of Wuhan (China) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact I was invited to give a talk as part of the Huazhong Agricultural University of Wuhan University series of talks on translation and media accessibility. The talk took place on 3rd December 2023 on Zoom and was attended by postgraduate students, and colleagues who work in the area of Translation Studies.
Year(s) Of Engagement Activity 2022
 
Description Talk to Bristol Old Vic 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact Working with practitioners to consider how to apply our knowledge to a large scale theatrical output.
Year(s) Of Engagement Activity 2021