Design Patterns for Inclusive Collaboration (DePIC)

Lead Research Organisation: Goldsmiths University of London
Department Name: Computing Department

Abstract

Abstracts are not currently available in GtR for all funded research. This is normally because the abstract was not required at the time of proposal submission, but may be because it included sensitive information such as personal details.
 
Description We worked with visually impaired users in a participatory manner to understand how visual interfaces could be translated to tactile interfaces. This resulted in a novel device that allowed users to feel sound.
Exploitation Route These results have been made available (hardware designs, software, methodology) in ways that will allow designers to create interfaces for the visually impaired, and in general to leverage cross-modal interaction for all users.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Other
URL http://eavi.goldsmithsdigital.com/?s=hapticwave
 
Description The research resulted in the elaboration of a device, called the HapticWave which was manufactured in a small production run of 3 units. It is a robust device with a high level of design and finish that has been deployed in real world settings in the UK and the US. It has been used in professional audio recording studio settings by 6 users. It has allowed the disabled community, notably visually impaired audio engineers, to be as productive as their sighted colleagues in the workplace. It has been covered by NBC television news in the USA. The Haptic Wave is a device allowing blind and visually impaired musicians to work in audio production studios to edit digital audio in the haptic domain without being able to see the standard GUI audio waveform. It was developed over the course of the 4 year, EPSRC funded Design Patterns for Inclusive Collaboration (DePIC) project. Four working units were designed, built, and deployed in real-world settings in Europe and the USA. It has had impact in gaining awards, being covered in the US television news, and most importantly, in the lives and work of the end-users with whom we worked. The main output submitted on this project is the peer reviewed journal article, Oussama Metatla, Fiore Martin, Adam Parkinson, Nick Bryan-Kinns, Tony Stockman, and Atau Tanaka, "Audio-Haptic Interfaces for Digital Audio Workstations," Journal on Multimodal User Interfaces, 2016, 1-12. The project has also yielded high level conference papers and book chapters. The first paper listed below was awarded a Best Paper Award at ACM SIGCHI. Tanaka, A., Parkinson, A. (2016) "Haptic Wave: A Cross-Modal Interface for Visually Impaired Audio Producers." Proceedings ACM conference on Human factors in computing systems. ACM-CHI, 2016 [Best Paper Award]. p. 2150-2161 Parkinson, A., Cameron, D., Tanaka, A. "HapticWave: Presenting the multiple voices, artefacts and materials of a design research project." In Research Through Design. Cambridge UK 2015. A. Tanaka and A. Parkinson, "The Problems with Participation." In Simon Emmerson (ed.), The Routledge Research Companion to Electronic Music: Reaching Out with Technology. Routledge (2018). A. Tanaka. "Embodied Musical Interaction: Body physiology, cross modality, and sonic experience" In S. Holland, M Wanderley, A. McPherson, K. Wilkie (ed.), Music and HCI. Springer (In Press). The working device has been presented by invitation at the Royal Society of Engineering, and at the International Blind Musicians conference. The haptic technology developed for the project became one of the technologies made available to industry in the EU technology-transfer project, the Horizon2020 Innovation Action, Rapid-Mix. This was further picked up by leading music industry hardware/software company, Native Instruments in a new research proposal to the European Commission. Four units were produced and deployed in real world settings for up to 6 months. These settings included home studios in London, to professional music production studios in Macon Georgia, to research labs at the University of Michigan in the USA. The device was used by professional audio producers who happened to be visually impaired, in the production of mainstream audio books, soundtracks, and country music recordings. The use by blind country musician, Joey Stuckey, in Macon Georgia was covered by US television news on the NBC broadcasting network. Finally, the group of 11 blind and visually impaired musician/producers we brought together for the project founded an Internet mailing list, the AIM list to discuss accessibility and their integration into the professional music industry workplace. Whilst the original aim of the mailing list was to discuss the planning of workshop sessions for the Haptic Wave project, the mailing list continues to this day after the project as a forum for blind and visually impaired musicians to trade tips on working with computer-based music production tools.
First Year Of Impact 2014
Sector Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Other
Impact Types Cultural,Societal
 
Description Horizion 2020 Innovation Action, ICT-22-2014: Multimodal and Natural computer interaction
Amount € 2,682,586 (EUR)
Funding ID 644862 
Organisation European Commission 
Sector Public
Country European Union (EU)
Start 02/2015 
End 01/2018
 
Title Haptic Wave 
Description We have developed the Haptic Wave, a piece of audio hardware with accompanying software that enables audio engineers with visual impairments to "feel" sound and use this tactility to complete audio editing tasks. The software can run as a cross platform piece of audio editing software, or be launched as an editor from many popular software environments used by audio engineers such as Reaper, Logic or Ableton Live. 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2014 
Impact The device has appeared on TV media in the US: METRO: https://www.youtube.com/watch?v=z-qOvmg-RYI WGXA TV: http://www.wgxa.tv/news/local/Technology-allowing-blind-musicians-317645691.html 41 NBC: http://www.41nbc.com/story/d/story/new-technology-helps-blind-musicians-produce/11378/EsZzsqH5rEGZCwiVPN9k7w The device has been used in the following studios: RNIB (Royal National Institute of Blind people) Talking Books Studios, Camden, London, UK Shadow Sound Studios, Macon, Georgia, US Alongside a variety of professional home studios. We have demonstrated the device at: NESTA SoundLab Demo, London, November 2015 Visually Impaired Musicians' Lives Conference, Institute of Education, University of London, March 2015 Research Through Design (RTD) Conference, Microsoft Research, Cambridge, March 2015 
URL http://eavi.goldsmithsdigital.com/projects/depic/
 
Description AccessNIME Workshop at NIME (New Interfaces for Musical Expression) Conference, Goldsmiths, 2014 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact We organised a workshop exploring how technologies that enable new modes of musical expression can be used to increase accessibility in music making.
Year(s) Of Engagement Activity 2014
URL http://depic.eecs.qmul.ac.uk/accessnime/
 
Description Haptic Wave at Visually-impaired Musicians' Lives 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We presented the Haptic Wave at the Visually-impaired musicians' lives conference at University College London
Year(s) Of Engagement Activity 2015
URL http://vimusicians.ioe.ac.uk/confer_train.html
 
Description Innovation in Haptics presentation at Royal Academy of Engineering 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact We were invited to present the Haptic Wave at the Royal Academy of Engineering, "Innovation in Haptics"
Year(s) Of Engagement Activity 2017
URL https://www.raeng.org.uk/events/events-programme/2017/july/innovation-in-haptics
 
Description NESTA SoundLab Demo 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact We demonstrated the Haptic Wave at an open day event at Nesta / Soundlab which showcased a series of tools for musicians with different impairments and abilities.
Year(s) Of Engagement Activity 2015
 
Description Participatory Design Workshop (series of 4 workshops and studio trials) 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We explored the problems facing audio engineers with visual impairments through Participatory Design workshops and used a further workshops and "real world" studio trials to design and refine a tool to address some of these problems.

One further outcome was that the initial workshop assisted Peter Bosher in setting up the "AIMS" mailing list / community, an active online group for audio engineers with visual impairments.
Year(s) Of Engagement Activity 2013,2014,2015
URL http://eavi.goldsmithsdigital.com/projects/depic/
 
Description Professional Studio Trials of Haptic Wave 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The Haptic Wave was trialled in a series of professional and home recording studios in Summer 2015. These included:
-Shadow Sound (Georgia, USA)
-RNIB (Royal National Institute of Blind People) Talking Books Studios (London, UK)
-professional home studios of study participants in London and Essex.
Year(s) Of Engagement Activity 2015
URL http://eavi.goldsmithsdigital.com/blog/hapticwave-in-the-us/