Interactive performance for musicians with a hearing impairment

Lead Research Organisation: University of Liverpool
Department Name: Architecture

Abstract

Musicians with a hearing impairment often use the vibration of their instrument, or the surface to which it is connected, to help them feel the sound that they create. Well-known musicians who have used this approach include Beethoven with the piano and Dame Evelyn Glennie with percussion instruments. There are different degrees of hearing impairment so in some cases the sound may still be perceptible to the musician. However, profoundly deaf musicians tend to rely on vibration. This can limit the type of musical instrument that can be played and the other musicians with whom they can play. When musicians with normal hearing play together they have access to a wide range of cues that facilitate musical interaction. These include the sound from their own instrument and other instruments, as well as visual or verbal contact with other musicians or a conductor. During a group performance, hearing-impaired musicians don't have access to all of these cues. This significantly limits the potential for musical collaboration.

This interdisciplinary research in the arts and humanities reaches out to the science of sound and vibration. It is concerned with understanding, enabling and enhancing musical interaction between hearing impaired musicians playing classical, pop, jazz, folk, or any other kind of music. It seeks an understanding of the musical cues needed by hearing-impaired musicians to facilitate group performance, and a technological solution based on vibration to provide the required cues. The research will look at the potential for using individual vibrating performance decks on which each musician can stand or sit, as well as vibration pads that can be attached to the body, and arrays of vibrating bars that transmit vibration signals from the different instruments. The research will give insight into how vibration signals can be tailored to suit different musicians and different types of music. In most cases the vibration signal will be a combination of the sound from other instruments. Sometimes this will include an electronic timing device such as a metronome. For improvisation sessions, the vibration of the deck/pads/bars might also be used to attract the attention of other musicians, or to facilitate communication between musicians when sign language or lip-reading during a performance is not possible.

Research into the tactile perception of music in this proposal makes use of art- and psychology-based insights into musical performance combined with a science-based insight into vibration. The research outcomes will be relevant to current and future generations of hearing impaired musicians and allow them to create music and collaborate interactively with a wide range of musicians. It will also open up pedagogic possibilities for teaching music to hearing impaired children. For charitable organisations, the research will help develop new strategies for music education as well as integration of the hearing impaired into mainstream education.

Planned Impact

The outcomes of this research will benefit current and future generations of hearing impaired musicians. It will help them to create music and collaborate with a wide range of hearing and hearing impaired musicians. This will enhance quality of life, musical creativity and musical performance.

This research has the potential to have a positive impact on music education. In schools for the deaf and for charitable organisations such as Music and the Deaf it will open up possibilities for the teaching of music to hearing impaired children. The findings will therefore help in the development of new strategies for music education as well as the integration of hearing impaired children into mainstream education.

Researchers in the field of music psychology will gain greater insight into the collaborative performance of music for both hearing and hearing-impaired musicians. Acousticians, musicians and sonic artists will benefit from the development of novel methods for converting music to a vibration signal and transmitting it to the human body. In the future it is hoped that this will inspire artists to consider and exploit the medium of vibration in communication and the arts using a range of media, including music.

Dissemination and communication of the project findings will take place through conference papers, a one-day meeting at RNCM, academic journal papers, magazine articles, media interviews and demonstrations. These give the opportunity to disseminate the project findings to musicians, charities, researchers and artists. They also allow discussion on implementation of the research, directions for future research, and possibilities for commercial production of the technological solutions.

All the above benefits and outcomes of this research relevant to hearing impaired musicians, music psychology, music education, art and creativity are realisable within the next decade.

Publications

10 25 50
publication icon
Hopkins C (2022) Deafness And Music: Can Vibration Be Used When Playing Music Together? in Frontiers for Young Minds

publication icon
Hopkins, C (2013) Inherent and learnt abilities for relative pitch in the vibrotactile domain using the fingertip in International Congress on Sound and Vibration (ICSV20)

publication icon
Hopkins, C (2014) Interactive performance for musicians with hearing impairments using the vibrotactile mode in Institute of Acoustics 40th Anniversary conference

publication icon
Ginsborg, J (2014) Interactive performance: Towards the use of vibrotactile technology by musicians with hearing impairments in International Symposium on Performance Science 2013 - European Association of Conservatoires

 
Title Evidence on how vibrotactile technology has successfuly been used to support d/Deaf people in music education at the Royal School for the Deaf Derby 
Description This video documents the responses from teacher, educators and children at Royal School for the Deaf Derby after successfully using vibrotactile stimuli in music education with d/Deaf children. 
Type Of Art Film/Video/Animation 
Year Produced 2020 
Impact The next step is to use the video to stimulate interest from other schools for the deaf and to help source philanthropic and charity funding to buy equipment for these schools. 
URL https://stream.liv.ac.uk/2qvwd9th
 
Description We have (a) identified key aspects that facilitate interactive performance between musicians with a hearing-impairment such as non-audio cues and (b) established the fundamental requirements needed for musicians with a hearing impairment to play together using vibration (i.e. using the vibrotactile mode).
Exploitation Route Could be used in other applications such as smartphones, cars etc - anywhere where information might usefully be transmitted using the vibrotactile mode.
Sectors Digital/Communication/Information Technologies (including Software),Education,Electronics,Leisure Activities, including Sports, Recreation and Tourism

URL https://stream.liv.ac.uk/kgfymdz4
 
Description We are using the University of Liverpool HEIF impact acceleration funding to loan the vibrotactile technology for music lessons in primary and secondary schools. This work is only just starting now (Feb 2018), so it is at an early stage.
First Year Of Impact 2018
Sector Education
Impact Types Societal

 
Description HEIF impact acceleration funding
Amount £9,200 (GBP)
Organisation University of Liverpool 
Sector Academic/University
Country United Kingdom
Start 01/2018 
End 06/2018
 
Description Partnership with Royal School for the Deaf Derby on using vibration for music education 
Organisation Royal School for the Deaf Derby
Country United Kingdom 
Sector Academic/University 
PI Contribution We loaned vibrotactile equipment to the music department at Royal School for the Deaf, Derby. This led to its use in lessons which we filmed with UKAN funds to disseminate the impact of the vibrotactile feedback on music education at Royal School for the Deaf, Derby.
Collaborator Contribution Time in-kind to set up the equipment at RSDD and plan it's use in lessons as well as provide feedback to researchers.
Impact Video providing evidence of successful use of vibrotactile feedback in music lessons for the deaf.
Start Year 2018
 
Description Creation of "Musical Vibrations" website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The work on vibrotactile presentation of music for established and aspiring d/Deaf musicians has been branded as "Musical Vibrations" with its own website (www.musicalvibrations.com).

Our website is being used to address two main routes to use the vibrotactile technology: (1) in schools for teaching music, (2) live music.

Schools:
We are now inviting schools and resource bases for deaf children to participate in a new research study looking at how vibrotactile feedback can contribute to the effective teaching and learning of music in the school classroom. The opinions of pupils, music teachers and teaching and learning support staff are being sought and we expect to work with up to 180 pupils over the course of the study.

Live music:
We want to make vibrotactile equipment available as standard at concert venues where live music is performed and, eventually, in the home. The website is being used to reach out to the live music industry.
Year(s) Of Engagement Activity 2018,2019
URL http://www.musicalvibrations.com/
 
Description Internet video for the project providing 'proof of principle' 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? Yes
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact 'Proof of principle' to show the potential for vibrotactile technology, this video shows three musicians playing 'Day Tripper' (The Beatles) in the anechoic chambers of the Acoustics Research Unit at Liverpool. They are playing using vibrotactile feedback without any visual or auditory feedback.



Uploaded to

https://stream.liv.ac.uk/kgfymdz4

http://www.youtube.com/watch?v=uaun3bYkYgo

'Proof of principle' to show the potential for vibrotactile technology, this video shows three musicians playing 'Day Tripper' (The Beatles) in the anechoic chambers of the Acoustics Research Unit at Liverpool. They are playing using vibrotactile feedback
Year(s) Of Engagement Activity 2013
 
Description Making music with a hearing impairment 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Making music with a hearing impairment
Year(s) Of Engagement Activity 2012
 
Description Radio interview 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? Yes
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact 20 minute radio interview on the AHRC project for National Radio New Zealand

Listen again at...

http://www.radionz.co.nz/national/programmes/nights/audio/2558569/vibrotactile-technology
Year(s) Of Engagement Activity 2013
 
Description The role of music in coping with sensory impairment 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The role of music in coping with sensory impairment
Year(s) Of Engagement Activity 2012
 
Description Uses and experiences of hearing aid technology by musicians with a hearing impairment 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? Yes
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Uses and experiences of hearing aid technology by musicians with a hearing impairment
Year(s) Of Engagement Activity 2013