Interactive performance for musicians with a hearing impairment
Lead Research Organisation:
University of Liverpool
Department Name: Architecture
Abstract
Musicians with a hearing impairment often use the vibration of their instrument, or the surface to which it is connected, to help them feel the sound that they create. Well-known musicians who have used this approach include Beethoven with the piano and Dame Evelyn Glennie with percussion instruments. There are different degrees of hearing impairment so in some cases the sound may still be perceptible to the musician. However, profoundly deaf musicians tend to rely on vibration. This can limit the type of musical instrument that can be played and the other musicians with whom they can play. When musicians with normal hearing play together they have access to a wide range of cues that facilitate musical interaction. These include the sound from their own instrument and other instruments, as well as visual or verbal contact with other musicians or a conductor. During a group performance, hearing-impaired musicians don't have access to all of these cues. This significantly limits the potential for musical collaboration.
This interdisciplinary research in the arts and humanities reaches out to the science of sound and vibration. It is concerned with understanding, enabling and enhancing musical interaction between hearing impaired musicians playing classical, pop, jazz, folk, or any other kind of music. It seeks an understanding of the musical cues needed by hearing-impaired musicians to facilitate group performance, and a technological solution based on vibration to provide the required cues. The research will look at the potential for using individual vibrating performance decks on which each musician can stand or sit, as well as vibration pads that can be attached to the body, and arrays of vibrating bars that transmit vibration signals from the different instruments. The research will give insight into how vibration signals can be tailored to suit different musicians and different types of music. In most cases the vibration signal will be a combination of the sound from other instruments. Sometimes this will include an electronic timing device such as a metronome. For improvisation sessions, the vibration of the deck/pads/bars might also be used to attract the attention of other musicians, or to facilitate communication between musicians when sign language or lip-reading during a performance is not possible.
Research into the tactile perception of music in this proposal makes use of art- and psychology-based insights into musical performance combined with a science-based insight into vibration. The research outcomes will be relevant to current and future generations of hearing impaired musicians and allow them to create music and collaborate interactively with a wide range of musicians. It will also open up pedagogic possibilities for teaching music to hearing impaired children. For charitable organisations, the research will help develop new strategies for music education as well as integration of the hearing impaired into mainstream education.
This interdisciplinary research in the arts and humanities reaches out to the science of sound and vibration. It is concerned with understanding, enabling and enhancing musical interaction between hearing impaired musicians playing classical, pop, jazz, folk, or any other kind of music. It seeks an understanding of the musical cues needed by hearing-impaired musicians to facilitate group performance, and a technological solution based on vibration to provide the required cues. The research will look at the potential for using individual vibrating performance decks on which each musician can stand or sit, as well as vibration pads that can be attached to the body, and arrays of vibrating bars that transmit vibration signals from the different instruments. The research will give insight into how vibration signals can be tailored to suit different musicians and different types of music. In most cases the vibration signal will be a combination of the sound from other instruments. Sometimes this will include an electronic timing device such as a metronome. For improvisation sessions, the vibration of the deck/pads/bars might also be used to attract the attention of other musicians, or to facilitate communication between musicians when sign language or lip-reading during a performance is not possible.
Research into the tactile perception of music in this proposal makes use of art- and psychology-based insights into musical performance combined with a science-based insight into vibration. The research outcomes will be relevant to current and future generations of hearing impaired musicians and allow them to create music and collaborate interactively with a wide range of musicians. It will also open up pedagogic possibilities for teaching music to hearing impaired children. For charitable organisations, the research will help develop new strategies for music education as well as integration of the hearing impaired into mainstream education.
Planned Impact
The outcomes of this research will benefit current and future generations of hearing impaired musicians. It will help them to create music and collaborate with a wide range of hearing and hearing impaired musicians. This will enhance quality of life, musical creativity and musical performance.
This research has the potential to have a positive impact on music education. In schools for the deaf and for charitable organisations such as Music and the Deaf it will open up possibilities for the teaching of music to hearing impaired children. The findings will therefore help in the development of new strategies for music education as well as the integration of hearing impaired children into mainstream education.
Researchers in the field of music psychology will gain greater insight into the collaborative performance of music for both hearing and hearing-impaired musicians. Acousticians, musicians and sonic artists will benefit from the development of novel methods for converting music to a vibration signal and transmitting it to the human body. In the future it is hoped that this will inspire artists to consider and exploit the medium of vibration in communication and the arts using a range of media, including music.
Dissemination and communication of the project findings will take place through conference papers, a one-day meeting at RNCM, academic journal papers, magazine articles, media interviews and demonstrations. These give the opportunity to disseminate the project findings to musicians, charities, researchers and artists. They also allow discussion on implementation of the research, directions for future research, and possibilities for commercial production of the technological solutions.
All the above benefits and outcomes of this research relevant to hearing impaired musicians, music psychology, music education, art and creativity are realisable within the next decade.
This research has the potential to have a positive impact on music education. In schools for the deaf and for charitable organisations such as Music and the Deaf it will open up possibilities for the teaching of music to hearing impaired children. The findings will therefore help in the development of new strategies for music education as well as the integration of hearing impaired children into mainstream education.
Researchers in the field of music psychology will gain greater insight into the collaborative performance of music for both hearing and hearing-impaired musicians. Acousticians, musicians and sonic artists will benefit from the development of novel methods for converting music to a vibration signal and transmitting it to the human body. In the future it is hoped that this will inspire artists to consider and exploit the medium of vibration in communication and the arts using a range of media, including music.
Dissemination and communication of the project findings will take place through conference papers, a one-day meeting at RNCM, academic journal papers, magazine articles, media interviews and demonstrations. These give the opportunity to disseminate the project findings to musicians, charities, researchers and artists. They also allow discussion on implementation of the research, directions for future research, and possibilities for commercial production of the technological solutions.
All the above benefits and outcomes of this research relevant to hearing impaired musicians, music psychology, music education, art and creativity are realisable within the next decade.
Publications
Fulford R
(2011)
Learning not to listen: the experiences of musicians with hearing impairments
in Music Education Research
Mate-Cid, S
(2012)
Perception and learning of relative pitch in the vibrotactile mode
in LIPAM 2012
Mate-Cid, S
(2012)
Establishing vibrotactile thresholds on the fingertip for people with and without a hearing impairment over the musical frequency range between C1 and C6
in International Congress on Sound and Vibration (ICSV19)
Hopkins, C
(2013)
Inherent and learnt abilities for relative pitch in the vibrotactile domain using the fingertip
in International Congress on Sound and Vibration (ICSV20)
Hopkins, C
(2014)
Interactive performance for musicians with hearing impairments using the vibrotactile mode
in Institute of Acoustics 40th Anniversary conference
Ginsborg, J
(2014)
Interactive performance: Towards the use of vibrotactile technology by musicians with hearing impairments
in International Symposium on Performance Science 2013 - European Association of Conservatoires
Fulford, R
(2014)
Sight, sound and synchrony: effects of attenuating auditory information on duo violinists behaviours in performance
in International Conference on Music Perception and Cognition
Title | Evidence on how vibrotactile technology has successfuly been used to support d/Deaf people in music education at the Royal School for the Deaf Derby |
Description | This video documents the responses from teacher, educators and children at Royal School for the Deaf Derby after successfully using vibrotactile stimuli in music education with d/Deaf children. |
Type Of Art | Film/Video/Animation |
Year Produced | 2020 |
Impact | The next step is to use the video to stimulate interest from other schools for the deaf and to help source philanthropic and charity funding to buy equipment for these schools. |
URL | https://stream.liv.ac.uk/2qvwd9th |
Description | We have (a) identified key aspects that facilitate interactive performance between musicians with a hearing-impairment such as non-audio cues and (b) established the fundamental requirements needed for musicians with a hearing impairment to play together using vibration (i.e. using the vibrotactile mode). |
Exploitation Route | Could be used in other applications such as smartphones, cars etc - anywhere where information might usefully be transmitted using the vibrotactile mode. |
Sectors | Digital/Communication/Information Technologies (including Software) Education Electronics Leisure Activities including Sports Recreation and Tourism |
URL | https://stream.liv.ac.uk/kgfymdz4 |
Description | We are using the University of Liverpool HEIF impact acceleration funding to loan the vibrotactile technology for music lessons in primary and secondary schools. This work is only just starting now (Feb 2018), so it is at an early stage. |
First Year Of Impact | 2018 |
Sector | Education |
Impact Types | Societal |
Description | HEIF impact acceleration funding |
Amount | £9,200 (GBP) |
Organisation | University of Liverpool |
Sector | Academic/University |
Country | United Kingdom |
Start | 01/2018 |
End | 06/2018 |
Description | Partnership with Royal School for the Deaf Derby on using vibration for music education |
Organisation | Royal School for the Deaf Derby |
Country | United Kingdom |
Sector | Academic/University |
PI Contribution | We loaned vibrotactile equipment to the music department at Royal School for the Deaf, Derby. This led to its use in lessons which we filmed with UKAN funds to disseminate the impact of the vibrotactile feedback on music education at Royal School for the Deaf, Derby. |
Collaborator Contribution | Time in-kind to set up the equipment at RSDD and plan it's use in lessons as well as provide feedback to researchers. |
Impact | Video providing evidence of successful use of vibrotactile feedback in music lessons for the deaf. |
Start Year | 2018 |
Description | Creation of "Musical Vibrations" website |
Form Of Engagement Activity | Engagement focused website, blog or social media channel |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | The work on vibrotactile presentation of music for established and aspiring d/Deaf musicians has been branded as "Musical Vibrations" with its own website (www.musicalvibrations.com). Our website is being used to address two main routes to use the vibrotactile technology: (1) in schools for teaching music, (2) live music. Schools: We are now inviting schools and resource bases for deaf children to participate in a new research study looking at how vibrotactile feedback can contribute to the effective teaching and learning of music in the school classroom. The opinions of pupils, music teachers and teaching and learning support staff are being sought and we expect to work with up to 180 pupils over the course of the study. Live music: We want to make vibrotactile equipment available as standard at concert venues where live music is performed and, eventually, in the home. The website is being used to reach out to the live music industry. |
Year(s) Of Engagement Activity | 2018,2019 |
URL | http://www.musicalvibrations.com/ |
Description | Internet video for the project providing 'proof of principle' |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | Yes |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | 'Proof of principle' to show the potential for vibrotactile technology, this video shows three musicians playing 'Day Tripper' (The Beatles) in the anechoic chambers of the Acoustics Research Unit at Liverpool. They are playing using vibrotactile feedback without any visual or auditory feedback. Uploaded to https://stream.liv.ac.uk/kgfymdz4 http://www.youtube.com/watch?v=uaun3bYkYgo 'Proof of principle' to show the potential for vibrotactile technology, this video shows three musicians playing 'Day Tripper' (The Beatles) in the anechoic chambers of the Acoustics Research Unit at Liverpool. They are playing using vibrotactile feedback |
Year(s) Of Engagement Activity | 2013 |
Description | Making music with a hearing impairment |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | Yes |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Making music with a hearing impairment |
Year(s) Of Engagement Activity | 2012 |
Description | Radio interview |
Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
Part Of Official Scheme? | Yes |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | 20 minute radio interview on the AHRC project for National Radio New Zealand Listen again at... http://www.radionz.co.nz/national/programmes/nights/audio/2558569/vibrotactile-technology |
Year(s) Of Engagement Activity | 2013 |
Description | The role of music in coping with sensory impairment |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | Yes |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | The role of music in coping with sensory impairment |
Year(s) Of Engagement Activity | 2012 |
Description | Uses and experiences of hearing aid technology by musicians with a hearing impairment |
Form Of Engagement Activity | A talk or presentation |
Part Of Official Scheme? | Yes |
Geographic Reach | National |
Primary Audience | Public/other audiences |
Results and Impact | Uses and experiences of hearing aid technology by musicians with a hearing impairment |
Year(s) Of Engagement Activity | 2013 |