Sensing Music Interactions from the Outside-In: Accessible Innovation Fusing Wearable Technology and Physical Prototyping

This fellowship will critically examine longstanding assumptions of digital musical instrument (DMI) development and create new methods for co-designing customised instruments with physically impaired and non-technical musicians. The aim is to develop disruptive, speculative, and inclusive methods that reshape the fundamental practice of DMI design, inviting broader participation in the development of new musical instruments and future visions of musicianship.

Musical instruments are rich cultural and technical artifacts that are used to translate human motion into sound. Embedded computation has led to an abundance of new DMIs, inspiring novel forms of music innovation in terms of performance and artistic expression. DMI development is a highly skilled interdisciplinary craft, combining music, engineering, manufacturing and ergonomics. Consequently, modern instrument design is exclusionary, and instruments are immutably ingrained with the cultural and technical assumptions of their creators, vastly reducing their potential for mass adoption and creating barriers to music innovation for non-technical and physically impaired musicians. Motion capture and gestural interaction technologies have shifted the locus of interaction away from physical objects and onto the body, realising human movement as the interface for performing music. However, the development of musical objects and gestural music interaction are often considered and pursued separately. By focusing on musical objects, instrument development is framed as a technical challenge assuming normative players and overlooking the diversity of musicians, audiences, and their environments. Gestural music interaction has shown great potential for music performance and accessibility; however, it neglects the significant role of tactile feedback in music training and the development of virtuosic performance.

This fellowship will take a holistic approach, coupling gestural interaction and physical prototyping. The approach departs from the conventional practice of embedding sensing technologies within an instrument, and exploits a new, emerging paradigm that relocates the technology onto the body, sensing player interactions using wearable devices on the wrist and hand. A radical switch in both technical and design approach that opens innovative new research directions for low-cost, rapidly produced musical instruments, which can be designed to have any shape, scale or structure. Crucially, this provides new ways for people to participate in the rapid co-design of novel, customised musical instruments that are tailored to their unique access requirements and artistic identity.

The fellowship addresses the UKRI priority areas of engineering, technology, AI, design research and the AHRC's Audience of the Future Challenge. The PI is ideally placed to examine this timely research opportunity, building on an extensive academic and entrepreneurial track record in gestural music systems. The Co-I and mentors are world leaders in their respective fields and strategic partnerships provide key accessibility guidance, links with Disabled communities and pathways for future impact. UWE, the host institution, plays a significant role in the West of England's (WoE) thriving creative industries and represents the ideal environment for this research. The fellowship will have the full support of the Centre for Print Research (CfPR), and will be located within The Bridge: a new £2.3M AHRC funded facility created to explore artistic applications of physical prototyping and interactive technologies (opening 2023).

The fellowship supports time for the PI, Co-I and two postdoctoral research associates (RAs). The Co-I will support the development of embedded wearable technology (WT). One RA will explore participatory methods for co-designing customised musical interfaces. The other RA will focus on signal processing techniques to sense interactions with musical objects.