RUDIMENTS: Reflective Understanding of Digital Instruments as Musical Entanglements
Lead Research Organisation:
Imperial College London
Department Name: Design Engineering (Dyson School)
Abstract
Many new digital musical instruments (DMIs) are created every year, each of which encodes cultural values both obvious and subtle, reflecting the aesthetic priorities of its designer, its constituent technologies, and those technologies' domains of origin. Recent developments in high-performance, low-cost embedded computing promise new frontiers in machine intelligence to be integrated into instruments. The musical opportunities are vast, but so are the risks: who will set the agenda for deploying this technology, and whose interests and aesthetics will be represented? RUDIMENTS takes a value-sensitive approach to DMI design, using embedded computing (miniaturised computers integrated into physical objects) to support rich sensory experience rather than analytical symbolic intelligence. This interdisciplinary project seeks to reconcile technical and ecological perspectives of DMI research, the former focused on engineering techniques or performer-instrument interaction, the latter on the interplay of environmental factors which shape musical creation. The technical perspective can prioritise analytical concepts over the inexpressibility of human experience, while the ecological perspective offers few practical suggestions for technologists. RUDIMENTS revisits the foundational assumptions of DMI research, stripping away layers of historical technical shortcuts and music theory to return to the fundamentals -- the rudiments -- of musical phenomena. The project proposes that instruments should not be viewed as self-contained technological objects at all, but as entanglements: complex and irreducible webs of relationships between humans and things. Musical entanglement design will provide a far-reaching new technical-artistic basis on which to explore the transformational potential of embedded computing and artificial intelligence, building instruments bottom-up from pre-reflective sensory experience and considering the cultural implications of every design decision.
People |
ORCID iD |
| Andrew McPherson (Principal Investigator) |
Publications
Caspe F
(2023)
FM Tone Transfer with Envelope Learning
Hayes B
(2024)
A review of differentiable digital signal processing for music and speech synthesis
in Frontiers in Signal Processing
Martinez Avila J
(2023)
The Stretchy Strap: supporting encumbered interaction with guitars
in Journal of New Music Research
| Title | Arca performs with the magnetic resonator piano |
| Description | Arca, a leading electronic musician and multidisciplinary artist, presented a series of concerts featuring the magnetic resonator piano (MRP), an augmented instrument created in my research. Four performances took place at the Armory in New York City in October 2023, two performances at the Pinault Collection in Paris in March 2024, a performance at the Sydney Opera House in May 2024. Collectively the audience for these shows was in the thousands to tens of thousands, accompanied by several high-profile media pieces (New York Times, Rolling Stone, etc.) and considerable attention on social media. |
| Type Of Art | Performance (Music, Dance, Drama, etc) |
| Year Produced | 2024 |
| Impact | The shows drew considerable attention to the magnetic resonator piano at the research in the Augmented Instruments Laboratory. The shows in Paris, where the magnetic resonator piano was a focal point of the performance, sold out within minutes. |
| URL | https://www.pinaultcollection.com/en/boursedecommerce/arca-presents-light-comes-name-voice |
| Title | Future Sounds: Classical Instruments Meet Digital Tech |
| Description | On 15 June 2024, I presented a talk and performance at the Great Exhibition Road Festival in South Kensington, London, entitled "Future Sounds: Classical Instruments Meet Digital Tech". The event included performances on new instruments from the Augmented Instruments Lab, performed by an improvising student ensemble from the Royal College of Music (coached by RCM head of improvisation Gerardo Gozzi). |
| Type Of Art | Performance (Music, Dance, Drama, etc) |
| Year Produced | 2024 |
| Impact | The event was in a large lecture theatre space and was well attended. The audience, which included children and adults, showed significant interest in the instruments and the technology behind them. |
| Title | Music from the Augmented Instruments Laboratory 2023 |
| Description | The Augmented Instruments Laboratory presented a concert featuring new digital musical instrument research by lab members and featured guests, including Iran Sanadzadeh of Monash University (Australia) and Jeff Snyder of Princeton University. The concert took place at Queen Mary University of London on 17 November 2023. |
| Type Of Art | Performance (Music, Dance, Drama, etc) |
| Year Produced | 2023 |
| Impact | The concert was well received and served to deepen collaborations with the two invited international artist-researchers. |
| URL | http://instrumentslab.org/news/2023/11/13/ail-concert.html |
| Description | The findings of this project to date have divided into two categories. The first is theoretical, where bringing together human-computer interaction, engineering, musical practice and philosophy has suggested new ways to approach the design of creative technologies. This project originally proposed to investigate so-called "entanglement" theories: a loose collection of philosophies and design methodologies which move beyond human-centredness and place the relationships amongst entities (rather than entities themselves) at the core of their knowledge systems. In the first phase of this project, we developed these initial ideas into a broader theoretical foundation on how to understand -- and ultimately to design -- digital musical instruments, when the boundaries between human and technology, nature and culture, are not fixed or pre-ordained. These ideas have been published in well-known journals and conferences in the field (one paper of which received an honourable mention award). Partly as a result of this work, interest in entanglement theory in the musical instrument field is growing rapidly; for example, entanglement was the main theme of the 2025 International Conference on New Interfaces for Musical Expression (Canberra, Australia). The second finding of this project has focused on the relationships between physical and digital systems. It has long been known that the integration of these two domains is a weak point in interface design, owing to challenges such as insufficient input/output bandwidth in digital systems, high latency, low reliability and other problems. This project highlighted another limitation which emerges out of decades of engineering practice: the essentially unidirectional relationships between domains. For example, it is common for electronic sensors to control digital sound synthesis processes, but nothing about that sound synthesis propagates back to the sensors themselves; this contrasts with acoustic instruments where every part of the instrument is bidirectionally coupled to every other part. While work in musical haptics (i.e. tactile feedback) has a long history, we have developed a new approach to musical interaction where an individual transducer can act as a sensor and actuator at the same time, enabling bidirectional coupling between physical and digital domains. This work has been published and demonstrated at several conferences, and an extended study with a group of 10+ designers is ongoing. |
| Exploitation Route | See previous description. The theoretical findings of this work, as published in several papers, can inform new design processes and methodologies for creative technologies. Translating critical theory into concrete design ideation is a challenging proposition: there is no single "right" approach since critical theory and philosophy often do not function as specifications for a technical system. However, critical theory can also have a profound effect on high-level design decisions, and this project has begun to illustrate how things could be done differently on account of so-called entanglement theories. The project has also developed technical tools, including bidirectional vibration transducers, which can be put to use by other designers in creating new instruments. |
| Sectors | Creative Economy Digital/Communication/Information Technologies (including Software) |
| Description | Cyborg Soloists |
| Organisation | Royal Holloway, University of London |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | I have provided new musical instruments and related technologies for use in the Cyborg Soloists project, a UKRI Future Leaders Fellowship led by Dr Zubin Kanga at Royal Holloway University of London. I have built and delivered the instruments and supported their use in new pieces. |
| Collaborator Contribution | Composers and performers on the Cyborg Soloists project, including Dr Kanga, have worked with instruments from my lab to create and perform new pieces. From this process I have gathered useful feedback that can improve future instrument research. |
| Impact | Several pieces have been written involving my instruments, including TouchKeys and a new optical keyboard scanner, and these have been performed at different venues and festivals around the UK and Europe. |
| Start Year | 2022 |
| Description | Royal College of Music - Centre for Performance Science |
| Organisation | Royal College of Music (RCM) |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | Starting in early 2023, I have forged a collaboration with several researchers and musicians in the Royal College of Music, including members of their Centre for Performance Science, an interdisciplinary research centre which also includes other academics at Imperial. I have made my augmented instrument research, including the magnetic resonator piano, available for use by RCM academics and students, and we have begun exploring future opportunities around studying the interaction between performer and instrument. |
| Collaborator Contribution | RCM has provided temporary space for setting up the magnetic resonator piano, and is currently searching for a long-term home for it. Academics and students have begun exploring the instrument, and we have begun undertaking joint events including a public performance at the Great Exhibition Road Festival in 2024. We are also making plans for joint funding bids to deepen institutional cooperation. |
| Impact | We presented joint event at the Great Exhibition Road Festival to take place in June 2024, where RCM improvisers performed with new augmented instruments from my research lab. Plans are being made for future public events, including projects with the composition department and joint funding bids with the Centre for Performance Science. |
| Start Year | 2023 |
| Description | Sonic Arts Research Centre, QUB |
| Organisation | Queen's University Belfast |
| Country | United Kingdom |
| Sector | Academic/University |
| PI Contribution | For several years, I have had regular contact with academics affiliated with the Sonic Arts Research Centre at Queen's University Belfast. This group spans music, psychology and engineering, and as a team they regularly collaborate in designing new instruments and studying how musicians use them. I have travelled to Belfast several times to meet them and led an outline stage funding bid (still under review) to a cross-council UKRI call, which looks at modes of knowledge creation at the arts-engineering boundary. I also built them a magnetic resonator piano (MRP), a new augmented piano from my research, which they now regularly use for performances and teaching. |
| Collaborator Contribution | Several academics at QUB have worked with me to develop new research ideas and to help write funding bids (three thus far - one not funded, one under review, one to be submitted soon). I am the PI on one and they lead on the other two. Performers and composers at QUB are writing and premiering new music for magnetic resonator piano. |
| Impact | Several new performances for magnetic resonator piano have been premiered at QUB. Members of my team have presented at their Sonorities festival. Joint funding bids are currently under review. |
| Start Year | 2023 |
| Description | TU Berlin |
| Organisation | Technical University Berlin |
| Country | Germany |
| Sector | Academic/University |
| PI Contribution | In summer 2024, I visited the Audio Communications Group at TU Berlin, where I presented a hands-on workshop on embedded audio programming using the Bela open-source hardware platform. I met with several researchers there and advised on projects including a software framework for running AI inference algorithms on real-time audio streams. |
| Collaborator Contribution | The partners provided desk and lab space for my visit, and helped connect me to other local institutions and researchers. Discussions during my visit have helped scope out new directions for my research. I briefly returned in January 2025 and expect to maintain regular contact in the coming years. |
| Impact | This is a multi-disciplinary collaboration involving music and electronic engineering. It remains in early stages and no public-facing activities have taken place yet. |
| Start Year | 2024 |
| Description | University of Bologna |
| Organisation | University of Bologna |
| Country | Italy |
| Sector | Academic/University |
| PI Contribution | I am working with Dr Michele Ducceschi at the University of Bologna, Department of Industrial Engineering, on new musical instrument design. My contributions involve hardware sensor and actuator systems and interaction strategies. |
| Collaborator Contribution | The contributions of Dr Ducceschi's lab focus on physical modelling of acoustic musical instruments, i.e. numerically simulating the sound of acoustic systems. They have developed several custom versions of their audio plugin software which we are exploring in joint research. |
| Impact | A joint paper was published at the New Interfaces for Musical Expression 2024 conference, and a further submission is currently under review for a 2025 conference. The latter of these pertains to a replica harpsichord which has been exhibited in a museum gallery setting. |
| Start Year | 2023 |
| Title | pybela |
| Description | pybela, developed by QMUL PhD researcher Teresa Pelinski with support from the Augmented Instruments Laboratory and an industrial internship at Bela (http://bela.io), is a toolchain and workflow for training AI models to deploy on the Bela embedded hardware platform. pybela consists of tools for capturing data on a Bela embedded platform (a 1GHz ARM-Linux computer running a real-time operating system), transferring the data to python running on a host computer for training models in pytorch, and deploying lightweight models back on the Bela board. |
| Type Of Technology | Software |
| Year Produced | 2023 |
| Open Source License? | Yes |
| Impact | pybela was used in a workshop at the Timbre Tools Hackathon at QMUL in February 2024. It also supported an artistic collaboration in Berlin in summer 2024. A paper on the technology is under review, and further workshops and deployments are planned. |
| URL | https://github.com/belaplatform/pybela |
| Description | Augmented Instruments Lab at Science Museum Lates |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | On 19 October 2023, the Augmented Instruments Laboratory demonstrated several new musical instruments at the Science Museum "Music Lates" event. That evening, the museum was open to the public with exhibitors from around the UK showing different projects related to music technology. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Augmented Instruments Laboratory - Open Talks |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | The Augmented Instruments Laboratory has organised a series of public research seminars which are presented as hybrid in-person/live-streamed events and recorded for later watching. Thus far the lab has held four such events: - 6 October 2023 (Eliot Bates) - 3 November 2023 (Anna-Kaisa Kalia) - 17 November 2023 (Iran Sanadzadeh) - 9 February 2024 (Alex Harker) |
| Year(s) Of Engagement Activity | 2023,2024 |
| URL | https://www.youtube.com/@AugmentedInstrumentsLab/streams |
| Description | Bela workshop at Timbre Tools hackathon |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Postgraduate students |
| Results and Impact | On 16 February 2024, I presented a workshop on using the Bela embedded platform (bela.io, a spinout of my lab) to create musical instruments. The workshop demonstrated a novel AI workflow created by PhD student Teresa Pelinski in which sensor data could be captured by the Bela board, processed on a computer to train a model, and then the model deployed again on Bela. The workshop served as an introductory session for the Timbre Tools hackathon held at QMUL the following week. |
| Year(s) Of Engagement Activity | 2024 |
| Description | Embedding Algorithms workshop |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | On 26 July 2024, I and two members of my research team presented at a daylong workshop at UdK Berlin entitled "Embedding Algorithms", which featured a mix of talks and performances by people working at the intersection of AI, embedded computing and artistic practice. Our contributions included a talk connecting post-humanist philosophy with technical and creative practices and a performance featuring embedded AI algorithms running on the Bela platform. |
| Year(s) Of Engagement Activity | 2024 |
| Description | Guthman Musical Instrument Competition |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | I was invited to be one of three judges in the annual Guthman Musical Instrument Competition held at Georgia Tech (Atlanta, USA) on 8-9 March 2024. The Guthman Competition has been running for over a decade. It brings in instrument designers from around the world, with 10 finalists chosen this year from over 60 entries. The judges choose the winning instruments and present panel discussions and media interviews. Overall the event reached hundreds of people in person and hundreds more on a live stream. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://guthman.gatech.edu |
| Description | Imperial College London Global Summer School |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Schools |
| Results and Impact | In August 2024, I organised a daylong workshop on building digital musical instruments as part of the Imperial College London Global Summer School, a 2-week residential programme for 16-17 year old students from around the world. The event included a presentation about the motivations and challenges of making new instruments, followed by a hands-on session making new instruments with sound synthesis and touch sensing. The event received very positive feedback from students and led to invitations to repeat the activity at future outreach sessions. |
| Year(s) Of Engagement Activity | 2024 |
| Description | Inaugural lecture at Imperial College London |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | On 7 February 2024 I delivered my inaugural lecture at Imperial College London entitled "Making technology to make music". This was an overview of my career to date in digital musical instrument design, aimed at a general audience. I explained why musical instrument research was an interesting and challenging problem and demonstrated some of the outcomes of the research of the Augmented Instruments Laboratory, including new hybrid acoustic-digital instruments and accessible musical instruments. |
| Year(s) Of Engagement Activity | 2024 |
| URL | https://www.imperial.ac.uk/events/171732/making-technology-to-make-music/ |
| Description | Magnetic resonator piano in L'Amour Ouf |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | In March-April 2024, Hollywood composer and producer Jon Brion extensively used the magnetic resonator piano (an instrument from my research) in the score for the French film L'Amour Ouf ("Beating Hearts"; budget €35m). The film appeared at the Cannes festival in October 2024 where its director Gilles Lellouche was nominated for the top Palme d'Or prize. The score was also nominated for a 2025 César Award in France. |
| Year(s) Of Engagement Activity | 2024 |
| Description | NIME Summer School - Mexico City |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | In June 2023, members of the Augmented Instruments Laboratory held a 3-day-long public engagement workshop involving building new digital instruments with Bela, an open-source embedded platform for audio and sensor processing created in the lab. The Spanish-language workshop was held in Mexico City as a summer school preceding the International Conference on New Interfaces for Musical Expression. While the conference targets an international academic audience, the summer school was aimed at members of the local community many of whom were not affiliated with academic institutions. Participants made their own instruments, learning aspects of hardware and programming in the process, and presented their results in a showcase before the conference. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.nime2023.org/program/summer-school |
| Description | Porto Electronic Music Symposium |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | On 15-16 December 2023, I presented a keynote talk and instrument-building workshop at the Porto Electronic Music Symposium, a conference organised with national arts funding from Portugal. The workshop showed people how to build simple digital instruments with touch sensing and sound synthesis using the Bela platform (http://bela.io), a spinout project of my research lab. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Synth Design Hackathon 2023 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Postgraduate students |
| Results and Impact | In February 2023, I presented a 4-day instrument-building workshop as part of the international Synth Design Hackathon. Around 15 students from QMUL participated, grouped into 4 teams building different instrument prototypes. QMUL was one of over a dozen institutions which participated, with several dozen instrument projects overall. One of the QMUL teams was chosen as a winner of the hackathon and presented their instrument at Superbooth 2023 in Berlin. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.synthux.academy/events/hackathon-2023 |
| Description | Talk at WiSSAP 2023 - Kanpur, India |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | On 20 December 2023 I presented an online talk at WiSSAP 2023, the Winter School of Speech and Audio Processing held in Kanpur, India. The workshop acts as a training and networking event for music and audio researchers in India, especially students. The talk was well received and led to follow-up inquiries on my work. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://wissap23.madhavlab.com |
| Description | Two music and disability workshops |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Third sector organisations |
| Results and Impact | In May 2024, I co-organised a discussion workshop at Imperial College London entitled "Accessible Instrument Design Beyond Techno-Solutionism". This was co-organised by Victoria Kinsella from Birmingham City University and the OHMI Trust, a disability arts charity specialising in new accessible instruments; travel costs were also co-supported by an AHRC Research Network grant. The event brought 24 researchers and practitioners from around the UK to address persistent pitfalls of technology-led approaches to supporting inclusive music making for disabled people. The discussions were released as a white paper, and connections made at the event have led to further collaborations. In February 2025, I organised a hands-on technical workshop at QMUL with help from charities OHMI and Drake Music and support from the AHRC Research Network grant. We called this event a "Sustain-a-thon", turning the typical hackathon format on its head to have participants focus on making existing designs more sustainable and replicable rather than building one-off prototypes that are often discarded soon after the event finishes. This led to significant technical progress on two accessible instruments previously developed by workshop participants, as a result of which it will be easier to replicate and distribute them in the future. |
| Year(s) Of Engagement Activity | 2024,2025 |