Datasounds, datasets and datasense: Unboxing the hidden layers between musical data, knowledge and creativity

Lead Research Organisation: Kingston University
Department Name: The School of Arts

Abstract

This network aims to identify core questions that will drive forward the next phase in data-rich music research, focused in particular on creative music making. The increased availability of digital music data combined with new data science techniques are already opening new possibilities for making, studying and engaging with music. This direction is only likely to speed up upending many current practices, opening up creative avenues and offering new opportunities for research. However, the rapid technological progress with new techniques producing surprising results in rapid succession, is often disconnected from the knowledge and knowhow gained by musicians through creativity, practice and research. By bringing together researchers and practitioners from different underlying disciplines and with a wide range of expertise the network will enable a better foundation for future research. Performers, composers, and improvisers will contribute through embodied knowledge and practice-based methods; researchers in psychology will bring insights about cognitive, affective and behavioural processes underpinning musical experience; and data scientists will add analytical expertise as well as relevant theories, methods and techniques. These will lead to significant conceptual breakthroughs in data driven approaches and technologies applied to music.
The new data-based technologies usually rely on large data sets, they can also produce very large amounts of data. As part of the network activities we will map the limitations of existing music representations, identify the gaps that need to be addressed and propose pathways to improve representation formats. We do not envision developing a single, all encompassing representation that captures the full richness of musical experience. Nevertheless, through the dialogue that this network will facilitate we will be able to outline ways of improving on existing representation formats and develop methods for visualising, analysing, and interpreting large data sets. The network will also consider ethical and legal implications such as how best to address the challenges that Artificial Intelligence (AI) poses to existing musical practices and the fear that this technology induces. Some of these are common to many fields where AI is being applied to tasks which were until very recently the preserve of humans. Music offers a unique perspective on these wider problems - the opacity of 'black box' generative models is a low-risk research challenge not a potentially dangerous tool that may entrench existing injustices. By embedding the ethical dimension into the discussion of the future of data driven music research the network will serve as a model for other fields.
The core activity of the network are two workshops where short presentations will provide a springboard for in-depth discussions; a concert by practitioners with relevant experience will help connect the theoretical discussions to the reality of music making. These will enable a multidimensional exchange of ideas and methods. Material from these workshops will be shared online to document the process and provide a platform to engage wider audiences with the approach taken and the significant results obtained.
Data driven technologies are already having an effect on the way in which we understand, make and consume music within current cultural and economic contexts, raising complex and unprecedented ethical and legal considerations. This network will identify core questions that can propel forward data driven research into creative music making that consider social and individual needs. We will also be able to outline specific research projects that address the shared concerns and bridge the gaps between the different methods that, in many ways, bound our disciplines. The network builds on previous AHRC funded research by the PI (AH/N504531/1 and AH/R004706/1) applying data to creative music making in a particular domain.

Publications

10 25 50
 
Title Duo improvisation between piano and AI-driven computer system 
Description Pianist David Dolan and composer Oded Ben-Tal (PI for this research) improvising as a duo with Ben-Tal's computer code acting as a semi-autonomous improvising agent. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2022 
Impact Performing these improvisation stimulates debate around the social meaning of creativity, the role of computers in creative activities and the use of music-data for creating music. 
URL https://stream.gsmd.ac.uk/View.aspx?id=65262~5i~airjgQkg8l&code=CP~VpKaoDWBMLE2BnP3rxWwUgBABSI9PtMzD...
 
Description Performance partnership between Federico Reuben and Franziska Schroeder 
Organisation Queen's University Belfast
Country United Kingdom 
Sector Academic/University 
PI Contribution This new collaboration came as a result of the researchers meeting at the first workshop (30/4/2022) organised by this network. The performance they developed through a research process makes innovative use of data tools to create an audio-visual performance illustrating the capacity of data to enable music creativity. The first performance took place during the second workshop organised by the network - 21/11/2022 in Stockholm, Sweden.
Collaborator Contribution The two collaborators Dr. Reuben and Prof. Schroeder developed their improvisation-based performance through discussions and rehearsals. Each of them is making use of data tools to open new opportunities for creating sounds and visuals. The research-creation process combines artistic exploration and technological development in tandem and through collaboration.
Impact This collaboration resulted in a public performance in concert. The collaboration is multi-disciplinary straddling music, technology (computing and AI), and visual art.
Start Year 2022
 
Description Practice-based research collaboration between Oded Ben-Tal and David Dolan 
Organisation Guildhall School of Music & Drama
Country Afghanistan 
Sector Academic/University 
PI Contribution The collaboration was enabled by the participation of Prof. Dolan in the first workshop organised by the network (York, 30/4/2022). Following the workshop Dolan and Ben-Tal (PI on the network) began developing collaborative research project on human-AI improvisation. The first performance resulting from this research took place as part of the second workshop organised by the network (Stockholm, 21/11/2022).
Collaborator Contribution Prof. Dolan is bringing his extensive expertise in improvisation - as an improvising pianist when he performs with the AI system; as a researcher on improvisation and creativity when we develop the research; and from his pedagogical experience as a teacher of improvisation.
Impact Outputs so far are performances of human-AI improvisation: 21/11/2022 in Stockholm; 3/2/2023 in London. The project is multi-disciplinary: music and computing/AI.
Start Year 2022
 
Description Interview for The Today programme on BBC radio 4 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Dr. Ben-Tal (network PI) was interviewed about his recent work with music AI. He was asked for his opinion on this fast developing technology and its impact on society and culture.
Year(s) Of Engagement Activity 2023