Humanising Algorithmic Listening

Lead Research Organisation: University of Sussex
Department Name: Sch of Media, Film and Music

Abstract

Humanities researchers are opening their ears. From sound studies to oral history, media archaeology to ethnography, researchers are listening to the sonic environments of our lives, both off and online. In response to this digital, sonic turn, the Humanising Algorithmic Listening (HAL) network will bring together experts from across the humanities, sociology, machine listening, critical theory philosophy and interactive computer music to develop a research agenda for the design and development of computational methods for audio analysis - listening algorithms - in the arts and humanities.

Machine listening algorithms already exist but need tuning to align with and enrich humanities research. Existing methods are capable of tasks such as recognising melodies, identifying instruments or musical genres. Their power is evidenced by commercial IT products such as iPhone's Siri, which uses voice recognition software, or Spotify's recommender system that magically creates musical playlists based on our previous preferences. Driven by technical innovation and commercial interests, these tools are becoming at once more powerful, more complex and their inner workings more opaque. In order to support humanist research as tools for discovery, interpretation and insight, computational power must be accompanied by critical consideration of what it means to listen through algorithms, what new ways of listening might be afforded, and how we can learn to interact meaningfully with these black-boxed processes.

A rich philosophical literature examines the active role that technologies play in mediating our relationship with the world. These ideas have been considered in the context of interactive computer music, where software agents that listen and respond as musical partners have been created, but have yet to be thoroughly explored in the wider contexts of technologically-mediated listening. Developing a critical framework is crucial, firstly to underpin the design of new computational methods in arts and humanities - for example to support the interpretation and discovery of large audio archives and the design of new creative sound-based technologies. And secondly to deepen understanding of the role that these listening algorithms play in our everyday lives, thus contributing to current global digital ethics policy-making efforts.

In the course of three two-day meetings an agenda for algorithmic listening in the arts and humanities will be advanced. Workshop 1, led by PI Eldridge, will explore cultural questions and state-of-the art machine listening methods. Workshop 2, led by Co-I Stapleton, will focus on critical perspectives, exploring philosophical theories of human-machine interagency through interactive computer music systems. Workshop 3 will bring cultural, technical and critical perspectives together to propose an agenda for the co-design of Listening Algorithms that will stimulate future research and cultural understandings of listening algorithms.

Journal publications in each of the core areas will propagate discussions throughout relevant academic disciplines. Wider cultural impact will be achieved through a public panel in Brighton Digital Festival and feature on national radio. The network will disseminate its findings within and beyond academia through a strong, open access web-presence; it will be managed by a cross-sector Advisory Board of international experts to ensure relevance for and impact across relevant academic and public stakeholder communities.

This network builds on the University of Sussex recent £3 million investment in a Humanities Lab (SHL). It is led by Dr Alice Eldridge, a music practitioner and programmer experienced in applied machine listening, and co-led by Dr Paul Stapleton at Sonic Arts Research Centre (SARC), Queen's University Belfast. Both SHL and SARC have a proven track record in developing and delivering technically and critically challenging transdisciplinary research.

Planned Impact

By developing a research agenda for advancing critical and technical research in machine listening algorithms, the network activities will benefit a range of cultural third sector, commercial and creative IT industries and ultimately the formation of policies around the ethics of machine listening.

IA 1. New methods for managing and discovering audio archives in libraries and museums.
An upcoming challenge for all archives across all repositories is the handling of vast digital collections. New computational methods not only automate existing procedures but potentially afford novel means of discovery and navigation. Richard Ranft, Head of Sound and Vision at the British library and Fiona Courage of The Keep are on the Advisory Board to ensure that the network is informed by and informs the emerging needs of public audio archives.

IA 2. New application areas for machine listening and data science.
The sonic turn in the humanities, coupled with the exponential growth of audio archives represents a significant new field of application for machine listening and learning communities, within industry as well as academia. Methods for clustering and categorising music and bioacoustics data align within an engineering, problem solving framework; the types of questions and sonic data arising from humanities research are likely to challenge extant methods and drive novel perspectives. The network activities include a data challenge that will introduce DH data to the largest online community of international data scientists (see description of Kaggle Competition in letters of support).

IA 3. Enriched understanding of machine agencies for creative music technology designers and users.
Quasi-intelligent algorithms in commercial music technology products are increasingly common place. There have been large investments by companies such as Novation and Native Instruments (NI) in making controllers that algorithmically bolster interaction between humans and sound synthesis software, and substantial effort in EU funded projects such as Integra to support user understanding of such technology. This part of the music-industry sector is represented in the network by music technologist Tim Exile, creator of NI's voice controlled synthesizer, The Mouth. This network will enable NI and other music technology companies to draw on the expertise of academics. Ultimately, a philosophically informed design can support the development of richer new musical instruments, and to spur new collaborations between academics and commercial partners.


IA 4. Policy
Despite the increasing presence of Listening Algorithms (Spotify, Hello Barbie etc.) in everyday life, policy around the ethics of these and other automated algorithmic processes is in its infancy. The European Data Protection Supervisor's announcement in Jan 2016 of a new Digital Ethics advisory body promises to address how to ensure the integrity of our values while embracing the benefits of new technologies, but these policies are still in development. The network aims to generate discussion that could inform this nascent area of policy making. To this end, Leverhulme Centre for the Future of Intelligence member, Simon Goldhill is an Advisory Board member. Core participant Palle Dahlstedt links this network to another European initiative focused on global impact of algorithms http://thebiospherecode.com.

IA 5 Wider Public
The increasing ubiquity and fundamental opacity of listening algorithms in everyday devices means that enhancing public understanding of algorithmic listening, one of many technologies of automation which are rapidly populating our lives, is an important issue. Arts practices and surrounding debates have an important role in supporting public understandings of associated technical and ethical issues. Communities with artistic and/or digital media interests will be reached via festival and radio appearances outlined in Pathways to Impact.

Publications

10 25 50
 
Description The ideas discussed throughout the network meetings continue to nourish the thinking and research of all members.
Exploitation Route The selection of talks curated on the website are a great resource for scholars wishing to think through the societal and cultural implications of machine listening.
Sectors Culture, Heritage, Museums and Collections

URL http://www.algorithmiclistening.org/
 
Description The ideas discussed have fed into ongoing research carried out by the PI and network member (C. Kiefer) into distributed machine listening in feedback systems. This has resulted in numerous public concerts and a recording released on confront records https://confrontrecordings.bandcamp.com/album/onca-gallery-brighton
First Year Of Impact 2018
Sector Culture, Heritage, Museums and Collections
Impact Types Cultural,Societal

 
Description INTERACT
Amount € 3,100 (EUR)
Funding ID WILDSENS 
Organisation INTERACT 
Sector Charity/Non Profit
Country Finland
Start 06/2018 
End 07/2018
 
Title Application of Machine Listening in Oral History 
Description This is a set of python notebooks introducing core machine listening methods to oral historians. The methods are not new in themselves, but constitute a knowledge transfer from one discipline to another. 
Type Of Technology Webtool/Application 
Year Produced 2017 
Open Source License? Yes  
Impact The notebooks were created for two workshops: One at Digital Humanities Conference 2016: Music Information Retrieval Algorithms for Oral History Collections (http://dh2016.adho.org/abstracts/10) 
 
Description DIY radio transmission event 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Two workshops of 14 people each were run at a local art gallery (ONCA). The workshops were led by a Soundtent - an artist run organisation - and introduced audiences to DIY electronics, giving an overview of how to make a radio transmitter from a raspberry pi. Audiences then had access to all necessary materials. The workshop sparked a couple of future collaborations between members. They workshop cemented links between the university, the art gallery ONCA, the South Downs Trust, various local media groups (Brighton Digital Festival) and the Universities of Sussex and Brighton music and media arts courses.
Year(s) Of Engagement Activity 2017
URL http://brightondigitalfestival.co.uk/events/diy-radio-transmission-pi-streambox-workshop
 
Description Long Table @ Brighton Digital Festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Humanising Algorithmic Listening in Culture and Conservation at Brighton Digital Festival. Network members hosted a public debate in an informal, semi-theatrical setting of the Rose Hill Arts centre. Audience members were largely artists and musicians interested in the implications of new technologies on daily life. Many reported knowing very little about the introduction of listening algorithms in every day technologies. The open format discussion was a real success. Following the success of this event the university plans further open format research talks at this venue.
Year(s) Of Engagement Activity 2017
URL http://brightondigitalfestival.co.uk/events/humanising-algorithmic-listening-culture-conservation
 
Description TECHNE workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact 40ish TECHNE students attended a day long workshop at CRiSAP, UAL. All were arts and humanities students new to coding. A wide range of responses from revulsion at workignwith maths to pure delight at being able to listen in new ways. Vast majority reported major new perspective on approaching working with oral archives
Year(s) Of Engagement Activity 2017
URL http://www.techne.ac.uk/for-students/techne-events/apr-2015/data-mining-the-audio-of-oral-history-a-...