Humanising Algorithmic Listening

Humanities researchers are opening their ears. From sound studies to oral history, media archaeology to ethnography, researchers are listening to the sonic environments of our lives, both off and online. In response to this digital, sonic turn, the Humanising Algorithmic Listening (HAL) network will bring together experts from across the humanities, sociology, machine listening, critical theory philosophy and interactive computer music to develop a research agenda for the design and development of computational methods for audio analysis - listening algorithms - in the arts and humanities.

Machine listening algorithms already exist but need tuning to align with and enrich humanities research. Existing methods are capable of tasks such as recognising melodies, identifying instruments or musical genres. Their power is evidenced by commercial IT products such as iPhone's Siri, which uses voice recognition software, or Spotify's recommender system that magically creates musical playlists based on our previous preferences. Driven by technical innovation and commercial interests, these tools are becoming at once more powerful, more complex and their inner workings more opaque. In order to support humanist research as tools for discovery, interpretation and insight, computational power must be accompanied by critical consideration of what it means to listen through algorithms, what new ways of listening might be afforded, and how we can learn to interact meaningfully with these black-boxed processes.

A rich philosophical literature examines the active role that technologies play in mediating our relationship with the world. These ideas have been considered in the context of interactive computer music, where software agents that listen and respond as musical partners have been created, but have yet to be thoroughly explored in the wider contexts of technologically-mediated listening. Developing a critical framework is crucial, firstly to underpin the design of new computational methods in arts and humanities - for example to support the interpretation and discovery of large audio archives and the design of new creative sound-based technologies. And secondly to deepen understanding of the role that these listening algorithms play in our everyday lives, thus contributing to current global digital ethics policy-making efforts.

In the course of three two-day meetings an agenda for algorithmic listening in the arts and humanities will be advanced. Workshop 1, led by PI Eldridge, will explore cultural questions and state-of-the art machine listening methods. Workshop 2, led by Co-I Stapleton, will focus on critical perspectives, exploring philosophical theories of human-machine interagency through interactive computer music systems. Workshop 3 will bring cultural, technical and critical perspectives together to propose an agenda for the co-design of Listening Algorithms that will stimulate future research and cultural understandings of listening algorithms.

Journal publications in each of the core areas will propagate discussions throughout relevant academic disciplines. Wider cultural impact will be achieved through a public panel in Brighton Digital Festival and feature on national radio. The network will disseminate its findings within and beyond academia through a strong, open access web-presence; it will be managed by a cross-sector Advisory Board of international experts to ensure relevance for and impact across relevant academic and public stakeholder communities.

This network builds on the University of Sussex recent £3 million investment in a Humanities Lab (SHL). It is led by Dr Alice Eldridge, a music practitioner and programmer experienced in applied machine listening, and co-led by Dr Paul Stapleton at Sonic Arts Research Centre (SARC), Queen's University Belfast. Both SHL and SARC have a proven track record in developing and delivering technically and critically challenging transdisciplinary research.

By developing a research agenda for advancing critical and technical research in machine listening algorithms, the network activities will benefit a range of cultural third sector, commercial and creative IT industries and ultimately the formation of policies around the ethics of machine listening.

IA 1. New methods for managing and discovering audio archives in libraries and museums.
An upcoming challenge for all archives across all repositories is the handling of vast digital collections. New computational methods not only automate existing procedures but potentially afford novel means of discovery and navigation. Richard Ranft, Head of Sound and Vision at the British library and Fiona Courage of The Keep are on the Advisory Board to ensure that the network is informed by and informs the emerging needs of public audio archives.

IA 2. New application areas for machine listening and data science.
The sonic turn in the humanities, coupled with the exponential growth of audio archives represents a significant new field of application for machine listening and learning communities, within industry as well as academia. Methods for clustering and categorising music and bioacoustics data align within an engineering, problem solving framework; the types of questions and sonic data arising from humanities research are likely to challenge extant methods and drive novel perspectives. The network activities include a data challenge that will introduce DH data to the largest online community of international data scientists (see description of Kaggle Competition in letters of support).

IA 3. Enriched understanding of machine agencies for creative music technology designers and users.
Quasi-intelligent algorithms in commercial music technology products are increasingly common place. There have been large investments by companies such as Novation and Native Instruments (NI) in making controllers that algorithmically bolster interaction between humans and sound synthesis software, and substantial effort in EU funded projects such as Integra to support user understanding of such technology. This part of the music-industry sector is represented in the network by music technologist Tim Exile, creator of NI's voice controlled synthesizer, The Mouth. This network will enable NI and other music technology companies to draw on the expertise of academics. Ultimately, a philosophically informed design can support the development of richer new musical instruments, and to spur new collaborations between academics and commercial partners.


IA 4. Policy
Despite the increasing presence of Listening Algorithms (Spotify, Hello Barbie etc.) in everyday life, policy around the ethics of these and other automated algorithmic processes is in its infancy. The European Data Protection Supervisor's announcement in Jan 2016 of a new Digital Ethics advisory body promises to address how to ensure the integrity of our values while embracing the benefits of new technologies, but these policies are still in development. The network aims to generate discussion that could inform this nascent area of policy making. To this end, Leverhulme Centre for the Future of Intelligence member, Simon Goldhill is an Advisory Board member. Core participant Palle Dahlstedt links this network to another European initiative focused on global impact of algorithms http://thebiospherecode.com.

IA 5 Wider Public
The increasing ubiquity and fundamental opacity of listening algorithms in everyday devices means that enhancing public understanding of algorithmic listening, one of many technologies of automation which are rapidly populating our lives, is an important issue. Arts practices and surrounding debates have an important role in supporting public understandings of associated technical and ethical issues. Communities with artistic and/or digital media interests will be reached via festival and radio appearances outlined in Pathways to Impact.