Structured machine listening for soundscapes with multiple birds

In this Early Career fellowship I will establish a world-leading capability in automatic inference about songbird communications via novel "machine listening" methods, working collaboratively with experts in machine listening but also experts in bird behaviour and communication. Automatic analysis has already shown benefit to researchers in efficiently characterising recorded bird sounds, but there are still many limitations in applicability, such as when many birds sing together. The techniques developed will specifically be designed to handle noisy multi-source audio recordings, and to infer not just the presence of birds but the structure of the signals and the interactions between them. Such methods will be a leap beyond the current state of the art in bioacoustics, allowing researchers to study not just sounds recorded in the lab under controlled conditions, but also field recordings and archive recordings found in public audio archives.

I will develop my techniques through specific application case studies. First through collaboration with David Clayton, an international expert on zebra finch behaviour and genetics, who recently moved his lab to my proposed host institution. The zebra finch is an important "model organism" in biology, because its genome is fully sequenced and it is a useful bird for probing aspects of songbird vocal development. I will collaborate with the Clayton lab to develop methods for automatically inferring the social interactions implicit in audio recordings of zebra finch colonies. Second, I will conduct international research visits to collaborate with other research groups who analyse bird sounds and bird social interactions. Third, I will study the case of automatically detecting bird activity in arbitrary sound archives, such as the soundscape recordings held by the British Library Sound Archive.

Importantly, not only will I apply modern signal processing and machine learning techniques, but I will also develop new techniques inspired by this application area. This fellowship is not about contributing from one field to another, but about building up UK research strength in this cross-discplinary research topic. In order to make the most of this possibility, I will host research workshops and an open data contest to serve as focal points for research attention, and I will also conduct a public engagement initiative to engage the widest possible enthusiasm for this exciting field of possibility.

This fellowship directly aligns with the "Working Together" priority, which is EPSRC's current overriding priority for ICT fellowships.

The prime beneficiaries outside my immediate field will be in research fields benefitting from the structured analysis of animal sounds and interactions. For example the improved techniques in zebra finch analysis will complement ongoing research into songbird genetics and individual differences, or research into conversational interactions in linguistics: the availability of more structured naturalistic data about animal communication could provide stronger empirical foundation to considerations of the evolution of communication systems. (This impact overlaps to some extent with that described in the Academic Beneficiaries section.)

The availability of these sound analysis techniques is also of interest to wildlife monitoring organisations such as the British Trust for Ornithology (BTO). They largely use manual surveying by volunteers and professionals to quantify the distributions of species: however, if high-quality automatic analysis were available their work could be made more efficient. Current academic and commercial software (examples include Raven, XBAT, Seewave, Praat, Sound Analysis Pro) allow users to inspect and detect bird sounds but are unable to analyse communication networks, nor can they use models of communication interactions to ensure high-quality detection. Analysing not just the presence of bird song and calls, but the networks of interaction between them, could be used as an indicator of the population health, reflecting issues such as habitat fragmentation which can impact the viability of a bird population. Downstream, detailed analysis of animal sound can thus form a strong evidence base for ecological policy decisions.

The application to audio archives shows another direct route to impact. Large audio collections such as those in the British Library Sound Archive are highly valuable to society, yet a lot of their value remains locked away because there is very little metadata associated that would facilitate different types of query. This research will directly enable the unlocking of some of this value, helping people to discover the presence of birds in large audio archives which may not be annotated for their bird sounds, indeed may have been collected for entirely different reasons.

The fellowship will also have an impact on the public understanding of bird sounds, bird social interactions, and signal processing and machine learning. These will be explicitly encouraged through the public engagement activities: through engaging talks, articles and exhibits I will aim, not to place the technology between the public and the birds, but to enchant the public with both the wonders of technology and the wonders of bird vocal communication.