Weakly-Supervised Learning for Classifying Acoustic Scenes and Events

The aim of this research is to develop supervised learning methods in order to classify audio signals for which the labels are weak or noisy. Audio classification minimally entails identifying what sounds (classes) are present in an audio signal. When it is also necessary to determine when the sounds occur, it is known as sound event detection (SED). In either case, the most effective approach is to use a dataset of example audio signals that are already labelled and then learn how to perform classification using these labelled examples. This is known as supervised learning. Unfortunately, collecting large amounts of labelled data can be prohibitively expensive. For sound event detection, it is common for only the class labels to be provided and not the timestamps. In other situations, the labels can even be inaccurate (noisy) because of the effort required to otherwise verify them. We refer to both as weakly-supervised learning problems, and formulate them under a single framework.

This research focuses on deep learning methods for audio classification. Broadly speaking, deep learning refers to the use of modern neural network architectures and large amounts of training data. This approach has produced state-of-the-art results in many tasks, including audio classification. However, we believe the deep learning methods for weakly-supervised learning are underdeveloped for general-purpose audio classification. Our contribution would be to introduce techniques that exploit the characteristics of audio, and in a way that adapts to the specific application. We also want to unify the treatment of weakly-supervised problems so that knowledge from seemingly different areas can be consolidated. This is something that is currently lacking in the audio domain.

The following objectives have been identified:
- Use an architecture based on convolutional networks for classification. The most recent advances such as residual networks and inception-style networks will be investigated.
- Consider how to design a network that is robust to label noise. We will look at pseudolabelling and loss function weighting conditioned on the examples.
- Consider how to design a network for sound event detection in which labels are weak. The most promising direction is to look at attention mechanisms that learn to weight the different segments of an audio clip. By labelling each segment the same as the audio clip, we can also consider the labels as noisy and use techniques that are robust to noise.
- Generally speaking, we want to formulate the various problems using probabilistic reasoning so that the solutions fit under a common framework and do not appear ad-hoc.