Fine-grained music source separation with deep learning models

Audio source separation is the task of splitting an audio signal into separate signals for each signal source. For music, the signal sources are the instruments that appear in the track, e.g. guitar, bass, piano, drums, and vocals. As an important task in music information retrieval, music source separation enables diverse applications on arbitrary music tracks that would need manual creation of stems otherwise. For example, in the context of music education, the creation of play-along tracks for students, facilitating by-ear transcription of relevant instruments, or automatic creation of karaoke backing tracks.

Current commercial software only focuses on splitting vocal, bass, and drums from the audio files and there is still no product that can simultaneously separate more instruments in a quality that is usable by musicians for remixing/remastering objectives. The project will fill this gap by developing advanced machine-learning algorithms for high-quality audio source separation. The ultimate goal is to automatically identify musical elements from any given song and extract them into independent tracks without quality loss, including vocal, instrumental, drums, bass, piano, electric guitar, acoustic guitar, synthesizer, etc.

The project will be led by AudioStrip, a London-based MusicTech company with a focus on building ML tools to free up time for licensed DJs/Producers and music businesses with a central focus on music source separation. The project will be collaborated with the Center for Digital Music at Queen Mary University of London, one of the biggest research centers in the UK focused on bridging music and artificial intelligence.