OMRAS2: A Distributed Research Environment for Music Informatics and Computational Musicology

Lead Research Organisation: Queen Mary University of London
Department Name: Sch of Electronic Eng & Computer Science

Abstract

Imagine you have just bought a new iPod, you rip loads of your dad's CDs into it (his music's cool) as well as your own, and pretty soon you have 10,000 tracks and the iPod is full. Now there's a problem. You've never listened to your dad's CDs (not that many of them anyway) and you're really not sure what The Human League sounds like, and there's another 500 CDs of his music in there. Where are the good songs? How can you ever build those really cool playlists to impress your friends with your vast musical knowledge?Online Music Recognition and Searching II A Distributed Framework for Music Informatics and Computational Musicology.Imagine you've just been given a gist subscription to a 2 million song online music store. You can choose 10,000 songs to download onto your music player, but there's a problem. You have never heard a vast majority of these songs so you're not sure which are the one's you like. How can you put together those playlists to impress your friends with your vast musical knowledge?The problem is simular for the radio DJ looking for a new playlist to keep their show on the cutting edge, or the professional violinist doing research into different performances of Vivaldi'd Four Seasons to find a new twist for an expectant audience, or the recors producer trying to find a mathimatical formula for number one singles (yes, they really do this).The answer to the above question and other interesting problems concerning large collections of digital music are exactly what the OMRAS2 project will address. When OMRAS2 is completed, you'll be able to get software that helps you build playlists with songs that you'll love even though you never heard them before; and there will be tools to help the violinist and record producer achive their goals too. Using tools from OMRAS2, your ipod will be able to predict the best sounds to use for the best chart topping number one. If you study music at University, you'll probably use OMRAS2 for analysing and comparing music.OMRAS2 aims to help technology researchers build and investigate the software that is needed to construct these super-tools. But that's not all. It will help musci researchers investigate interesting aspect of music, such as what variations of that riff in Purple Haze did Jimi Hendrix play and how did the differ, and how did different pianists interpret Bach's Goldberg Variations. OMRAS2 will also look deeply at how music and information about music (like CD Insert booklets, but more and online) will be enjoyed at home, not just downloading, but also searching, recomending, browsing and so on. And it wont be hard to use:OMRAS2 will use interfaces that look and react like familiar music software like Adobe Audition or RealAudio player.

Publications

10 25 50
 
Description The project demonstrated:

(1) the utilities of high-level semantic features of musical audio (including musicological terms and free-form labels such as social tags) in multimedia content management,

(2) the use of low-level audio features and probabilistic statistical models to derive high-level semantic descriptors automatically, facilitating navigation in large online audio collections,

(3) the utilities of the Semantic Web, and Semantic Web technologies in online audio content navigation and delivery,

(4) the use of digital signal processing and machine learning for the manipulation of digital audio content on the semantic level, allowing interaction with notes, chords or performance characteristics such as vibrato by re-synthesising audio from parametrised descriptors.



Several computational algorithms were developed for automatic annotation of musical audio, including novel methods for audio transcription, chord recognition, key recognition, tempo and beat detection, structural segmentation and music similarity.



Several Semantic Web ontologies were created for describing and publishing music related metadata on the Semantic Web. The Music Ontology, a core framework connecting the OMRAS2 ontologies became a de-facto standard in music-related data publishing.
Exploitation Route The are numerous potential uses of OMRAS 2 technologies in a non-academic context. This includes music search engines that are based on audio similarity characterised by acoustic features of sound, search engines and Semantic Web user agents that access audio archives by high level semantic concepts. These include musicological terms such as keys, chords or rhythm, and social-contextual similarity like artist collaborations. These tools can be used in online services, content management platforms, libraries and archives, educational institutions, etc.. Lower level music signal processing tools developed by the project may be utilised in music production and delivery.
The project developed several easy to use tools allowing the exploitation of project outcomes by academic communities and industry partners. These tools include Sonic Visualiser, and application for examining high or low-level features of sound in the context the audio waveform. Sonic Visualiser is equally useful for researchers working on audio signal processing and machine learning algorithms, as well as musicologists analysing audio collections. The program is supported by a C++/Python Application Programming Interface, that allows distributing content analysis algorithms in a standard plugin format, that can be used by several host applications, including audio editors and batch audio processors.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software)

URL http://www.omras2.org
 
Description EPSRC Digital Economy Research in the Wild
Amount £250,102 (GBP)
Funding ID EP/I001832/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 09/2010 
End 03/2012
 
Description Follow on fund
Amount £83,127 (GBP)
Funding ID EP/H008160/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 02/2010 
End 02/2011
 
Description Jisc
Amount £94,894 (GBP)
Organisation Jisc 
Sector Public
Country United Kingdom
Start  
 
Description Software sustainability
Amount £947,047 (GBP)
Funding ID EP/H043101/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 04/2010 
End 09/2014
 
Description The Royal Society Wolfson Research Merit Award
Amount £25,000 (GBP)
Funding ID to be confirmed. 
Organisation The Royal Society 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2015 
End 03/2020
 
Description Goldsmiths College 
Organisation Goldsmiths, University of London
Country United Kingdom 
Sector Academic/University 
PI Contribution Various academics from Goldsmiths have collaborated with QM over the years, including Prof T Crawford, Prof A Tanaka, Prof M D'Inverno
Collaborator Contribution Research knowhow, co-writing papers, access to software, joint grant proposals.
Impact The Transforming Musicology grant from AHRC is one outcome.
 
Title Audio Segmentation VAMP plugin 
Description This software analyses musical audio files and divides them into musically meaningful segments, typically 4-10 per song. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2008
Licensed Yes
Impact It is often downloaded with Sonic Visualiser - see other IP entry
 
Title Sonic Visualiser 
Description Sonic Visualiser is an application for viewing and analysing the contents of music audio files. This is an open source software framework that works with its own plugin architecture called VAMP. Sonic Visualiser is Free Software, distributed under the GNU General Public License (v2 or later) and available for Linux, OS/X, and Windows. Sonic Visualiser is intended for use by people in various academic disciplines, but also by non-academics, professional audio users, and interested hobbyists. It supports loading additional analysis plugins that other developers or institutions can publish, and at least 10 institutions (besides QM) have chosen to publish plugins in this format. Sonic Visualiser was first published by the Centre for Digital Music in 2007 and has been continuously developed at the Centre and maintained as free, cross-platform open-source software ever since. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2006
Licensed Yes
Impact According to our ongoing user survey that has been carried out since the start of 2014 (having 1053 responses to date, all from people who were actually using Sonic Visualiser at the time): * 49% of respondents use Sonic Visualiser primarily in a personal capacity, 10% for professional work, 33% in academia at some level (the rest "other"); * 46% use it primarily in the field of musicology or music analysis, 19% in music composition or production, 16% in audio engineering or signal processing, 5% in software development, 2% speech processing (the rest "other"); * 55% said they found it "very enjoyable" to use and 41% "moderately enjoyable"; * 27% said they found it "very easy" to use and 62% "moderately easy"; * 59% of respondents had some additional analysis plugins installed on top of the default configuration; among these respondents, the mean number of additional plugin sets installed was 3.4. Sonic Visualiser's version-update-checker logs keep a count of distinct IP addresses from which SV has been used (for users who have agreed to this). They show over 138,000 distinct addresses during the past year, with around 4,000 distinct addresses per week on average. An impression of the extent of usage can be gained by viewing illustrative Youtube videos about sound analysis made by third parties: at a quick scan I count at least 100 that show Sonic Visualiser, of varying quality. Sonic Visualiser has also been used in online course material, such as the popular Coursera "Audio Signal Processing for Music Applications", as well as in courses for academics in non-technical disciplines, such as the Digital Humanities at Oxford Summer School. This is a free, open source project; it was used as an Impact Case study for REF 2014. See the Sonic Visualiser website for details: http://sonicvisualiser.org Link to a page about the new release: http://sonicvisualiser.org/new-in-v3.html Sonic Visualiser has been available for about a decade now, and this is one of the most substantial updates it's ever had (hence the updated major release number).
 
Title Soundbite 
Description Soundbite is an iTunes plugin developed under this project. It is currently available from www.isophonics.net as freeware. Commercial licences are under negotiation. Under the Platform Grant it was adapted for an Android platform and reconstructed as a client-server architecture. 2017: A commercial licence for the core technology that was signed with Music XRay, a company based in the USA, several years ago is starting to yield royalties. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2008
Licensed Yes
Impact It has been downloaded to hundreds of (unregistered) users. It was deployed in various grants to analyse collections of music for recommendation, navigation and play listing. It has been deployed in several commercial music aggregators workflows, enabling trials of large scale music recommendation systems.