Learning the Structure of Music

Lead Research Organisation: Plymouth University
Department Name: Computing, Communications & Elec, Sch of

Abstract

This project is aimed at the development of models and tools for the application of novel probabilistic machine learning techniques to the analysis of music. The underlying theme of the project is the learning of patterns linking different data arising simultaneously from the same piece of music. The sources of data will be as follows: a) musical scores (MIDI format), b) audio (recordings of the pieces), c) worm data (charting performance information), d) EEG data (of subjects listening to the music) and d) fMRI data (of subjects listening to the music).The linking patterns that we will be seeking involve pairs of data streams as follows: a) musical scores with worm data, b) musical scores with fMRI data and c) audio with EEG data. The first pair will be used to identify typical performance patterns of particular performers. The second and the third pairs will be used to identify the effects in the brain of particular musical patterns (such as melodic sequences, musical phrasings, harmonic progressions, etc.).The project will advance our understanding of the relationship between musical structure and performance and experience. The potential of such developments is quite wide ranging, with potential application in music therapy and entertainment. For example, it will contribute to the development of systems for artificial performance of music imitating the style of a performer on pieces that he or she may have never played before and systems for musical composition tailored to achieve specific effects (or moods) on the listener.
 
Description This project contributed to Artificial Intelligence and Music Technology. As for Artificial Intelligence it contributed to the advancement of Machine Learning, in particular its application to a range of data types: we developed new algorithmic and statistical methods for the analysis of music data, EEG and fMRI scans. As for the field of Music Technology it developed an innovative programming tool for music composition using the constraints satisfaction method. Finally, we believe that we advanced our understanding of music performance. We have gained a much better idea of how to model musical performance and how variations in the way musicians perform a piece may affects listeners.
Exploitation Route According to the National Music Council, the British music industry generates approximately 150,000 full-time UK jobs. The emergence of new economic powers in Asia poses a threat to a number of UK industries, including the music industry. Development of new music technology meets the government's strategies to keep Britain competitive internationally as the country is re-branding its industry to operate at very high-levels of scientific and technical expertise. The new methods for EEG and fMRI analyses that we developed are proving to be useful for the field of Brain-Computer Interface. It had a direct impact on the Brain-Computer Music Interface project that we are currently developing in collaboration with the music therapy unit of the Royal Hospital for Neuro-disability, London.

Our constraint satisfaction-based compositional system (Strasheela) has been released in the public domain (Open Source software). It has been used in other projects in our own laboratory and it has been occasionally used by third parties in research and creatively by composers.

On the whole, this project addressed basic research questions which do not have a single direct route for exploitation. We contributed to UK's expertise in our field and formed young researchers.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Healthcare

URL http://cmr.soc.plymouth.ac.uk/index.html
 
Description Our research papers have been cited by peers in the community. The technology we have developed led to the design of brain-computer music interface system which was used by a Locked-in syndrome patient at the Royal Hospital for Neuro-disability, Putney.
First Year Of Impact 2011
Sector Creative Economy,Healthcare
Impact Types Cultural,Societal

 
Description Johannes Kepler University 
Organisation Johannes Kepler University of Linz
Country Austria 
Sector Academic/University 
Start Year 2006
 
Description Leibniz Institute for Neurobiology 
Organisation Leibniz Association
Department Leibniz Institute for Neurobiology
Country Germany 
Sector Academic/University 
Start Year 2006