Information and neural dynamics in the perception of musical structure

Lead Research Organisation: Queen Mary, University of London
Department Name: Sch of Electronic Eng & Computer Science


Music is one of the things that makes us human. No known human society exists without music; and no other species seems to exhibit musical behaviour, in the same sense as humans. It is an open question where music came from (in terms of evolution), but it is self-evident that it arises from the human brain: for there to be music, a brain was involved somewhere, even if only in listening. What is not evident at all is how brains (or the minds to which they give rise) make, or even perceive, music. This project aims to understand how human musical behaviour can be modelled using computers, by building programs which embody theories of how the musical mind works, and then comparing them with humans engaged in musical activity and also by comparing their predictions with those of an expert music analyst. This means that the project will contribute to various areas of study: computer music, statistical methods for cognitive modelling (and therefore to cognitive linguistics, because the same kinds of models can be used there), musicology, and neuroscience (both in a better understanding of brain function and with new methods for neural signal analysis). Long term outcomes are likely to be computer systems that help music education, that play music musically, and that interact with human musicians musically; understanding that helps musicians do what they do more effectively; and understanding that helps brain scientists and psychologists understand more about how the brain and the mind work. Above all, since musicality is so fundamental to humanity, the project aims to help understand some of what it means to be human.
Description We proposed a new information-theoretic measure of the complexity between random variables, called "binding information", which is sensitive to the presence of higher-order interactions. We related this to other information-theoretic measures such as entropy, predictive information and erasure entropy. For binary random variables, binding information is maximized by a parity process.

We also extended our previous work on symbolic predictive information rate (PIR) to real-valued signals such as autoregressive Gaussian processes AR(N) and moving average processes MA(N). We showed that PIR is unbounded for MA(N) processes. Applying this to analysis of musical audio (Steve Reich: Drumming), we found that Bayesian surprise and predictive information rate correspond to significant events in the score and structural features of the music.

We introduced a music composition tool based on searching a space with varying predictive information rate, formed as a trade-off between periodicity, repetition and noise. This tool, the "Melody triangle", was implemented firstly within a room using a Kinect motion sensor, and subsequently as an Android smartphone app. The Melody triangle allows people to indicate which melodies they "like", allowing us to survey user preferences and compare against our information measures.
Exploitation Route In music, the measures of information applied to music could give music composers and listeners new insights into the structure of music that they work with. It could also lead to accessible adaptive composition tools allowing users to select the complexity of music they prefer. By matching preference to musical experience of listeners, it could also be used as part of music recommendation systems. Other potential applications include improved prediction models for music coding and compression.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software)

Description The Melody Triangle Android App was released to allow users to create music using the concepts from this project, and has been installed over 1000 times. For more details see
First Year Of Impact 2013
Sector Digital/Communication/Information Technologies (including Software),Leisure Activities, including Sports, Recreation and Tourism
Impact Types Cultural

Description Platform Grant: Digital Music
Amount £1,161,334 (GBP)
Funding ID EP/K009559/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Academic/University
Country United Kingdom
Start 01/2013 
End 01/2018
Description Workshop on Information Dynamics of Music (IDyOM) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact A one-day workshop providing a forum for dissemination and discussion of cutting edge research on dynamic predictive processing of musical structure in: probabilistic and information-theoretic models; cognitive, psychological and neural processing; musicological analysis.
Year(s) Of Engagement Activity 2013