Platform Grant: Digital Music

Lead Research Organisation: Queen Mary University of London
Department Name: Sch of Electronic Eng & Computer Science

Abstract

The Centre for Digital Music (C4DM) at Queen Mary University of London is a world-leading multidisciplinary research group in the field of Music & Audio Technology. Established in 2001, and now with over 60 members, including academics, research staff/fellows, research students, and regular visitors, we are arguably the largest group in this area in the world. Much of our research is within the important RCUK Digital Economy / Communities and Culture theme, and our core competence in Digital Signal Processing has already been identified by EPSRC as "important to the UK" and an area for growth. The UK has the largest creative sector in Europe, and one of the world's largest music industries (CBI: Creating Growth, 2010).
Over the last 5 years, the music industry has undergone massive changes, with new digital and streaming services such as Spotify opening up new opportunities for startup businesses and SMEs, as well as large-scale music research. In parallel with this, as part of a strategic investment in creative technology and digital media at Queen Mary ("qMedia"), we have expanded our group to strengthen our expertise in music perception & cognition, music & language, and performance, to complement our existing strengths in signal processing, music informatics, and interaction.
This aim of this Platform Grant is to provide underpinning funding to allow us to continue to take a strategic view of our research. Our short- and medium-term strategy is threefold: "Music is out there" (Exo) plus "Music is in here" (Endo), built on underlying core competencies such as in digital signal processing (Core). We will do this through: undertaking adventurous feasibility studies, allowing rapid progress towards our strategic objectives; retaining strategic skills in Key RAs, maintaining important strengths and competencies in the group; strengthening our international links and building new interdisciplinary links through internship and outreach activities; maximizing the impact of our research through a new knowledge exchange strategy, including business partnerships and reproducible software and data; and undertaking public engagement activities to promote our research to a wide audience. We will also continue to help our RAs successfully develop their careers towards academia or industry, through activities such as mentoring, guest lectures, event organization, exchange visits and training courses, and we will also introduce new mentoring schemes for Investigators to maximize their potential.
Over the 5 years of the proposed Platform Grant, this concerted programme of research will achieve a step-change in modelling of human music processing and in its application to challenging industrial-scale digital music opportunities.

Planned Impact

Potential beneficiaries of the research supported by this Platform Grant outside of the academic research community include anyone who could benefit from latest research in audio and music. Examples from various sectors are given below.

Commercial private sector:
* Commercial companies designing audio equipment, through easier access to new audio research
* Companies wishing to provide new ways for their customers to discover or access music based on music recommendation research
* Musicians and sound artists, through their ability to use new research methods and processes for new creative outputs
* Computer games companies, for improved game music and audio, or for new types of audio or music-based computer games
* Hearing aid and cochlear implant manufacturers, through access to new research applicable to hearing improvement
* Audio archiving companies, through access to the latest algorithms and methods for music information retrieval
* Television and radio companies, through ability to use the latest research in the design of new technology-based programmes or in new audio production processes.

Policy-makers and others in government and government agencies:
* Police and security services, through access to methods for analysing and separating audio signals.

Public sector, third sector and others:
* Healthcare workers who work with music and audio, such as music therapists, through new music and audio visualization tools
* Museums, through tools to measure and visualise acoustic properties of objects
* Libraries, through improved open access to our research results for the benefit of their users
* Standards organizations, through access to new research methods on which to base forthcoming standards
* Science promotion organizations, through availability of high-quality usable research tools attractive to people who may be interested in science.

Wider public:
* People interested in exploring music or other audio recordings at home, school, college or university, using the latest research, either for educational or general interest purposes
* Teachers in schools, colleges or universities who want to use our research-based software for teaching audio or music
* Audiences of creative output involving audio and music, through availability of new creative outputs or technology facilitated by our audio and digital music research.

Researchers employed on the project, other members of the group, or visitors to the group:
* Improved skills in research in the digital music sector, which may be transferred into e.g. the commercial private sector when they leave the group.
 
Title Art Inside Music (curated by Paul Edlin, QMUL music director) 
Description This event organised by C4DM is part of the C4DM Concert series 2015-2016. The performance was part of "Some Things Rich and Strange", an Inside Out Festival and Music@QMUL event. The performance lasted 40 mins. Arno Babajanian's epic Piano Trio in F# minor performed by Elaine Chew (piano), Hilary Sturt (violin), and Ian Pressland (cello), with music-driven spectral art created by Alessia Milo. Babajanian's dark, ethereal, and wild trio was illuminated by specially designed vectorial drawings created in Processing, playing with time, light, darkness, colours, frequencies, and their spectral energy. The spectral art was generated in real-time in response to the music and projected on the musicians. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2015 
Impact C4DM already has a world-leading reputation in research. Through the concerts, which are free for the public to attend, C4DM research and researchers have benefited from public exposure and being able to test and validate their technology "in the wild". The concert series demonstrates the potential for C4DM to be a driver for musical innovation in London and to have a more prominent profile in the larger arts community. Working with external musicians to present these concerts presents scope for future collaborations. 
URL https://sites.google.com/site/c4dmconcerts1516/home/classical/artinsidemusic
 
Title C4DM RTSFX project at SONAR+D 2016 festival 
Description RTSFX (Real-Time Sound Effects) is an online library of real-time synthesised (rather than sampled) sound effects for a variety of different objects and environmental sounds; 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2016 
Impact Report from Will Wiklinson, C4DM PhD student: For our RTSFX project, the most encouraging thing was the number of people that interacted with our sound effects demo and how easily they grasped the concept, even if they had no experience in the area. This was the aim of the project - to engage non-technical people in a conversation about sound synthesis and why it is worth pursuing - to help them understand its potential and encourage more interest / investment in synthesis research. Around the same time / just after Sonar our sound effects website was presented to industry professionals (sound designers working on film/tv), and observing so many non-technical people playing with the site at Sonar helped me understand better the way people will interact with it and lead to some tweaks being made to make the experience more reliable and enjoyable. 
URL https://sonar.es/en/2016/
 
Title C4DM project Tape.pm at Sonar+D 2016. 
Description Several C4DM creative projects were shown in public for the first time at Sonar+D 2016. Amongst these was Tape.pm by CDT-funded PhD student Victor Loux, an interactive object which explores novel ways to record and share an improvisation that is created on a musical instrument (at Sónar this was Collidoscope, another C4DM research project). Report from Victor Loux: "Tape.pm is a project that developed during my Masters' advanced placement at b00t consultants, that links with the Collidoscope (developed by Ben Bangler and Fiore Martin) which was also shown at Sónar and I helped with setting up. Tape.pm explores novel ways to record and share an improvisation that is created on a musical instrument (at Sónar this was Collidoscope). Tape.pm was praised by attendants at the conference for its simple and creative approach to recording - and that it encouraged further exploration by the attendants. As such, the design of the object is purposefully ambiguous to encourage this exploration. The exhibition of Tape.pm at Sónar+D allowed me to gain first-hand access to public feedback about this piece and to gauge general interest and reactions and to observe how people engaged with it as well as to take notes about how to improve its design. Importantly, doing so at Sónar+D allowed me to place myself in a professional context at an international event and to engage with a much more focussed audience - people attending were specifically interested in audio and interaction. Outcomes: this initial testing with an international audience at Sónar+D, which was largely composed of non-musicians, allowed me to formulate more specific questions and testing goals for a further formal study in a controlled environment that happened a few weeks later. Because Tape.pm gives a take-away "souvenir" code, I was able to use web analytics to analyse the actual usage and timings of people using Tape.pm even after the event finished, and use this data as a basis from which to form more realistic conditions for a controlled "lab" study. The outcomes of the observations Sónar+D will be used in the APP report for this project." 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2016 
Impact Tape.pm was praised by attendants at the conference for its simple and creative approach to recording - and that it encouraged further exploration by the attendants. 
URL https://sonar.es/en/2016/
 
Title Chinese New Year Concert (curated by Luwei Yang) 
Description The event took place on Monday 1 February 2016. The full performance lasted 120 mins. The Chinese New Year Concert is organised jointly by the Chinese Students' Association (CSSA) and Confucius Institute at Queen Mary. It not only celebrates Chinese New Year, but celebrates the fusion of arts and science at Queen Mary, combining music and performance with digital instruments. The 2016 event featured award-winning Chinese instrumentalists, C4DM members and the Magnetic Resonator Piano and the Mood Rater music technology. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2016 
Impact C4DM already has a world-leading reputation in research. Through the concerts, which are free for the public to attend, C4DM research and researchers have benefited from public exposure and being able to test and validate their technology "in the wild". The concert series demonstrates the potential for C4DM to be a driver for musical innovation in London and to have a more prominent profile in the larger arts community. Working with external musicians to present these concerts presents scope for future collaborations. 
URL http://www.qmul-cssa.org.uk/concert2016/
 
Title Computer Music! by Stephen Travis Pope and other deep neural networks (curated by Bob Sturm) 
Description The event took place on Monday 23 November. The full performance lasted 90 mins. Featuring music by Stephen Travis Pope, Oded Ben-Tal, Dan Stowell, and Bob L. Sturm. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2015 
Impact C4DM already has a world-leading reputation in research. Through the concerts, which are free for the public to attend, C4DM research and researchers have benefited from public exposure and being able to test and validate their technology "in the wild". The concert series demonstrates the potential for C4DM to be a driver for musical innovation in London and to have a more prominent profile in the larger arts community. Working with external musicians to present these concerts presents scope for future collaborations. 
URL https://sites.google.com/site/c4dmconcerts1516/home/fixedmedia/computermusic
 
Title D-Box concerts 
Description Public evening concert at QMUL with a 'hackable' digital musical instrument featuring 10 performers of varying backgrounds. Showed a range of creative techniques including circuit bending. Repeated in part one month later at a regular concert series at a pub near Goldsmiths College, London. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2014 
Impact Musicians will use the instruments in further performances of their own (including internationally, e.g. Japan tour by one musician). Interest by audiences in learning more about instrument design and technical platform. 
 
Title Inspired by Digital: Final Concert 
Description World preimere at London's Cadogan Hall of the five compositions created through the Inspired by Digital project by young composers using technologies developed by Centre for Digital Music researchers. Performers included members of the London Chamber Orchestra. 
Type Of Art Composition/Score 
Year Produced 2013 
Impact Impact included technology outreach to young composers and to the general public. 
URL http://www.qmul.ac.uk/publicengagement/activities/items/89531.html
 
Title Inspired by Digital: Study Day 
Description At the Study Day, C4DM researchers Elaine Chew (MuSA_RT visualisations of tonal space), Andrew McPherson (Magnetic Resonator Piano), and Andrew Robertson (B-Keeper and its family of beat trackers) presented their technologies and underlying research and theories. Contemporary composers Lisa Bielawa, Cheryl Frances-Hoad, Eduardo Miranda, and Gabriel Prokofiev led creative discussions of ways emerging composers could integrate these technologies in a new work. The programme concluded with a showcase of performances featuring these technologies. Composition proposals were received from ~40 young composers, from which 5 were selected for the ensuing workshop and performance. 
Type Of Art Composition/Score 
Year Produced 2012 
Impact Young composers were introduced to new music technologies, and given an opportunity to write new music for the technologies. 
URL http://elainechew-research.blogspot.co.uk/2012/11/1112-inspired-by-digital-call-for.html?q=Inspired+...
 
Title Inspired by Digital: Workshop Day 
Description Five new works by young composers are workshopped in an open rehearsal with a London Chamber Orchestra ensemble, conductor Fergus Macleod, composer Cheryl Frances-Hoad, and researchers Elaine Chew, Andrew McPherson, and Andrew Robertson from QMUL's Centre for Digital Music. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2013 
Impact Music and technological education and outreach to the university population. 
URL http://www.qmul.ac.uk/publicengagement/activities/items/89531.html
 
Title Mood Conductor Performance at ACII2013 
Description Mood Conductor is an audience-performer interactive system developed to let audience members contribute to the creation of improvised performances by conveying emotional cues to performers. The system relies on a web app and a visualisation system. The Mood Conductor app can be used from mobile devices or laptops. Mood Conductor was selected as the single artistic event of the fifth biannual Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII 2013) which was held in Geneva, Switzerland on September 2-5, 2013. The performance was held in the International Conference Centre of Geneva (ICCG), near the United Nations headquarters. The performance was given by a professional vocal quartet (VoXP) from Strasbourg (France) with whom we collaborated. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2013 
Impact The performance was very successful and a large number of audience members attended it (about 250 people). More than 100 audience members interacted with our Mood Conductor technology during the performance, creating a novel type of concert experience. We received a lot of very positive feedback on the system from the Affective Computing community gathered by the ACII conference. 
URL https://www.flickr.com/photos/matdiffusion/sets/72157634162667686/
 
Title Mood Conductor Performances at the Barbican Arts Centre 
Description Mood Conductor is an audience-performer interactive system developed to let audience members contribute to the creation of improvised performances by conveying emotional cues to performers. The system relies on a web app and a visualisation system. The Mood Conductor app can be used from mobile devices or laptops. Two public performances were held with the Mood Conductor system during the Hack The Barbican festival at the Barbican Arts Centre in London in August 2013. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2013 
Impact These two performances contributed to make the Hack The Barbican festival a very successful event gathering a lot of visitors. Audience members enjoyed the novel experience made possible by the Mood Conductor system allowing them to interact with the performers over the course of the concert. The performances also helped improving the system to prepare another performance which was held in Switzerland (Geneva) as part of the International Conference on Affective Computing. 
URL http://lanyrd.com/2013/hackthebarbican/scmwgx/
 
Title Musical Banditry (curated by Elaine Chew) 
Description The event took place on Monday 21 December 2015. The full performance lasted 60 mins. As Stravinsky famously said, "Lesser artists borrow, great artists steal." The art of stealing was illustrated by Bob Sturm's Concatenative Variations of a Passage by Mahler and by Paul Lansky's Etudes and Parodies (winner of the 2005 International Horn Society competition) for horn trio as performed by Joel Ashford, Hilary Sturt, and Elaine Chew. Part of Lansky's piece was compared to the Beatles' While My Guitar Gently Weeps, as performed by Florian Guillaume, Giulio Moro, Keunwoo Choi, and Elio Quenton, culminating in a concurrent performance of the two. Musical similarities were made explicit using C4DM research in music informatics. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2015 
Impact C4DM already has a world-leading reputation in research. Through the concerts, which are free for the public to attend, C4DM research and researchers have benefited from public exposure and being able to test and validate their technology "in the wild". The concert series demonstrates the potential for C4DM to be a driver for musical innovation in London and to have a more prominent profile in the larger arts community. Working with external musicians to present these concerts presents scope for future collaborations. 
URL https://sites.google.com/site/c4dmconcerts1516/home/classical/musicalbanditry
 
Title Mycelia by Imogene Heap 
Description Mycelia is an experimental project by the artist Imogen Heap based on a fair trade music eco-system. As quoted in the Guardian: "Imogene Heap's current project is to bring to life an entirely new landscape for distributing and monetising music and all its related data and content. Spurred on by the technology originally designed by libertarians to create the crypto-currency bitcoin, she's releasing her next song, Tiny Human, as an event and an experiment. What she hopes to emerge is the core of a revolutionary system she refers to as Mycelia. It could completely transform the music industry." 
Type Of Art Composition/Score 
Year Produced 2016 
Impact Imogen Heap (UK) is releasing raw music material (i.e., multi-track audio, lyrics, album cover and credits) suitable for IM AF encoding. The IM AF format is part of a software application developed by C4DM (see appropriate section of this report for further details of the software). 
URL http://myceliaformusic.org/
 
Title New Interfaces Concert (curated by Andrew McPherson and Astrid Bin) 
Description Wednesday 4 November 2015, 7pm This event is part of the C4DM Concert Series 2015-2016. The full performance lasted 120 mins. Laurel Pardue performed on an augmented violin with live visuals. Dianne Verdonk performed on "La Diantenne", a new metal-sheet instrument capturing rich gestural interactions. Tim Exile performed on his "Flow Machine", a hardware/software setup for live sampling and transformation of sound. Bogdan Vera and friends performs on the D-Box. The performance was coupled with a study of audience response conducted by C4DM/MAT PhD student Astrid Bin. The audience was introduced to the instruments before the performance and record their responses during the performance using a web-based system developed by Astrid. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2015 
Impact C4DM already has a world-leading reputation in research. Through the concerts, which are free for the public to attend, C4DM research and researchers have benefited from public exposure and being able to test and validate their technology "in the wild". The concert series demonstrates the potential for C4DM to be a driver for musical innovation in London and to have a more prominent profile in the larger arts community. Working with external musicians to present these concerts presents scope for future collaborations. 
URL https://sites.google.com/site/c4dmconcerts1516/home/newinterfaces/newinterfacesconcert
 
Title Open Symphony Audience Labs project (M Barthet, C4DM) 
Description Open Symphony (OS) is a novel technology-mediated immersive music performance system which reimagines the music experience for a digital age. Open Symphony promotes person-person engagement and allows the social benefits of music-making to foster a meaningful mutual experience. The idea was conceived to acknowledge and capitalise on a listener's ever-growing digital presence within live music settings; designing a system to resolve the disconnect between reality and digital-reality and transform digital technology's possibilities to encourage active creative experiences. Western performing arts practices have traditionally restricted creative interactions from audiences. Open Symphony was designed to explore audience-performer interaction in live music performance assisted by digital technology. The Open Symphony system transforms the traditional unidirectional musical chain by adding creative communication in the reverse direction, from the audience to performers. During the project, the system was experienced by about 250 audience and performer participants in controlled (lab) and "real world" settings. Feedback on usability and user experience was overall positive and live interactions demonstrated significant levels of audience creative engagement. We identified further design challenges around audience sense of control, learnability and compositional structure. Both audience and performer participants who trialed the system felt challenged by the novelty-audience members took creative decisions and decoded their effects, and performers followed scores generated while they were playing. Such challenges lead to positive affective responses for some audience participants, who felt engaged in the performance and closely connected to performers, and to frustration for others, who misunderstood the system or wished to have more control. Open Symphony showed the potential to create performances that are "open," "engaging," "empowering," and "unusual". Our results highlighted the potential of our system for music pedagogy and audience engagement agendas. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2016 
Impact The Platform Grant (PG) funding enabled to (i) evaluate and improve our participatory performance digital technology by collaborating with performers and audiences outside academia through focus groups and concerts, and to (ii) disseminate our research as part of public engagement events (about 250 participants) as well as academic publications. Our audience can be segmented into expert musicians and non experts from a wide range of domains (school, university, creative industry, design, engineering, administration) and age (11-50+). Thanks to the Platform Grant, our work was published and presented in the tier 1 ACM Conference on Human Factors in Computing Systems (CHI), 7-12 May 2016, California (USA). We collaborated with musicians from University of California Berkeley and organised a series of interactive performances and an evaluation study at CHI 2016. This helped strengthen an ongoing publication which was accepted in a high impact factor journal in our field after we included the new results we obtained thanks to the grant (IEEE Multimedia Magazine, Impact Factor February 2017: 1.36). The PG award also allowed two PhD students to attend talks on state-of-the-art in Human Computer Interaction from top universities and companies on a variety of topics, such as ubiquitous computing, visualization, usability and user experience design. This proved a useful source of inspiration for the students' PhD research directions. We have also raised the awareness of QMUL and our EPSRC FAST-IMPACt Program Grant project at the international ACM CHI conference by showcasing short demo videos from our group at the booth. The award supported the participation of QMUL PhD students Yongmeng Wu and Leshao Zhang in the tier 1 ACM Conference on Human Factors in Computing Systems (CHI), 7-12 May 2016, California (USA). We hired professional musicians (eight in total) for our concerts using complementary funding sources (AHRC Audience Labs and QMUL Centre for Public Engagement grants, PI: Barthet). In 2015-2016, this project was awarded the Open Symphony, AHRC Sound and Music Organisation Audience Labs Award (only sixth UK projects selected). 
 
Title Participation in the monthly Spotify Music hackaton. Spotify New York 
Description Maria participated in the monthly Spotify music hackathon held at Spotify New York. She collaborated with two other researchers and created folkviz (https://github.com/nazareno/folkmusicmap) which stands for folk music visualization. This project explores three ways to visualize results from undergoing folk music research and facilitate the exploration of music similarity in big data collections. 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2016 
Impact Not known. 
URL https://github.com/nazareno/folkmusicmap
 
Title With/Without (curated by Elaine Chew) - C4DM Concert Series 2015-2016 
Description Friday 24 June 2016, 7.30pm Schott Music Shop (48 Great Marlborough St) - performance-dialogue: 75 mins (an educational event created in collaboration with Kingston University) Elaine Chew presented the world premiere of Oded Ben-Tal's Sonata for Magnetic Resonator Piano and Electronics together with performances of contemporary piano pieces by Carmine Cella, Cheryl Frances-Hoad, and Dorien Herreman's computer program MorpheuS. The concert featured dual performances of pieces, with and without the magnetic resonator piano (MRP), interspersed with performer/composer presentations on the making of the music. MRP inventor Andrew McPherson also gave an introduction to the instrument. Audience members had a chance to engage in further dialogue with the creators in an open Q&A. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2016 
Impact C4DM already has a world-leading reputation in research. Through the concerts, which are free for the public to attend, C4DM research and researchers have benefited from public exposure and being able to test and validate their technology "in the wild". The concert series demonstrated the potential for C4DM to be a driver for musical innovation in London and to have a more prominent profile in the larger arts community. Working with external musicians to present these concerts presents scope for future collaborations. 
URL https://sites.google.com/site/c4dmconcerts1516/home/classical/with_without
 
Description The nature of the Platform Grant is that there are many findings, coming from the more than 20 sub-projects that were developed and executed as part of the programme of research, as well as the workshops, concerts, artistic interventions and collaborations that permeated the project. It isn't possible to provide a brief summary of such a widely diverse and multitudinous set of findings. Briefly however, there were 86 publications in academic journals and conferences, several allied to open source software, and in once case, open hardware.

Worth highlighting are the 15 Collaborations and Partnerships that the Platform Grant produced, the 29 Engagement Activities and the 17 Artistic and Creative Products. Significant among these are the interactions with the artist, Di Mainstone, which helped start her wonderful Human Harp series of interventions that turn suspension bridges into musical instruments (Brooklyn Bridge, Clifton Suspension Bridge - see https://humanharp.org). Another significant public engagement project, Collidoscope (http://collidoscope.io) was launched at the Sonar+D in 2015, appeared there again in 2016 in revised form and the same year at Ars Electronica. This instrument is available as open source software and open hardware.
Exploitation Route The funding has already led to several grants, most notably the UKRI CDT in AI and Music. The open source software and hardware are available, as is an Open Multitrack Testbed. These continue to be used. The outcomes from the various interventions for Bela and Augmented Instruments continue to be used by academics and companies around the world.
Sectors Digital/Communication/Information Technologies (including Software),Environment,Other
 
Description There were many non-academic impacts arising from this project, as listed in the full report. Some of the artistic collaborations, especially with Di Mainstone (humanharp.org) continue to have impact and promote the work of the Centre for Digital Music. We were able to influence international standards, namely in the Media Value Chain Ontology (MVCO, ISO/IEC 21000-19) which facilitates rights tracking for fair and transparent royalties payment and in the Audio Value Chain Ontology (AVCO, ISO/IEC 21000-19/AMD1) that extends MVCO to address issues arising from content re-use and sharing in the audio domain. The most significant follow-on grant is the UKRI CDT in AI and Music (www.aim.qmul.ac.uk) which has helped strengthen and broaden the Centre for Digital Music's (c4dm) interactions with industry, including part- and fully-funded studentships. This has also helped grow c4dm to over 80 active researchers. For further details of currently active and recent grants, see http://c4dm.eecs.qmul.ac.uk/projects.html. Some funds were used to commission a professionally produced promotional video of c4dm's activities, which remains a focal point on the centre's web presence and is regularly used at meetings and events. The Platform Grant sponsored the first and some subsequent editions of the Workshop on Intelligent Music Production, the 6th of which has been re-scheduled for September 2021. In this way, the grant helped establish a successful gathering that is respected by music professionals as well as academics. (see https://intelligent-music-production.github.io) Platform Grant funding was also used to develop the Bela low latency music interface platform, including the funding for a workshop. This led to a successful spinout in 2017 (https://bela.io). The board is used widely in academic and commercial projects around the world.
First Year Of Impact 2013
Sector Creative Economy
Impact Types Cultural,Societal
 
Description Audio-Visual Media Research Platform
Amount £1,577,223 (GBP)
Funding ID EP/P022529/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 08/2017 
End 07/2022
 
Description EPSRC Early Career Fellowship
Amount £897,000 (GBP)
Funding ID EP/N005112/1 
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 01/2016 
End 12/2020
 
Description QMUL Centre for Public Engagement small award (PI: Mathieu Barthet)
Amount £1,000 (GBP)
Organisation Queen Mary University of London 
Sector Academic/University
Country United Kingdom
Start  
 
Description RA position for three years for Brecht de Man
Amount £1 (GBP)
Organisation Yamaha Motors 
Sector Private
Country United Kingdom
Start 10/2016 
End 09/2017
 
Description Trans-Atlantic Program Digging into Data Challenge
Amount £646,000 (GBP)
Funding ID ES/R004005/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 10/2017 
End 09/2019
 
Title Research tools for music interaction, learning, creation and sharing using IM AF 
Description Research tools (incl. conformance bitstreams aiming for achieving interoperability among emerging music services) for music interaction, learning, creation and sharing using IM AF are made available at: https://code.soundsoftware.ac.uk/projects/mpegdevelopments 
Type Of Material Improvements to research infrastructure 
Year Produced 2016 
Provided To Others? Yes  
Impact • The work on MPEG-A: Interactive Music Application Format (IM AF) has inspired not only commercial products such as STEMS by Native Instruments (2015) and LANDR by MixGenius (2016) but also artists such as Emilio Molina Martínez (ES), Tracy Redhead (AU) & Imogen Heap (UK) releasing raw music material (i.e., multi-track audio, lyrics, album cover and credits) suitable for IM AF encoding. Creative projects also reported experimenting with different IM AF layouts focusing on its different features (i.e. remixing, karaoke, chords, groups and rules). • The work on MPEG-M: Multimedia Service Platform Technologies resulted on MixRights: Fair Trade Music Ecosystem launched at Mycelia Hack Weekend, Sonos Studios, London, 1-3 Apr. 2016 and more recently it further presented at Interactive Music Hack-Fest, MAT & C4DM Studios, QMUL, London, 11 June 2016, Sonar+D, Barcelona, 16-18 June 2016 and Mycelia Hack Weekend, Sonos Studios, London, 8-10 July 2016. 
URL https://code.soundsoftware.ac.uk/projects/mpegdevelopments
 
Title Annotation of the meter of the GTZAN dataset (Elio Quinton) 
Description In the context of his PhD research, Elio Quinton is annotating the meter of the GTZAN dataset. There is no publicly available dataset of commercial music annotated with meter, to his knowledge. The only dataset having meter information is the Meetrens Tune Collection, which consist of Dutch folk tunes and songs either as MIDI files, audio recordings or score, but in either case are strictly monophonic melodies. As a consequence, the annotations would increase the value of a dataset that has already been used extensively in the community (more than a 100 published papers) and annotated with tempo and key. So far Elio has annotated the dataset in its entirety. However, in order to have a measure of the reliability of the annotations, the dataset should be annotated two or more times by different annotators. To complete this annotation task, Elio recruited professional musicians to carry it out. 
Type Of Material Database/Collection of data 
Year Produced 2015 
Provided To Others? Yes  
Impact The initial aim has been completed. New questions generated during the first leg of the project are being investigated. The creation of this dataset has enabled two major streams of findings. On one hand, the creation of this dataset has enabled the validation of my research work, as stated in the original grant application. On the other hand, the analysis of the corpus itself and its comparison with other existing annotation corpuses (this is ongoing work) leads to valuable findings. Some of the findings are: enabling the assessment of the quality of other publicly available annotations datasets, augmenting the value of existing corpuses (including the one created under this grant). 
URL http://www.isophonics.net/content/metrical-structure-annotations-gtzan-dataset
 
Title CHiME-Home 
Description The CHiME-Home dataset is a collection of annotated domestic environment audio recordings, described in: P. Foster, S. Sigtia, S. Krstulovic, J. Barker, M. D. Plumbley. "CHiME-Home: A Dataset for Sound Source Recognition in a Domestic Environment," in Proceedings of the 11th Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2015; and available from https://archive.org/details/chime-home 
Type Of Material Database/Collection of data 
Year Produced 2015 
Provided To Others? Yes  
Impact The dataset is being used in task 4 of the DCASE2106 Challenge, for performance evaluation of systems for the detection and classification of sound events. The challenge is organised by the Audio Research Group of Tampere University of Technology, by the QMUL Centre for Digital Music and by IRCCYN, and by the University of Surrey, under the auspices of the Audio and Acoustic Signal Processing (AASP) technical committee of the IEEE Signal Processing Society. See http://www.cs.tut.fi/sgn/arg/dcase2016/task-audio-tagging 
URL https://archive.org/details/chime-home
 
Title Multitrack Testbed (Team: QMUL) 
Description The goal of this project is to populate the recently developed Open Multitrack Testbed (multitrack.eecs.qmul.ac.uk), an online platform for storing, searching and browsing multitrack audio and mixtures thereof. It is a resource intended for but not limited to users such as researchers in MIR and intelligent music production, as well as sound engineering students and audio software developers. The researchers' aim is to realise the testbed's potential impact by including the most popular multitrack datasets around, complemented with extensive metadata. The idea of publishing a multi-track database has been raised several times and a few prior attempts have been made within the Queen Mary researchers' group. The project is led by Brecht De Man, who collected audio files for a number of multi-track recordings. György Fazekas (FAST IMPACt) advised on semantic technologies, having developed the Studio Ontology Framework which is being used in the testbed. Mariano Mora-Mcginity (also a FAST IMPACt team member) finished developing the testbed (barring solving any issues that may arise during the annotation) to a state in which it 1. shows all content for the user to browse through; 2. has a search interface to search and filter using any of the metadata fields; 3. allows an authorised user (password protected with customisable permission) to enter new metadata associated with audio on the testbed server (c4dm.eecs.qmul.ac.uk/ multitrack) or elsewhere on the web. The C4DM team has annotated over 600 multitracks have been annotated, comprising over 3000 individual tracks. Full details have been maintained about the instrumentation, genre, recording process, artifacts, attribution.... This is a sufficient number for this data to now be used in machine learning and data mining applications. It is contributing to research on music mixing practices and multitrack audio processing. 
Type Of Material Database/Collection of data 
Provided To Others? No  
Impact Several researchers have referenced the testbed in their papers since its inception, and it was well received during a session on multitrack audio datasets of the Unconference at 15th Conference of the International Society for Music Information Retrieval in Taipei, Taiwan, November 2014. Furthermore, within C4DM the local multitrack repository is extensively used for research, but as the licenses are not open - or unclear - disseminating data and results is less than straightforward. This is especially a problem when funding bodies, journals or conferences require open data for sustainability and reproducibility purposes. The research team is positive that this resource, the first of its kind, has the potential of becoming the main resource for any researcher, student or developer who needs multitrack audio and mixtures thereof, provided it has the critical mass to become appealing as a go-to platform. It is evident that there is already a strong support base for this idea, and already extensive usage of the still limited dataset. Multitrack audio is a valuable resource for a wide range of audio professionals, educators, students, developers, and researchers. Many of these communities are not that aware of C4DM and its activities. Notably, this project is expected to have significant impact with music technology students and educators, who will use the content in learning and teaching music production skills. R&D areas in which large and diverse sets of high quality, multi-stream audio are essential in music information retrieval (MIR) research, development of improved music production tools, and tasks like audio source separation, audio segmentation and research in auditory perception. 
URL http://multitrack.eecs.qmul.ac.uk
 
Title New dataset of fiddle music (Elaine Chew& Luwei Yang) 
Description As a result of employing the PhD student Luwei Yang (under the supervision of Dr Elaine Chew, Queen Mary) and in collaboration with Dr Chew, a Hidden Markov Model has been designed, with different observation likelihood distributions (Gaussian Model and Gaussian Mixture Model), for portamento detection. As part of this research, a new dataset of fiddle music has been created for evaluation of portamento detection. The dataset comprises of 20 fiddle performances by Benny Martin, Mark O'Connor, Johnny Gimble, Bonnie Rideout and Ian Walsh. 
Type Of Material Database/Collection of data 
Provided To Others? No  
Impact This research project is still ongoing. 
 
Title The Media Value Chain Ontology model (Panos Kudumakis, C4DM) 
Description Media Value Chain Ontology model for time-segments and multi-track audio including reference software and conformance datasets will be published by ISO/IEC MPEG in due time as: "ISO/IEC 21000-19 AMD 1 Extensions on Time-Segments and Multi-Track Audio". 
Type Of Material Computer model/algorithm 
Year Produced 2016 
Provided To Others? Yes  
Impact 1) Research tools (incl. conformance bitstreams aiming for achieving interoperability among emerging music services) for music interaction, learning, creation and sharing using IM AF are made available at: https://code.soundsoftware.ac.uk/projects/mpegdevelopments; 2) Software (incl. conformance bitstreams aiming for achieving interoperability among emerging music services) for music interaction, learning, creation and sharing using IM AF is made available at: https://code.soundsoftware.ac.uk/projects/mpegdevelopments; 3) Public Engagement activities / Creative & artistic projects: inspired artists such as Emilio Molina Martínez (ES), Tracy Redhead (AU) & Imogen Heap (UK) are releasing raw music material (i.e., multi-track audio, lyrics, album cover and credits) suitable for IM AF encoding. Creative projects also reported experimenting with different IM AF layouts focusing on its different features (i.e. remixing, Karaoke, chords, groups and rules). 
URL https://code.soundsoftware.ac.uk/projects/mpegdevelopments
 
Description Andrew McPherson's collaboration with Drake Music (Bella platform) 
Organisation Drake Music
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Bela is a high-performance embedded platform for creating interactive audio systems. Andrew McPherson has run a half dozen public workshops and hackathons using the platform, all of which have been well attended (most filled to capacity). One of the hackathons was a collaboration with the charity Drake Music, which focuses on creating musical instruments for disabled individuals. Following these events, he built up a network of over 20 collaborators using the hardware.
Collaborator Contribution Drake Music (charity for music making for the disabled) - held an Accessible Music Hackathon at QMUL on 6 February 2016, attendance 35 people (full capacity). Possible future host for MAT internship projects. Ongoing collaboration in their monthly DMlab instrument design sessions.
Impact Bela, the high-performance embedded platform for creating interactive audio systems, has had a major impact on researchers, makers and musicians alike. On 29 February 2016, McPherson launched a Kickstarter campaign for Bela which has been extremely successful, exceeding its £5k fundraising target it less than 4 hours and reaching 350% of its goal in 24 hours. This campaign is still underway but will result in the shipment of Bela kits to hundreds of individuals worldwide. It has also led to a substantial amount of social media attention and offers to collaborate with several major music technology platform developers.
Start Year 2015
 
Description Brecht de Man's collaboration with CITAR 
Organisation Catholic University of Portugal
Country Portugal 
Sector Academic/University 
PI Contribution The grant funded a one week placement of Brecht De Man at the Catholic University of Portugal. Brecht de Man gave a seminar on his research that was attended by students, educators and researchers from the Catholic University of Portugal. In the process of experiement participation, the student learned about C4DM technologies and research.
Collaborator Contribution The collaboration with CITAR was developed in particular with its Digital Creativity group and focused on sound design and audio engineering. The lead UCP researcher was Dr Pedro Duarte Pestana, assistant professor and researcher in audio engineering at UCP, and a recording engineer with over 15 years of experience. The students and educators at the Catholic University of Portugal participated in the experiments conducted by Brecht de Man. They have provided mixes on a relatively large number of songs, thereby providing a necessary 'mirror' set to the other large corpus of mixes that de Man has built at McGill University. Besides a direct contribution towards de Man's doctoral thesis, this experiment will lead to a Journal of the Audio Engineering Society paper.
Impact The findings of this award are contributing to a deeper understanding of music mixing practices. This is informing music technology education. The multitracks, mixes of those multitracks and evaluations of those mixes are all used in a large multitrack audio testbed. This testbed is used by music technology researchers, educators and students, in order to improve music and audio mixing. The C4DM researchers found out how and why reverberation is added to tracks in a mix. We have devised a metric, the equivalent impulse response, which is strongly correlated with how multitrack reverberation is perceived, and is meaningful for analysis and interpretation. The C4DM researchers are also investigating how mixes are made and evaluated by the students and educators at the Catholic University of Portugal, and comparing the results with similar experiments performed at other institutions across the world. This ongoing work will help us understand how music mixing practices are influenced by culture and region. A C4DM open source software tool, the Web Audio Evaluation Tool, was improved and updated based on its use in this project. Data gathered in the experiment has made a major contribution to C4DM's multitrack testbed.
Start Year 2015
 
Description Brecht de Man's collaboration with CITAR 
Organisation Catholic University of Portugal
Country Portugal 
Sector Academic/University 
PI Contribution The grant funded a one week placement of Brecht De Man at the Catholic University of Portugal. Brecht de Man gave a seminar on his research that was attended by students, educators and researchers from the Catholic University of Portugal. In the process of experiement participation, the student learned about C4DM technologies and research.
Collaborator Contribution The collaboration with CITAR was developed in particular with its Digital Creativity group and focused on sound design and audio engineering. The lead UCP researcher was Dr Pedro Duarte Pestana, assistant professor and researcher in audio engineering at UCP, and a recording engineer with over 15 years of experience. The students and educators at the Catholic University of Portugal participated in the experiments conducted by Brecht de Man. They have provided mixes on a relatively large number of songs, thereby providing a necessary 'mirror' set to the other large corpus of mixes that de Man has built at McGill University. Besides a direct contribution towards de Man's doctoral thesis, this experiment will lead to a Journal of the Audio Engineering Society paper.
Impact The findings of this award are contributing to a deeper understanding of music mixing practices. This is informing music technology education. The multitracks, mixes of those multitracks and evaluations of those mixes are all used in a large multitrack audio testbed. This testbed is used by music technology researchers, educators and students, in order to improve music and audio mixing. The C4DM researchers found out how and why reverberation is added to tracks in a mix. We have devised a metric, the equivalent impulse response, which is strongly correlated with how multitrack reverberation is perceived, and is meaningful for analysis and interpretation. The C4DM researchers are also investigating how mixes are made and evaluated by the students and educators at the Catholic University of Portugal, and comparing the results with similar experiments performed at other institutions across the world. This ongoing work will help us understand how music mixing practices are influenced by culture and region. A C4DM open source software tool, the Web Audio Evaluation Tool, was improved and updated based on its use in this project. Data gathered in the experiment has made a major contribution to C4DM's multitrack testbed.
Start Year 2015
 
Description China Summer Camp Field Research (Nick Bryan-Kinns) 
Organisation Hunan Agricultural University, Changsha.
Country China 
Sector Academic/University 
PI Contribution It was a collaboration with Hunan University (China) to create interactive artifacts inspired by the Dong ethnic minority culture in rural China, and was part of The New Channel Program 2016 (Design for Social Innovation and Sustainability Network) led by Hunan University. The QMUL team created a popup maker space in Hengling Village, Tongdao County, Hunan Province with locally-sourced found objects and materials. They explored ethnic musical box design using local materials in a remote location, off grid, building on local cultural elements to inspire our interactive designs. We worked with low-cost/ low-fi making equipment with engagement with local population, students, and designers. Four interactive artifacts were created in the rural village, exhibited in China, and returned to the UK for exhibition in London: KeepMake; Doye Boxes; Dong Shine; and Dong Tunes. Participants included two academics and three students from Hunan University, one academic and three students from QM, and one UK company.
Collaborator Contribution It was a collaboration with Hunan University (China) to create interactive artifacts inspired by the Dong ethnic minority culture in rural China, and was part of The New Channel Program 2016 (Design for Social Innovation and Sustainability Network) led by Hunan University. New relationships with researchers and design students from Hunan University were established leading to potential future research collaborations.
Impact Data from field work - video recordings, collaborative artworks created and ethnographic observations of local peoples' interaction with them. Interactive artefacts for exhibition, study, and research design inspiration. 1) Public Exhibitions: 18-25 Aug 2016 Sino-Italy design center, Huaihua, China; 9-16 Sep 2016 Inter/sections, London, UK. 2) Social Media Impact: Tweets by Nick Bryan-Kinns about the Summer Camp (with #hengling16) received 3.4k impressions in Aug 2016. 3) Reported Reported in Xinhua, China's premier news agency, and other national Chinese news outlets. Xinhua Net, China Daily, China.org. 4) Website http://eecs.qmul.ac.uk/~nickbk/hengling16/ Including: • Design Diary • Detailed descriptions, images, and pdf presentations of each artefact and the design process • Videos of the artefacts The C4DM team lead gathered detailed feedback from C4DM students on what is necessary for future rural China design field trips. Also, strengthening of existing collaboration with researchers Hunan University.
Start Year 2016
 
Description Collaboration with MPIO Seewiesen (Dan Stowell) 
Organisation Max Planck Society
Department Max Planck Institute for Ornithology
Country Germany 
Sector Academic/University 
PI Contribution Lisa Gill visited Dan Stowell from C4DM at Queen Mary in November for five days. The joint work and collaboration led to MPIO members contributing to a publication and to further grant proposals in future.
Collaborator Contribution MPIO research member Lisa Gill collaborated with Dan Stowell on the development of a method by Stowell and applied to Gill et al's data finds, something which increased detail of animal communication patterns, specifically patterns related to whether a pair of animals is a breeding pair or not.
Impact Method developed by Stowell and applied to Gill et al's data finds increased detail of animal communication patterns, specifically patterns related to whether a pair of animals is a breeding pair or not. A journal article was accepted for IEEE/ACM Trans Audio Speech and Language Processing. Preprint: https://arxiv.org/abs/1612.05489 Plans in 2016 were made for future collaboration have been with Lisa Gill and (supervisor) Manfred Gahr, Max Planck Institute for Ornithology, Seewiesen.
Start Year 2015
 
Description Collaboration with Tristan Begueria on D-Box (Andrew Mcpherson) 
Organisation National School of Mines of Saint-Étienne
Country France 
Sector Academic/University 
PI Contribution Tristan spent a 6-month placement at C4DM, Queen Mary, working on the D-Box musical instrument, which comes out of the Hackable Instruments project originally funded by Andrew McPherson's EPSRC First Grant. The project placement lasted from 1 April to 30 September 2015. The Platform Grant funded allowed to cover Tristan's stagy for an extra month to continue working on the D-Box musical instrument project with Andrew Macpherson.
Collaborator Contribution Andrew Mcpherson from C4DM has integrated results of Tristan Begueria's summer research project into the D-Box hackable instrument, which will in turn be used in workshops, performances and an upcoming crowdfunding campaign.
Impact This project and collaboration with Tristan developed the capabilities of the D-Box hackable musical instrument arising from previous support from the EPSRC Platform Grant and the EPSRC Hackable Instruments project (EP/K032046/1), producing a new way to interact with live-sampled sounds. The extra 1-month funded period for Tristan stay's at C4DM resulted in a finished instrument and a partial draft of a paper, and submission of a final manuscript will take place after we conduct a user study.
Start Year 2015
 
Description Computational Creativity, Perception, and Music collaboration (Dr. K. Agres, Prof Geraint Wiggins) 
Organisation Aalborg University
Country Denmark 
Sector Academic/University 
PI Contribution Dr. Agres led the experimental design, empirical data collection, data analysis, and writing up of the Uplifting Trance music experiments. Dr. Agres is leading the studies/papers with David Meredith, with Marcus Pearce and Moshe Bar, and with QMUL researchers (Stephen McGregor, Matthew Purver, and Geraint Wiggins).
Collaborator Contribution Collaborators on all of the projects have contributed to the experimental design and have helped write or revise the manuscripts. In the trance music study, musical change detection project, and distributional semantics experiment, the collaborators contributed methods relating to the computational modelling.
Impact This is an ongoing collaboration and paper in progress with David Meredith (Aalborg University, Denmark), and with Moshe Bar (Bar-Ilan University, Israel). Both of these projects are multi-disciplinary, involving music cognition and computer science, and music perception and visual perception, respectively. The uplifting trance paper has been published in Frontiers in Psychology: Cognitive Science and the authors have been asked to submit a book chapter for an Oxford University Press volume on Music and Consciousness (this has been submitted and is now under review). The trance work is also multi-disciplinary, involving aspects of music theory, machine learning, psychology, and music production. The following papers are still in preparation: Agres, K., Bar, M., & Pearce, M. (in preparation). A Cross-Modal Comparison of Veridical and Schematic Expectations in Music and Vision. Agres, K., & Meredith, D. (in preparation). A computational model of musical change detection and memory performance. Agres, K., McGregor, S., Rataj, K., Purver, M., & Wiggins, G. (in preparation). Modeling metaphor perception with distributional semantics vector space models.
Start Year 2016
 
Description Computational Creativity, Perception, and Music collaboration (Dr. K. Agres, Prof Geraint Wiggins) 
Organisation Bar-Ilan University
Country Israel 
Sector Academic/University 
PI Contribution Dr. Agres led the experimental design, empirical data collection, data analysis, and writing up of the Uplifting Trance music experiments. Dr. Agres is leading the studies/papers with David Meredith, with Marcus Pearce and Moshe Bar, and with QMUL researchers (Stephen McGregor, Matthew Purver, and Geraint Wiggins).
Collaborator Contribution Collaborators on all of the projects have contributed to the experimental design and have helped write or revise the manuscripts. In the trance music study, musical change detection project, and distributional semantics experiment, the collaborators contributed methods relating to the computational modelling.
Impact This is an ongoing collaboration and paper in progress with David Meredith (Aalborg University, Denmark), and with Moshe Bar (Bar-Ilan University, Israel). Both of these projects are multi-disciplinary, involving music cognition and computer science, and music perception and visual perception, respectively. The uplifting trance paper has been published in Frontiers in Psychology: Cognitive Science and the authors have been asked to submit a book chapter for an Oxford University Press volume on Music and Consciousness (this has been submitted and is now under review). The trance work is also multi-disciplinary, involving aspects of music theory, machine learning, psychology, and music production. The following papers are still in preparation: Agres, K., Bar, M., & Pearce, M. (in preparation). A Cross-Modal Comparison of Veridical and Schematic Expectations in Music and Vision. Agres, K., & Meredith, D. (in preparation). A computational model of musical change detection and memory performance. Agres, K., McGregor, S., Rataj, K., Purver, M., & Wiggins, G. (in preparation). Modeling metaphor perception with distributional semantics vector space models.
Start Year 2016
 
Description LCO New: Explore Inspired by Digital 
Organisation London Chamber Orchestra
Country United Kingdom 
Sector Private 
PI Contribution The Centre for Digital Music provided technologies, access to QMUL locations, and technical and artistic support for the events. There were frequent (2-3 times each week) meetings to coordinate the various details of the events.
Collaborator Contribution The London Chamber Orchestra provided organisational expertise, expert musicians, and a platform (concert series and education/outreach programmes) through which to create the series of Inspired by Digital events: a study day, a workshop, and a final concert.
Impact These are listed under Artistic Outputs, and include new music compositions by young composers, workshops, and concerts.
Start Year 2012
 
Description MIREX Music/ Speech Classification and Detection Task 
Organisation City, University of London
Country United Kingdom 
Sector Academic/University 
PI Contribution The research team at C4DM/QMUL led the organisation of the MIREX task in music/speech classification and detection, including: task website creation and maintenance, software development for evaluation metrics, receiving and checking submissions and extended abstracts, debugging code submissions, evaluating submitted systems, and publishing results. In addition the team contributed in creating the music/speech detection training and test datasets.
Collaborator Contribution - Music Informatics Research Group of City University London (Dr Tillman Weyde, Dr Dan Tidhar): task announcement; led the creation of the training and test datasets for the music/speech detection task; gave feedback on task design and organisation. - Department of Informatics, University of Piraeus, Greece (Dr Aggelos Pikrakis): led the creation of the test dataset for the music/speech classification task; gave feedback on the design of aforementioned task. - The British Library, Digital Scholarship Department & British Library Labs (Dr Aquiles Alencar-Brayner): provided recordings from the BL Sound Archive, which were used for creating the music/speech detection test dataset.
Impact The following outputs and outcomes were recorded: 1) Successfully organised a public evaluation campaign through the Music Information Retrieval Evaluation eXchange (MIREX) framework, with two tasks: music/speech classification and music/speech detection. 2) Datasets: (i) Curated a private dataset for music/speech classification. (ii) Curated a public dataset using recordings from the Internet Archive for music/speech detection (http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection#Dataset_2) (iii) Curated a private dataset using recordings from the British Library Sound Archive for music/speech detection. 3) Received 22 code submissions from 12 international teams, and 11 extended abstracts (2-3 pages each) describing the submitted methods. 4) Task description website: http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection 5) Task results website: http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection_Results 6) 2 posters presenting submitted systems for the proposed task were presented in the MIREX poster session, taking place at the International Society for Music Information Retrieval Conference (ISMIR 2015), taking place in Malaga, Spain on 25-30 October 2015. The collaboration is in the field of music technology / music informatics, hence from the C4DM perspective it is not interdisciplinary (from another perspective, anything related to music technology can be considered interdisciplinary!).
Start Year 2015
 
Description MIREX Music/ Speech Classification and Detection Task 
Organisation The British Library
Country United Kingdom 
Sector Public 
PI Contribution The research team at C4DM/QMUL led the organisation of the MIREX task in music/speech classification and detection, including: task website creation and maintenance, software development for evaluation metrics, receiving and checking submissions and extended abstracts, debugging code submissions, evaluating submitted systems, and publishing results. In addition the team contributed in creating the music/speech detection training and test datasets.
Collaborator Contribution - Music Informatics Research Group of City University London (Dr Tillman Weyde, Dr Dan Tidhar): task announcement; led the creation of the training and test datasets for the music/speech detection task; gave feedback on task design and organisation. - Department of Informatics, University of Piraeus, Greece (Dr Aggelos Pikrakis): led the creation of the test dataset for the music/speech classification task; gave feedback on the design of aforementioned task. - The British Library, Digital Scholarship Department & British Library Labs (Dr Aquiles Alencar-Brayner): provided recordings from the BL Sound Archive, which were used for creating the music/speech detection test dataset.
Impact The following outputs and outcomes were recorded: 1) Successfully organised a public evaluation campaign through the Music Information Retrieval Evaluation eXchange (MIREX) framework, with two tasks: music/speech classification and music/speech detection. 2) Datasets: (i) Curated a private dataset for music/speech classification. (ii) Curated a public dataset using recordings from the Internet Archive for music/speech detection (http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection#Dataset_2) (iii) Curated a private dataset using recordings from the British Library Sound Archive for music/speech detection. 3) Received 22 code submissions from 12 international teams, and 11 extended abstracts (2-3 pages each) describing the submitted methods. 4) Task description website: http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection 5) Task results website: http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection_Results 6) 2 posters presenting submitted systems for the proposed task were presented in the MIREX poster session, taking place at the International Society for Music Information Retrieval Conference (ISMIR 2015), taking place in Malaga, Spain on 25-30 October 2015. The collaboration is in the field of music technology / music informatics, hence from the C4DM perspective it is not interdisciplinary (from another perspective, anything related to music technology can be considered interdisciplinary!).
Start Year 2015
 
Description MIREX Music/ Speech Classification and Detection Task 
Organisation University of Piraeus
Country Greece 
Sector Academic/University 
PI Contribution The research team at C4DM/QMUL led the organisation of the MIREX task in music/speech classification and detection, including: task website creation and maintenance, software development for evaluation metrics, receiving and checking submissions and extended abstracts, debugging code submissions, evaluating submitted systems, and publishing results. In addition the team contributed in creating the music/speech detection training and test datasets.
Collaborator Contribution - Music Informatics Research Group of City University London (Dr Tillman Weyde, Dr Dan Tidhar): task announcement; led the creation of the training and test datasets for the music/speech detection task; gave feedback on task design and organisation. - Department of Informatics, University of Piraeus, Greece (Dr Aggelos Pikrakis): led the creation of the test dataset for the music/speech classification task; gave feedback on the design of aforementioned task. - The British Library, Digital Scholarship Department & British Library Labs (Dr Aquiles Alencar-Brayner): provided recordings from the BL Sound Archive, which were used for creating the music/speech detection test dataset.
Impact The following outputs and outcomes were recorded: 1) Successfully organised a public evaluation campaign through the Music Information Retrieval Evaluation eXchange (MIREX) framework, with two tasks: music/speech classification and music/speech detection. 2) Datasets: (i) Curated a private dataset for music/speech classification. (ii) Curated a public dataset using recordings from the Internet Archive for music/speech detection (http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection#Dataset_2) (iii) Curated a private dataset using recordings from the British Library Sound Archive for music/speech detection. 3) Received 22 code submissions from 12 international teams, and 11 extended abstracts (2-3 pages each) describing the submitted methods. 4) Task description website: http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection 5) Task results website: http://www.musicir.org/mirex/wiki/2015:Music/Speech_Classification_and_Detection_Results 6) 2 posters presenting submitted systems for the proposed task were presented in the MIREX poster session, taking place at the International Society for Music Information Retrieval Conference (ISMIR 2015), taking place in Malaga, Spain on 25-30 October 2015. The collaboration is in the field of music technology / music informatics, hence from the C4DM perspective it is not interdisciplinary (from another perspective, anything related to music technology can be considered interdisciplinary!).
Start Year 2015
 
Description Marcus Pearce's collaboration with Bastiaan van der Weij: A probabilistic model of musical metre perception 
Organisation University of Amsterdam
Country Netherlands 
Sector Academic/University 
PI Contribution Funding from the Platform Grant has allowed C4DM to offer a three month research internship as part of the "Internship scheme to attract excellent PhD students and RAs from other universities (from a few days up to 3 months)." Bastiaan van der Weij's stay at C4DM was partly covered by the University of Amsterdam and partly by C4DM. For his PhD research, van der Weij is working on a probabilistic model of rhythm perception, using methodologies very similar to those used by Marcus Pearce in his IDyOM framework. Both models are probabilistic and incremental models whose parameters are estimated from a large body of data. Both models aim to model the flux of listener's expectation as a piece of music unfolds, as a function of that listener's earlier musical experience. Both models use predictive distributions and information content to quantify these expectations.
Collaborator Contribution The research internship served as an opportunity for van der Weij to become strongly familiar with Pearce's research and to be able to exchange ideas in person on a regular basis. The internship provided a valuable opportunity for broadening van der Weij's experience and extending his network. Similarly, the visit to Queen Mary provided an opportunity for researchers at the Music Cognition Lab to be introduced to the research done at the Music Cognition Group in Amsterdam. Thus, the objectives of the research internship were: 1) Integrating a novel model of rhythm perception into the IDyOM framework, under supervision and guidance of Marcus Pearce, who originally developed this framework; 2) Kicking-off a partnership between Bastiaan van der Weij and Marcus Pearce, who acted as co-supervisor during van der Weij's stay at C4DM.
Impact The work has demonstrated that probabilistic methods can predict metrical categories in music perception with significant accuracy and identified advanced modelling methods that further increase accuracy. This lays the ground for investigating probabilistic models of metre perception as a way of simulating and understanding the effects of cross-cultural exposure on musical metre perception. The collaboration strengthened ties between the Music Cognition Group at the UvA and the Music Cognition Lab at QMUL.
Start Year 2015
 
Description Marcus Pearce's collaboration with the Science Musium in London 
Organisation Science Museum Group
Department The Science Museum
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution The project consisted of a Live Science Residency in the Science Museum - the Queen Mary team (Marcus Pearce and Léna Delval) spent 5 weeks in the Museum running experiments on musical preference with visitors to the gallery. The purpose was twofold: first, to facilitate data collection and second, to engage the public with our research. The project employed Léna Delval as a research assistant on a fixed term contract for 4 months. Léna is now applying for PhD positions.
Collaborator Contribution Live Science Residency placement at the Live Science Museum.
Impact Pearce's (project lead) and Delval's (researcher assistant employed on the project) Live Science Residency took place in November and December 2015. Between 11/11/15 and 11/12/15, the Residency attracted 371 visitors ranging in age from 8 to 78, who often came as families specifically to take part in our research and find out more about musical preferences.
Start Year 2015
 
Description Open Symphony (Mathieu Barthet & Kate Hayes) 
Organisation Guildhall School of Music & Drama
Country Afghanistan 
Sector Academic/University 
PI Contribution M. Barthet and his team designed and developed technological and artistic components for the creative participation of audiences of live music performances. They organised interactive concerts and conducted evaluation studies with performers and audiences. They disseminated the research through academic publications and online blog, photo and video resources.
Collaborator Contribution The musician partner, Kate Hayes, acted as music director of the project and developed some of the artistic components of the system. She led the artistic part in music rehearsals and performances.
Impact This multidisciplinary collaboration allowed to innovate combining art, science and technology: - Delivered public engagement events in UK and abroad reaching a wide audience (from school students to seniors from various background) - Publication in high impact factor journal - Award of other funding (QMUL Centre for Public Engagement)
Start Year 2015
 
Description Participation in BBC Audio Research Partnership 
Organisation British Broadcasting Corporation (BBC)
Department BBC Research & Development
Country United Kingdom 
Sector Public 
PI Contribution Prof Sandler attends meetings of the steering board of the BBC Audio Research Partnership. Other activities include open research events, e.g. at BBC Salford or New Broadcasting House
Collaborator Contribution Contributions to research in EP/L019981/1, especially in writing the grant. Hosting internship students from our CDT in Media and Arts Technology, and from the Centre for Digital Music.
Impact Research grants, from both EPSRC and TSB - see web site for details.
Start Year 2011
 
Description Research visit by Maria Pantelli, New York University (NYU) 
Organisation New York University
Country United States 
Sector Academic/University 
PI Contribution Pantelli's PhD is part of a bigger collaboration, namely the "Deep History of Music (DHOM)" project, involving researchers from six universities. This research visit was expected to contribute to new data resources collected from NYU and affiliated institutions and is also linked to a grant proposal, written by professor Armand Leroi, Imperial, in collaboration with Queen Mary. During Maria's stay at New York University between 22 May and 15 August 2016 she had the opportunity to meet with people from both academia and the industry. In particular she attended a meeting between academics of New York University and NYU Abu Dhabi discussing potential collaborations between their institutions and Queen Mary. Following this meeting she has been invited to attend the 3rd workshop on "Cross-disciplinary and Multicultural Aspects of Musical Rhythms", Abu Dhabi, 17-20 March 2017, as a way to keep the NYU-QM collaboration going. What is more, she attended local events held at Spotify New York and she presented her work to industry-related audience promoting research of Queen Mary University of London. The findings of this work are summarized in a research paper (see next section) that has been submitted to the 42nd IEEE International Conference on Acoustics, Speech and Signal Processing 2017. The work focuses on the characterisation of singing styles in world music and includes the development of a software package and an interactive visualization. For testing the algorithms developed in this research, music/speech annotations were created for a corpus of 360 world music recordings and melody contours were annotated for a set of 30 world music recordings.
Collaborator Contribution The purpose of this research visit was to increase knowledge and resources from collaboration with the music technology group at New York University (NYU). Maria and collaborators developed a software package and an interactive visualisation for characterisation of singing styles in world music. She also developed methods for the representation of music metadata using graphs, in particular hypergraphs, and the application of community detection algorithms to identify meaningful partitions of the graph. Maria collaborated with faculty and students in the Music Research Lab at NYU, as evidenced by the co-authored paper listed below: M. Panteli, R. Bittner, J. P. Bello, and S. Dixon, "Towards the Characterization of Singing Styles in World Music," IEEE International Conference on Acoustics, Speech and Signal Processing, submitted.
Impact For testing the algorithms developed in this research, music/speech annotations were created for a corpus of 360 world music recordings and melody contours were annotated for a set of 30 world music recordings. Other outcomes of this visit included the representation of music metadata using graphs, in particular hypergraphs, and the application of community detection algorithms to identify meaningful partitions of the graph. Maria and collaborators developed a software package and an interactive visualisation for characterisation of singing styles in world music. She also developed methods for the representation of music metadata using graphs, in particular hypergraphs, and the application of community detection algorithms to identify meaningful partitions of the graph. Maria collaborated with faculty and students in the Music Research Lab at NYU, as evidenced by the co-authored paper listed below: M. Panteli, R. Bittner, J. P. Bello, and S. Dixon, "Towards the Characterization of Singing Styles in World Music," IEEE International Conference on Acoustics, Speech and Signal Processing, submitted.
Start Year 2016
 
Description Steve Reich's Clapping Music application (Marcus Pearce, Sam Duffy) 
Organisation London Sinfonietta
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution The C4DM members Sam Duffy and Marcus Pearce have been writing up the research previously conducted as journal articles for publication.
Collaborator Contribution The workshops in schools were conducted by Sam Duffy (PG funded) and Tim Palmer. Tim has been contributing to the journal articles we are writing. The London Sinfonietta were involved in the original development of the app (not funded by the PG) and have contributed to the journal articles in terms of proof reading, commenting and supplying images and information.
Impact Duffy, S. and Pearce, M. (in review). What makes rhythms hard to perform? An investigation using Steve Reich's Clapping Music. Submitted to PLoS One. Duffy, S., Palmer, T., Burke, A., Palczynski, B. & Pearce, M. (in prep.). Using Digital Technology to Stimulate Engagement with a New Music Genre: Steve Reich's Clapping Music App. The collaboration is multi-disciplinary - Tim Palmer is a professional musicians and senior lecturer at Trinity Laban. Andrew Burke and Barbara Palczynski work at the London Sinfonietta, an orchestra specialising in contemporary classical music. Winner of the 2017 Public Engagement Involve Award at the QMUL Engagement and Enterprise awards.
Start Year 2016
 
Title Audio Segmentation VAMP plugin 
Description This software analyses musical audio files and divides them into musically meaningful segments, typically 4-10 per song. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2008
Licensed Yes
Impact It is often downloaded with Sonic Visualiser - see other IP entry
 
Title Sonic Visualiser 
Description Sonic Visualiser is an application for viewing and analysing the contents of music audio files. This is an open source software framework that works with its own plugin architecture called VAMP. Sonic Visualiser is Free Software, distributed under the GNU General Public License (v2 or later) and available for Linux, OS/X, and Windows. Sonic Visualiser is intended for use by people in various academic disciplines, but also by non-academics, professional audio users, and interested hobbyists. It supports loading additional analysis plugins that other developers or institutions can publish, and at least 10 institutions (besides QM) have chosen to publish plugins in this format. Sonic Visualiser was first published by the Centre for Digital Music in 2007 and has been continuously developed at the Centre and maintained as free, cross-platform open-source software ever since. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2006
Licensed Yes
Impact According to our ongoing user survey that has been carried out since the start of 2014 (having 1053 responses to date, all from people who were actually using Sonic Visualiser at the time): * 49% of respondents use Sonic Visualiser primarily in a personal capacity, 10% for professional work, 33% in academia at some level (the rest "other"); * 46% use it primarily in the field of musicology or music analysis, 19% in music composition or production, 16% in audio engineering or signal processing, 5% in software development, 2% speech processing (the rest "other"); * 55% said they found it "very enjoyable" to use and 41% "moderately enjoyable"; * 27% said they found it "very easy" to use and 62% "moderately easy"; * 59% of respondents had some additional analysis plugins installed on top of the default configuration; among these respondents, the mean number of additional plugin sets installed was 3.4. Sonic Visualiser's version-update-checker logs keep a count of distinct IP addresses from which SV has been used (for users who have agreed to this). They show over 138,000 distinct addresses during the past year, with around 4,000 distinct addresses per week on average. An impression of the extent of usage can be gained by viewing illustrative Youtube videos about sound analysis made by third parties: at a quick scan I count at least 100 that show Sonic Visualiser, of varying quality. Sonic Visualiser has also been used in online course material, such as the popular Coursera "Audio Signal Processing for Music Applications", as well as in courses for academics in non-technical disciplines, such as the Digital Humanities at Oxford Summer School. This is a free, open source project; it was used as an Impact Case study for REF 2014. See the Sonic Visualiser website for details: http://sonicvisualiser.org Link to a page about the new release: http://sonicvisualiser.org/new-in-v3.html Sonic Visualiser has been available for about a decade now, and this is one of the most substantial updates it's ever had (hence the updated major release number).
 
Title Soundbite 
Description Soundbite is an iTunes plugin developed under this project. It is currently available from www.isophonics.net as freeware. Commercial licences are under negotiation. Under the Platform Grant it was adapted for an Android platform and reconstructed as a client-server architecture. 2017: A commercial licence for the core technology that was signed with Music XRay, a company based in the USA, several years ago is starting to yield royalties. 
IP Reference  
Protection Copyrighted (e.g. software)
Year Protection Granted 2008
Licensed Yes
Impact It has been downloaded to hundreds of (unregistered) users. It was deployed in various grants to analyse collections of music for recommendation, navigation and play listing. It has been deployed in several commercial music aggregators workflows, enabling trials of large scale music recommendation systems.
 
Title AVA - automatic vibrato and portamento detection and analysis tool (Elaine Chew & Luwei Yang). 
Description AVA is an automatic vibrato and portamento detection and analysis tool. It accepts raw audio and automatically tracks the vibrato and portamento to display their expressive parameters for inspection and further statistical analysis. The applications of AVA include music education and expression analysis, and its outputs provide a useful base for expression synthesis and transformation. A free application for automatic vibrato and portamento analysis has been developed an made available online-see link below. An appropriate copyright agreement accompanies each download. 
Type Of Technology Software 
Year Produced 2017 
Impact Not applicable at this time. 
URL https://luweiyang.com/research/ava-project/
 
Title Accessible Peak Level plugin 
Description Peak level meters, along with other components which rely on the sense of sight for their use, are inherently inaccessible to people living visual impairments. Our AccessiblePeakMeter is the first plug-in which makes these previously inaccessible meters completely accessible. It uses real-time sonification to deliver information to the user about audio levels and peaks in audio signals, and so supports core activities in audio production. The AccessiblePeakMeter comes as a VST, AU or AAX plug-in, two of the main industry standards for the deployment of digital audio effects into professional DAWs (e.g. Cakewalk Sonar, Cockos Reaper, Ableton Live). The plug-in can be run on both Windows (32/64) and Mac platforms and it is free for download and completely open source. It is released under the Cokos WDL license, which in short means you can alter it and redistribute it freely, even without providing the source code of your derivative work. 
Type Of Technology Software 
Year Produced 2015 
Open Source License? Yes  
Impact AccessiblePeakMeter (http://depic.eecs.qmul.ac.uk/apm/downloads.html#download) received the Best Solution by a Large Organization award in the AT&T and New York University Connect Ability Challenge, a global software competition for software technologies that improve the lives of people living with disabilities. The winner were announced on Sunday July 26 2015, the day of the 25th anniversary of the Americans with Disabilities Act. AccessiblePeakMeter was also featured in the issue 19 of the Computer Science for Fun magazine, which explains the AccessiblePeakMeter in a bright, simple and effective way. AccessiblePeakMeter was shortlisted for QM Public Engagement Award 2015. 
URL http://depic.eecs.qmul.ac.uk/apm/
 
Title Accessible Spectrum Analyser 
Description AccessibleSpectrumAnalyser is a real time spectrum analysis plug-in that allows visual impaired users to inspect spectrograms using the same sonification as the accessible peak meter. Instead of monitoring the amplitude of the audio signal though, you will be monitoring the frequency components of the signal within a customizable selection of frequencies. It is released under the Cokos WDL license, which in short means you can alter it and redistribute it freely, even without providing the source code of your derivative work. 
Type Of Technology Software 
Year Produced 2015 
Open Source License? Yes  
Impact AccessibleSpectrumAnalyser is a real time spectrum analysis plug-in that allows visual impaired users to inspect spectrograms using the same sonification as the accessible peak meter. Instead of monitoring the amplitude of the audio signal though, you will be monitoring the frequency components of the signal within a customizable selection of frequencies. 
URL http://depic.eecs.qmul.ac.uk/apm/spectrum.html
 
Title Bela Audio Platform 
Description Bela (formerly called BeagleRT) is an ultra-low-latency real-time audio platform for the BeagleBone Black embedded computer. It can be used to build musical instruments, including the D-Box hackable musical instrument developed for the EPSRC Hackable Instruments project. The software is of particular interest to audio and music researchers, especially those building real-time systems. With less than 1ms of latency between action and sound, Bela performs faster than any other computer-based environment on the market, including high-spec laptops. It also features audio-rate sampling of every analog and digital input which makes design of sensor systems convenient. In 2015, a built-in browser-based IDE was added along with support for the Pure Data graphical computer music language widely used in the digital music community. Since 2016, we have continued to add support for other programming languages and hardware accessories, as well as extending the documentation and online resources available to the community. 
Type Of Technology Software 
Year Produced 2014 
Open Source License? Yes  
Impact Bela has gained significant traction amongst researchers, musicians and hobbyists in the 2014-17 period, with the number of users and community members growing every month. On 29 February 2016, we launched a Kickstarter crowdfunding campaign for Bela, seeking £5k to build and distribute the hardware. We exceeded our funding goal in less than 4 hours (out of a 32-day campaign), ultimately raising £55k from over 500 backers. Following the campaign, we received invitations to collaborate from major academic institutions (IRCAM in France, STEIM in the Netherlands, University of Virginia, University of Edinburgh, University of Glasgow, Goldsmiths University and several others) and industry (Cycling74, makers of the popular Max/MSP software). Meanwhile, since 2014 we have held 10+ workshops on Bela and/or the D-Box instrument including 2 workshops at the Sonar music festival, a workshop at the NIME 2015 conference (Baton Rouge, LA, USA), an Audio Music Hackathon sponsored by Harman Audio, and an Accessible Music Hackathon with the charity Drake Music (held at QMUL). Following the successful Kickstarter campaign, we spun out a new company, Augmented Instruments Ltd, to commercialise the platform, and relaunched a public web shop in October 2016. We maintain several community resources, including a blog, wiki, web forum and several social media accounts. Our users actively contribute to these resources and frequently share projects they make using Bela. Community development itself has become an area of research for us (e.g. Morreale et al., "Building a maker community around an open hardware platform", Proc. CHI 2017), and we have received a grant from Innovate UK (Feb-Oct 2017) to further develop the platform and build our community. For more information see the entry for spinout company Augmented Instruments Ltd. 
URL http://bela.io
 
Title Collidoscope 
Description The Collidoscope is a unique hardware device and real-time granular sampler/synthesizer for collaborative performance and exploration that was developed as a side project by C4DM researchers (Ben Bengler and Fiore Martin). Collidoscope's intuitive 'hands-on' approach to sampling and real-time sound manipulation captivates the general public as well as professional musicians alike. 
Type Of Technology Physical Model/Kit 
Year Produced 2015 
Impact Exhibition in public, including international exhibition. Engagement with public in the design process and at public events. Dissemination of information about the project through website, twitter, and micro-videos. Dissemination of all software, hardware, and 3D designs as open source. 
URL http://collidoscope.io/
 
Title Isophonics triple store for music metadata 
Description Software is being developed to port company-specific data structures into more generic formats accepted by relevant research groups, mostly via the use of widely accepted Semantic Web (RDF) ontologies. A triple store of audio track metadata is under development, which has SPARQL endpoints at: http://138.37.95.193:8080/openrdf-workbench/repositories/iso-rec/query http://138.37.95.193:8080/openrdf-sesame/repositories/iso-rec?query (requires authentication, and urls subject to change in the future) 
Type Of Technology Software 
Year Produced 2015 
Impact It is expected that the project will help to inform decisions regarding how to store, process and share metadata and feature data from licensed audio collections. The project supports ongoing development of a Proof of Concept mobile app which creates playlists and music recommendations by interacting with a server back end. Development of server middleware which utilizes the SPARQL endpoint rather than a company-specific database is ongoing. Using the SPARQL endpoint will enable the Proof of Concept mobile app (see 12) to access audio feature data from multiple collections, rather than the single collection used to date. 
 
Title MixRights: Fair Trade Music Ecosystem 
Description This application is based on international MPEG standards for achieving interoperability such as IM AF format for interactive music services (remixing, Karaoke & collaborative music creation) and MVCO ontology for IP rights tracking aims to enable both a rich user experience and a royalties fair and transparent music ecosystem. 
Type Of Technology Webtool/Application 
Year Produced 2016 
Impact MPEG-A: Interactive Music Application Format (IM AF) offers rich user experience by supporting, among other features, multi-track audio with volume sliders for DJ mixing and lyrics for Karaoke applications, while from the other side, MPEG-21: Media Value Chain Ontology (MVCO) facilitates transparent content governance by supporting in a machine proccesable way the relationships' representation among user roles and their permissible actions on a particular IP entity. Therefore, this project/demonstrator based on international MPEG standards for achieving interoperability such as IM AF format for interactive music services (remixing, karaoke & collaborative music creation) and MVCO ontology for IP rights tracking aims to enable both a rich user experience and a royalties fair and transparent music ecosystem. Both IM AF & MVCO standards involved contributions by C4DM. This work is taking place under the auspices of ISO/IEC MPEG and it has already been approved and supported by both MPEG Reqs & Systems groups. Publication stage: • "Text of ISO/IEC 21000-19 PDAM 1 Extensions on Time-Segments and Multi-Track Audio", Ref. as ISO/IEC JTC1/SC29/WG11/N16178, Geneva, Switzerland, June 2016. • "Requirements for Media Value Chain Ontology", Ref. as ISO/IEC JTC1/SC29/WG11/N15352, Warsaw, Poland, June 2015. 
URL https://code.soundsoftware.ac.uk/projects/mpegdevelopments
 
Title Open Symphony web-based creative participation system 
Description - Node.js web application for creative participation of audiences - Processing module to dynamically generate graphic scores for performers - Web application for audiences to conduct performers 
Type Of Technology Webtool/Application 
Year Produced 2016 
Impact not known 
 
Title OpenFrameworks Add-on for gesture to envelope converter 
Description ofxEnveloper is an openFrameworks addon that converts gestures into an envelope type of graphical information, by recognising inflection points from the gesture. Thresholds for horizontal and vertical lines can also be defined. It is particularly aimed for animation and sound applications, and for use with drawing surfaces (such as Wacom tablets or touchscreen interfaces). Possible uses: 1 audio effect or EQ envelopes; 2 animation curves; 3 creation/manipulation of audio and/or visual sequences; 4 arpeggiators. Compatibility: openFrameworks 0.9.0 It is released under the Cokos WDL license, which in short means you can alter it and redistribute it freely, even without providing the source code of your derivative work. 
Type Of Technology Software 
Year Produced 2015 
Open Source License? Yes  
Impact Not known. 
URL http://depic.eecs.qmul.ac.uk/?q=ofxenveloper
 
Title Software package MOTIF 
Description During her stay at NYU, the PhD student Maria Pantelli from the Centre for Digital Music collaborated with Rachel Bittner from NYU and together they implemented the software package MOTIF: a melody description framework for audio recordings. The software is available at https://github.com/rabitt/motif . 
Type Of Technology Software 
Year Produced 2016 
Open Source License? Yes  
Impact The software is available at https://github.com/rabitt/motif. Maria also created an interactive demo for the investigation of music outliers which she demonstrated at the Monthly Music Hackathon, Spotify New York. The demo is available at http://eecs.qmul.ac.uk/~mp305/Outliers_world_tabs.html. 
URL https://github.com/rabitt/motif
 
Title SoundBite Application for iTunes 
Description SoundBite is the quickest and easiest way to create great-sounding playlists in iTunes. Once installed, SoundBite will get to know your music collection and when it's done, you're ready to create playlists to suit your mood with just a couple of mouse clicks! This updated version of the SoundBite app for iTunes works with the new Mac Operating Systems. The technology is now again accessible to many potential desktop users. 
Type Of Technology Webtool/Application 
Year Produced 2014 
Impact From a commercial perspective, the app provides a slick demonstrator for potential investors without the overhead of digging deep into mobile platforms. The app was presented during Queen Mary University of London's School of Elec. Eng. and Computer Sc. Research Showcase (3 April 2014) which attracted a lot of visitors including many industrials. The app raised the interest of our partners Omniphone and Abbey Road Studios we collaborate with on research projects. We demonstrated the app to Rocket Music, a company founded by Elton John, and are currently in discussion with them to create a new digital music product. From a pedagogical perspective, the SoundBite app represents a useful way to introduce UG and PG students to some of the Music Information Retrieval technologies developed by Queen Mary University of London. 
URL http://www.isophonics.net/LatestMacVersion
 
Title iOS Crossmodal memory game: ShapeTones 
Description ShapeTones is an audiovisual memory game for iOS (iPhone/iPad). A sequence of 3 shapes and tones ("ShapeTones") is played, and the player tries to reproduce it. Tapping different areas of the screen trigger different ShapeTones. The game starts with 3 ShapeTones. As the game evolves, more ShapeTones become available. When a new ShapeTone is added, a trial screen is shown to demonstrate where each ShapeTone is triggered. Some surprises happen along the way. As a one player game, the sequence is created automatically. As a two player game, one player creates the sequence, passes the device to the other player, who tries to repeat it. They then swap the roles. 
Type Of Technology Software 
Year Produced 2015 
Impact Not currently known. 
URL http://depic.eecs.qmul.ac.uk/?q=shapetones
 
Company Name Augmented Instruments Ltd 
Description On 8 September 2016, a spinout company Augmented Instruments Ltd was formed to lead the commercialisation of the Bela embedded audio platform. Bela was created within the Centre for Digital Music as an open-source hardware and software tool for creating musical instruments and interactive audio systems, targeted at the maker community. The spinout company now handles sales and support to the general public, and has received a £30k grant from Innovate UK to further develop the Bela technology. 
Year Established 2016 
Impact The spinout company now handles sales and support to the general public, and has received a £30k grant from Innovate UK to further develop the Bela technology.
 
Description "Steve Reich's Clapping Music" (Marcus Pearce and Sam Duffy) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact "Steve Reich's Clapping Music" was a project funded by the Digital R and D fund for the Arts to examine whether a game-based app could stimulate engagement with a new music genre (minimalism) and improve ensemble rhythmic performance. During the project an adult focus group of 100 people was recruited. 59 participants completed the study, submitting all three surveys over a 6-week period whilst also playing the game and exploring the app content. In addition, over 3,000 short surveys were collected from players around the world. 35GB of Gameplay data was collected from over 45,000 users of the app.

An important part of the continuation and development of this project is outreach. The team are still in the early stages of this work but have already contacted a number of schools, with some positive response, to present a workshop on the app that they have developed "Steve Reich's Clapping Music". The workshop will aim to engage young students (11-16) in a musical genre they may not yet have experienced (Minimalism) which we hope they will continue to explore through the app. They also plan to run sessions on the process of developing the app at events such as 'Get into Tech' which aim to encourage young adults to consider a career in technology. The app was developed through an interdisciplinary project team of researchers, technologists and musicians, to transform a piece of music into a game to encourage learning and skills acquisition. The research team hopes to stimulate thinking on how to transform an idea into a compelling game.

The team are working with The London Sinfonietta, an orchestra specialising in contemporary music, with a strong relationship with Steve Reich, to develop the workshops that will be offered.
Year(s) Of Engagement Activity 2015
 
Description 115th MPEG Meeting, Geneva (CH), 30 May - 3 Jun 2016 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The work by C4DM on MVCO AMD 1 (Time Segments & Multi-track Audio) reached PDAM 1 stage (June 2016) and a ballot was issued. It is envisaged (depending on the ballot results) to be published as an ISO/IEC standard in June 2017. A meeting report is available at: 115th MPEG Meeting, Geneva (CH), 30 May - 3 Jun 2016.
Year(s) Of Engagement Activity 2016
URL https://code.soundsoftware.ac.uk/attachments/download/2142/1PanosGeneva115MPEGMeetingReport.pdf
 
Description 116th MPEG Meeting, Chengdu (CN), 17 - 21 Oct 2016 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact C4DM's work on MVCO AMD 1 (Time Segments & Multi-track Audio) promoted to DAM 1 stage (Oct. 2016) with 'no comments'. It will be published as an ISO/IEC standard in June 2017. A meeting report is available at: 116th MPEG Meeting, Chengdu (CN), 17 - 21 Oct 2016.
Year(s) Of Engagement Activity 2016
URL https://code.soundsoftware.ac.uk/attachments/download/2163/1PanosChengdu116MPEGMeetingReport.pdf
 
Description ACM Conference on Human Factors in Computing Systems 2016 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The C4DM team (M. Barthet et al.) presented our work on the 'Participatory Music Performance in a Digital Age: Open Symphony' project. We organised interactive music performances during the Interactivity track of the ACM CHI 2016 conference, engaging both local performers and international audiences. Approximately 150 participants took part in the interactive live music performances and evaluation. Overall feedback was very positive with a wide majority of participants expressing interest in taking part again. It enabled M Barthet et al. to conduct a second evaluation of the system which was essential to be able to publish their work in a high impact factor journal.
Year(s) Of Engagement Activity 2014,2016
 
Description Andrew Macpherson's D-Box presentation, Humanising Music Technology workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact The Humanising Music Technology workshop was held at Kingston University, Monday 16 November 2015. The event focused on the role of technology in our engagement with music and the intricate relationship between technology and creativity. The event covered a broad range of approaches ranging from art music to the fan-based Chiptune phenomena. As part of the event, Andrew Mcpherson and Alan Chamberlain (from Nottingham and working on the FAST project, also funded by EPSRC) presented musical hacking. After the presentation, participants had the opportunity to make music using a 'hackable' electronic instrument.
Year(s) Of Engagement Activity 2015
URL http://www.kingston.ac.uk/events/item/1856/16-nov-2015-humanising-music-technology/
 
Description BeagleRT workshop C4DM 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact We organised a workshop for PhD students in the Centre for Digital Music and those on the Media & Arts Technology programme introducing the new BeagleRT real-time embedded audio environment. It showed how the environment could be used for the students' research and introduced basic concepts of real-time audio programming.

Many participants expressed interest in using the platform in their own work.
Year(s) Of Engagement Activity 2014
 
Description BeagleRT workshop Imperial 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Other academic audiences (collaborators, peers etc.)
Results and Impact Organised audio programming workshop at Imperial College Advanced Hackspace, to happen 14 November.

n/a (upcoming)
Year(s) Of Engagement Activity 2014
 
Description Ben Bengler's presentation at the ACM Creativity and Cognition conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Ben Bengler is currently funded by Platform Grant for additional post-phd research to maximise impact of his research including paper writing (intended to be interim bridging funding). In this time Ben has written a paper that was presented at ACM Creativity and Cognition conference (Glasgow, June 2015) which is the "premier forum for presenting the world's best new research investigating computing's impact on human creativity in a broad range of disciplines including the arts, design, science, and engineering". This was the presentation of a study of why people get bored of interactive exhibits, even though they report that they find them interesting.

Outcome: Identification of key features of public interactive pieces which retain an audience's interest, or not.
Year(s) Of Engagement Activity 2015
 
Description C4DM Concert Series 2015-2016 (organisers: Elaine Chew & Andrew Mcpherson) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Public/other audiences
Results and Impact Designed to challenge the ways we think about and make music, the concert series of the Centre for Digital Music at Queen Mary University of London presents cross-cutting performances drawing upon ongoing research in the themes of music cognition, creativity, and expression, new interfaces and augmented instruments, music informatics, and audio engineering at C4DM. This inaugural series was made possible, in part, by generous support from the centre's EPSRC Platform Grant for Digital Music.

The Classical music stream, directed by pianist and operations researcher Professor Elaine Chew, offers boundary-crossing concerts juxtaposing classical music with C4DM technologies and other art forms. The New Interfaces stream, directed by composer-violist and new instruments designer Dr Andrew McPherson showcases augmented instruments and interfaces developed at C4DM in concert. The Fixed Media stream, directed by computer music composer and electrical and computer engineer Dr Bob Sturm highlights electro-acoustic compositions enhanced by digital music technologies.

C4DM already has a world-leading reputation in research. Through the concerts, which are free for the public to attend, C4DM research and researchers have benefited from public exposure and being able to test and validate their technology "in the wild". The concert series demonstrates the potential for C4DM to be a driver for musical innovation in London and to have a more prominent profile in the larger arts community. Working with external musicians to present these concerts presents scope for future collaborations.

These are the concerts that have taken place so far:

1) Art Inside Music (curated by Paul Edlin, QMUL music director), Thursday 22 October 2015, full performance: 40 mins (part of Some Things Rich and Strange, an Inside Out Festival and Music@QMUL event)
Arno Babajanian's epic Piano Trio in F# minor performed by Elaine Chew (piano), Hilary Sturt (violin), and Ian Pressland (cello), with music-driven spectral art created by Alessia Milo. Babajanian's dark, ethereal, and wild trio is illuminated by specially designed vectorial drawings created in Processing, playing with time, light, darkness, colours, frequencies, and their spectral energy. The spectral art was generated in real-time in response to the music and projected on the musicians.

2) New Interfaces Concert (curated by Andrew McPherson and Astrid Bin), Wednesday 4 November 2015, full performance: 120 mins
Laurel Pardue performed on an augmented violin with live visuals. Dianne Verdonk performs on "La Diantenne", a new metal-sheet instrument capturing rich gestural interactions. Tim Exile performed on his "Flow Machine", a hardware/software setup for live sampling and transformation of sound. Bogdan Vera and friends performed on the D-Box. The performance was coupled with a study of audience response conducted by C4DM/MAT PhD student Astrid Bin. The audience was introduced to the instruments before the performance and record their responses during the performance using a web-based system developed by Astrid.

3) Computer Music! by Stephen Travis Pope and other deep neural networks (curated by Bob Sturm), Monday 23 November 2015, full performance: 90 mins
Featuring music by Stephen Travis Pope, Oded Ben-Tal, Dan Stowell, and Bob L. Sturm

4) Musical Banditry (curated by Elaine Chew), Monday 21 December 2015, full performance: 60 mins
As Stravinsky famously said, "Lesser artists borrow, great artists steal." The art of stealing was illustrated by Bob Sturm's Concatenative Variations of a Passage by Mahler and by Paul Lansky's Etudes and Parodies (winner of the 2005 International Horn Society competition) for horn trio as performed by Joel Ashford, Hilary Sturt, and Elaine Chew. Part of Lansky's piece was compared to the Beatles' While My Guitar Gently Weeps, as performed by Florian Guillaume, Giulio Moro, Keunwoo Choi, and Elio Quenton, culminating in a concurrent performance of the two. Musical similarities were made explicit using C4DM research in music informatics.

5) Chinese New Year Concert (curated by Luwei Yang), Monday 1 February 2016, full performance: 120 mins
The Chinese New Year Concert was organised jointly by the Chinese Students' Association (CSSA) and Confucius Institute at Queen Mary and not only celebrates Chinese New Year, but celebrated the fusion of arts and science at Queen Mary, combining music and performance with digital instruments. The event featured award-winning Chinese instrumentalists, C4DM members and the Magnetic Resonator Piano and the Mood Rater music technology.
Year(s) Of Engagement Activity 2006,2015
URL https://sites.google.com/site/c4dmconcerts1516/
 
Description C4DM at Sonar MarketLab 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact C4DM presented a public exhibition of its research at the Sonar+D 2016 festival in Barcelona, 16-18 June 2016. Sonar+D is a high-profile annual exhibition of music technology attended by thousands of industry professionals, musicians and members of the general public. C4DM was chosen by competitive application for a booth on the exhibition floor. We showed 9 recent research projects supported by EPSRC funding (EP/K009559/1, EP/K032046/1, EP/G03723X/1). The booth attracted significant interest from a broad cross-section of the Sonar audience: it was nearly full for most of the three days, and anecdotally appeared to have more traffic than most of the booths at the exhibition. In parallel with this in-person exhibition, Sonar+D featured C4DM in their social media feeds, and we were interviewed for a broadcast on Spanish television. Aside from public engagement, the exhibition led to several new industry contacts, and it provided valuable feedback to the attending PhD students and postdocs on how to showcase their research in public.

The projects shown were:

1) Bela, an open-source platform for ultra-low-latency audio and sensor processing, which launched on Kickstarter in 2016;
2) TouchKeys, a transformation of the piano-style keyboard into an expressive multi-touch control surface, launched on Kickstarter in 2013 and spun out into a company in 2016;
3) Collidoscope, a collaborative audio-visual musical instrument which was a viral hit online with over 10M views;
4) RTSFX (Real-Time Sound Effects), an online library of real-time synthesised (rather than sampled) sound effects for a variety of different objects and environmental sounds;
5) Moodplay, a mobile phone-based system that allows users to collectively control music and lighting effects to express desired emotions;
6) Augmented Violin, a sensor-based extension of the traditional violin to give students constructive feedback on their playing
7) Tape.pm, an interactive object which explores novel ways to record and share an improvisation that is created on a musical instrument;
8) Aural Character of Places, an interactive online demo of soundwalks conducted around London;
9) MixRights, a demo of how content reuse is enabled by emerging MPEG standards, such as IM AF format for interactive music apps and MVCO ontology for IP rights tracking, driving a shift of power in the music value chain.

Several potential collaborations may result from this visit, including commercial opportunities related to the TouchKeys and Bela projects, and possible joint projects with other university labs who attended (e.g. IRCAM and the Music Technology Group at Universitat Pompeu Fabra in Barcelona). The publicity surrounding this exhibition may also serve as a recruitment tool for future research students in the Music and Acoustic Technology research area.

A particularly valuable outcome, especially for the PhD student attendees, was experience in showcasing their research to the public. Several students commented that they received valuable feedback on their projects and learned general skills in public engagement over the course of the three days. In one case (Tape.pm), interactions with the public helped generate a dataset which will be analysed in future research.

Finally, attendees at Sonar+D also had the opportunity to see other booths and talks at the event, generating new ideas and connections for future projects.
Year(s) Of Engagement Activity 2016
URL https://sonar.es/en/2016/
 
Description C4DM visit to Universitat Pompeu Fabra, Barcelona 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact On 24 March 2016 a group of students and academics from C4DM visited the Universitat Pompeu Fabra in Barcelona to take part in the annual Doctoral Student Workshop held by the Department of Information and Communication Technologies, UPF. Three (3) academics and ten (10) students attended the workshop.

The following posters were presented from C4DM:
* Doing it Wrong: Towards an understanding of the roles of error, risk and fiasco in DMI design for performance by Astrid Bin
* Measuring Affective, Physiological and Behavioural Differences in Solo, Competitive and Collaborative Games by Daniel Gabana Arellano
* Intelligent Machine Music Listening: An Experimental Approach by Francisco Rodríguez-Algarra
* Automatic Machine Transcription of Wildlife Bird Sound Scenes by Gnostothea-Veroniki Morfi
* Automatic Expressive Feature Analysis by Luwei Yang
* Automatic detection of outliers in world music collections by Maria Panteli
* Physically-Musically Inspired Probabilistic Models for Audio Content Analysis by Pablo A. Alvarado
* Building self-contained digital musical instruments with active tactile feedback using Bela embedded audio platform by Robert Jack
* Robust and Efficient Joint Alignment of Multiple
* Musical Performances by Siying Wang

After the morning poster session, the group got a chance to see the latest research of the Music Technology Group including designs of new musical instruments, analysis techniques for non-Western music, and computer-assisted monitoring of a musician's movements during performance.

A group of three (3) students and two (2) academics also had the chance to visit Hangar (https://hangar.org/en/), an arts centre in Barcelona. Marta Garcia, the Director of the Research Programmes, showed the group around the making facilities, studios, video editing suite, and audio synth studio.

The most significant impact was potential research collaborations formed between UPF and QM students who met each other at the workshop. At least one student from QM is considering taking a research visit to UPF. There were also discussions of ongoing exchanges such as inviting UPF students to take part in future EECS research showcases.
Year(s) Of Engagement Activity 2006,2016
URL http://portal.upf.edu/web/etic_doctoral_workshop/home
 
Description Centre for Digital Music promotional film 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact This film was made for promoting C4DM activities to a general audience. Despite being a world-leading research group, C4DM did not have a media influence at the image of its academic and scientific excellence. The production of an inspiring promotional film was a first contribution towards a better outreach effort from C4DM. More specifically, it is formulated in accessible plain English and is displayed on the home page of C4DM website in order to generate public awareness, communicate research outcomes, disseminate knowledge and encourage public engagement and dialogue. Given its accessible character, it is also a valuable asset to reach potential partners outside of our academic niche as well as prospective students and researchers.

The production of this films grants C4DM the ability to engage with a non-expert audience in a systematic way. An interesting by-product of this is that it also rose awareness about public engagement paradigms within C4DM researchers.

Note: At the time of reporting, the film is still in the process of being preparted for publication for a variety of legal and administrative reasons; these have been cleared and it will be posted very soon on Youtube.
Year(s) Of Engagement Activity 2017
 
Description D-Box performance at the launch event of the Being Human Festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact To be confirmed by Andrew.
Year(s) Of Engagement Activity 2015
URL http://beinghumanfestival.org/event/being-human-festival-launch-day/
 
Description D-Box presentation FabLab Cardiff 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Postgraduate students
Results and Impact Presented the D-Box hackable instrument at FabLab Cardiff (Cardiff School of Art and Design), with an audience including arts students and members of the "maker" community. Followed by questions, discussions and hands-on demos.

Introduced our research to a broader maker community outside science and engineering, and may lead to further collaborative projects e.g. a placement project for a PhD student.
Year(s) Of Engagement Activity 2014
 
Description Exhibition of Collidoscope at the Ars Electronica Festival 2016, Liz, Austria 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Ars Electronica, founded in 1979, is one of the world's foremost media art festivals and attracts 158,000+ visitors a year, hosted by Ars Electronica Centre - the Museum of the Future. Exhibiting at such a high profile international venue was one of the key objectives for the presentation of Collidoscope. Collidoscope was shown at Ars Electronica as part of the Ars Electronica u19 Festival 2016. Exhibitions were open from 10am to 7pm on 8 September 2016 to 12 September 2016 (5 full days). Ars Electronica requested "hands-on sessions" and that the participants would "experiment their musical creations". The target audience was under 19s, so the sessions were highly interactive, and required substantial advanced planning (it was not meant to be a demo or showcases).

One of the outcomes of this activity is a future ongoing collaboration with Ars Electronica.

More about Collidoscope can be seen here: http://collidoscope.io/
Year(s) Of Engagement Activity 2016
URL http://www.aec.at/radicalatoms/en/
 
Description HORSE2016 workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact HORSE 2016 was a one-day workshop/debate devoted to "horses and Potemkin villages in applications of machine learning" that took place on the 19 September 2016. 60 people attended the event. The key finding of HORSE2016 is that many people are interested in the topics it covered. The organiser (Bob Sturm, Centre for Digital Music, Queem Mary) posted a blogpost featuring the videos of the event on Sep 25 (https://highnoongmt.wordpress.com/2016/09/25/horse2016-success/), and it received 426 views from around the world, and was featured on reddit. The next day (yesterday) it received 258 views. All videos on the HORSE2016 YouTube Channel (https://www.youtube.com/channel/UCKazfHUF5QmT55EKT5Wem1Q) have had 1131 views so far (Feb 15 2017).
Year(s) Of Engagement Activity 2016
URL https://highnoongmt.wordpress.com/2016/09/25/horse2016-success/
 
Description Hunan University Summer Camp Public Engagement 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Ben Bengler and Nick Bryan-Kinns from C4DM, Queen Mary, took part in the two week Hunan University Summer interdisciplinary project on Social Innovation in which they led a team of QM and Chinese students to develop interactive audio and visual pieces with local Dong minority musicians for public performance in rural China. It was a highly adventurous piece of research work - creative interactive objects in a remote rural location in China with minority cultural groups. It was quite a significant piece of public engagement and interdisciplinary co-creation with rural Chinese participants. The key findings of the research were the development of methods to engage rural populations in co-creation.

The Platform Grant funded Ben Bengler for 3 months to produce high quality video and website to promote the work of the Summer Camp and to catalogue and collate the video and images collected during the Summer Camp to help with writing of research papers. Ben Bengler also contributed to the writing of a full paper for SIG-CHI 2016 conference (the leading conference in the field), and to our submission to the QM Public Engagement Award 2015.
Year(s) Of Engagement Activity 2015
URL http://eecs.qmul.ac.uk/~nickbk/NewChannel2015/
 
Description Interactive Workshop at QMUL's A Mind of Music festival: Open Symphony (M. Barthet, C4DM) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact Western performing arts practices have traditionally restricted creative interactions from audiences. Open Symphony was designed to explore audience-performer interaction in live music performance assisted by digital technology. The Open Symphony system transforms the traditional unidirectional musical chain by adding creative communication in the reverse direction, from the audience to performers.

Approximately 80 local East London school students attended this creative workshop at Queen Mary University of London during which they learned about music making and conducted performers using the Open Symphony mobile app. Students actively engaged in the activity and their tutors reported it had fostered a lot of interest and discussions in class after the event.

The most significant outcome from this activity is to have sparked the interest of young people in contemporary music helping them to understand music better and bridge the gap between them and performers.
Year(s) Of Engagement Activity 2016
 
Description Listening in the Wild 2015 (organiser: Dan Stowell) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact The Listening in the Wild: Animal and machine audition in multi source environments research workshop took place at Queen Mary, London, 28th August 2015. The workshop aimed to address the following questions: a) How do animals recognise sounds in noisy multisource environments?, b) How should machines recognise sounds in noisy multisource environments?

The workshop brought together researchers in engineering disciplines (machine listening, signal processing, computer science) and biological disciplines (bioacoustics, ecology, perception and cognition), to discuss complementary perspectives on making sense of natural and everyday sound.
Year(s) Of Engagement Activity 2015
URL http://www.eecs.qmul.ac.uk/events/view/listening-in-the-wild-2015
 
Description Live Science Museum Residency (Marcus Pearce): Why do we like the music we like? 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact The project is ongoing, so no significant impact or outcome has been generated from this activity yet.

The project employed Léna Delval as a research assistant on a fixed term contract for 4 months. Léna is now applying for PhD positions. The project consisted of a Live Science Residency in the Science Museum - Pearce and Delval spent 5 weeks in the Museum running experiments on musical preference with visitors to the gallery. The purpose was twofold: first, to facilitate data collection and second to engage the public with our research. Between 11/11/15 and 11/12/15, the Live Science Residency attracted 371 visitors ranging in age from 8 to 78, who often came as families specifically to take part in our research and find out more about musical preferences.

Pearce and Delval intend to publish their data which includes a large and well-validated public dataset of emotion and preference ratings for music in a range of different genres. This may be used to support future research in music emotion recognition and modelling of musical emotions and preference decisions. They also expect this work to lead to one or more journal articles. Work on these will commence once the analysis of the data has been completed.
Year(s) Of Engagement Activity 2015
 
Description Mailing of the SC4FN magazine to Primary schools 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact This "Computing Sounds Wild" issue contained articles contributed by various C4DM people (Paul Curzon, Dan Stowell, Jane Waite and Peter W McOwan, Jo Brodie) which promoted the "machine listening" research topics studied in C4DM - how to use computational thinking to understand natural sound and related topics. Articles included:

* How machines recognise birdsong
* What makes music sound evil?
* A visual way to explore soundscapes
* Delia Derbyshire
* Strange sounds from Apollo 11
* How far can you hear?
* Controlling music with gesture

Further mailing of magazines was planned, but was stopped as the funds allocated from the grant were used in full.
Year(s) Of Engagement Activity 2016
URL http://www.cs4fn.org/magazine/magazine21.php
 
Description Mathematics and Computation in Music 2015 (organiser: Elaine Chew) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The Fifth Biennial International Conference on Mathematics and Computation in Music (MCM2015) was held 22-25 June-2015 at Queen Mary University of London (QMUL) in the United Kingdom. MCM is the flagship conference of the Society for Mathematics and Computation in Music (SMCM), whose official publication is the Journal of Mathematics and Music (JMM). The conference brought together researchers from around the world who combine mathematics or computation with music theory, music analysis, composition and performance. MCM 2015 provided a dedicated platform for the communication and exchange of ideas amongst researchers in mathematics, informatics, music theory, composition, musicology, and related disciplines. The 2015 event was co-hosted by the Schools of Electronic Engineering and Computer Science (Centre for Digital Music) and the School of Mathematical Sciences, and was promoted by QMUL in association with the Society for Mathematics and Computation in Music, London Mathematical Society, Institute of Musical Research, and the EPSRC.

Videos of the keynotes can be found under the KEYNOTES tab and through bit.ly/mcm2015-keynotes.
Year(s) Of Engagement Activity 2015
URL http://mcm2015.qmul.ac.uk/
 
Description Music Cognition Seminar (Aalborg, Denmark) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Dr K. Agres (Centre for Digital Music, QMUL) delivered a lecture to Aalborg University music researchers in November 2016. This was a lecture for PhD and postdoc students, most of whom did not have a background in music psychology. The lecture sparked lively debate, questions and discussion of a multi-disciplinary nature, and helped inspire a collaborative research project. The most significant outcome / impact was the comparison of behavioural with computational results, and the manuscript in progress describing this work.
Year(s) Of Engagement Activity 2016
 
Description Princeton University Seminar by the Centre for DIgital Music 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact This was a seminar at the Department of Mathematics, Princeton University, by PhD student Francisco Rodríguez Algarra (supervisor: Bob Sturm), Centre for Digital Music.Among the attendees to the seminar at Princeton, there was great interest in the call for employing proper experimental procedure, and not blindly trusting standard benchmarks. Though its specific examples are in MIR, the Centre made efforts to connect with general machine learning methods. Before the presentation, a private conversation was held with the host, Dr. Joakim Anden. He was very happy to pursue building upon our work to improve his methodology.

The Centre have sowed the seeds for a future collaboration with Dr. Andén, specifically exploiting our analysis to improve the scattering-based music content analysis systems he developed. The interest show by parties not currently involved in MIR shows great promise of future collaboration.
Year(s) Of Engagement Activity 2016
 
Description Research visit, New York University (NYU) by Maria Pantelli, Centre for DIgital Music 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact During her research visit at the NYU, the PhD student Maria Pantelli from the Centre for Digital Music was invited to give an interview for the "I care if you listen" music blog (https://www.icareifyoulisten.com/2016/07/monthly-music-hackaton-science-music/) and give a talk at the corresponding social event held at Spotify New York in conjunction with the Monthly Music Hackathon: Science of Music. The audience of this blog and hackathon involves mostly people from the music and tech industries and some educational institutions including universities and high schools.

Maria also collaborated with faculty and students in the Music Research Lab at NYU, something that is reported seperately under the Collaborations/Partnerships.
Year(s) Of Engagement Activity 2016
URL https://www.icareifyoulisten.com/2016/07/monthly-music-hackaton-science-music/
 
Description Second workshop on Intelligent Music Production (WIMP2) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Audio Engineering and Music Production are inherently technical disciplines, often involving extensive training and the investment of time. The processes involved implicitly require knowledge of signal processing and audio analysis, and can present barriers to musicians and non-specialists. The emerging ?eld of Intelligent Music Production addresses these issues via the development of systems that map complex processes to intuitive interfaces and automate elements of the processing chain.The first workshop on Intelligent Music Production was hosted by Birmingham City University last year, with support from QMUL's Centre for Digital Music. There were no plans or strategy for continued activity. With the support of the Platform Grant, the organisers hosted and organized the second workshop, greatly expanded the activities, and laid the groundwork for future events, funding, dissemination and collaborative activities.

The 2nd Workshop was hosted by the Centre for Digital Musi, Queen Mary University of London. It took place on the 13 September 2016, It had over a hundred participants, including some from Japan, USA, Norway, Germany and France. It included people from academia, industry and some from the wider public. It was promoted widely. It had a twitter feed (still active) and videos from the event have been uploaded to two YouTube channels. The event provided an overview of some of the tools and techniques currently being developed in the ?eld. It also delivered insight for audio engineers, producers and musicians looking to gain access to new technologies. The day consisted of presentations from leading academics, industry keynotes, and additional posters and demonstrations.

Activities in the workshop included further promotion, development and update of our Open Multitrack Testbed. Some talks in the workshoo, both by others and by C4DM people, made use of this testbed.

There were 23 publications, 5 of which were from C4DM;
• Sean Enderby, Thomas Wilmering, Ryan Stables and György Fazekas, 'A semantic architecture for knowledge representation in the digital audio workstation,' 2nd AES Workshop on Intelligent Music Production, London, UK, 13 September 2016.
• B. De Man, N. Jillings, D. Moffat, J. D. Reiss and R. Stables, "Subjective comparison of music production practices using the Web Audio Evaluation Tool," 2nd AES Workshop on Intelligent Music Production, London, UK, 13 September 2016.
• B. De Man, J. D. Reiss, "The Open Multitrack Testbed: Features, content and use cases," 2nd AES Workshop on Intelligent Music Production, London, UK, 13 September 2016.
• D. Ward, J. D. Reiss, "Loudness algorithms for automatic mixing," 2nd AES Workshop on Intelligent Music Production, London, UK, 13 September 2016.
• Adib Mehrabi, Simon Dixon and Mark Sandler, 'Towards a comprehensive dataset of vocal imitations of drum sounds,' 2nd AES Workshop on Intelligent Music Production, London, UK, 13 September 2016.

Within the 23 publications, there were over 40 citations of previous C4DM work.
One workshop presentation has already led to 1 further C4DM publication;
• Jillings, Nicholas and Wang, Yonghao and Reiss, Joshua D. and Stables, Ryan, 'JSAP: A Plugin Standard for the Web Audio API with Intelligent Functionality,', Audio Engineering Society Convention 141, Sept. 2016

Several publications by others included demonstrations of creative projects. Notably,
• Francois Pachet of Sony unveiled his 'Flow machines,' which does automatic music composition. Soon after the Workshop, Sony released a press release about the work, which has now been covered widely in the international press and on social media.
• Owen Campbell demonstrated the ADEPT framework for adaptive audio effects
• Iver Jordal demonstrated artificial Neural Networks for Cross-Adaptive Audio Effects
• Gerard Roma showcased new source separation and upmixing technologies

Overall, the 2nd workshop was a success. Since our last Reserchfish report in March 2017, the workshop has evolved into a recurring annual event. The third workshop was Sept. 15 2017 at Salford. It included two FAST-related publications (Brecht De Man, Joshua D. Reiss and Ryan Stables, 'Ten Years Of Automatic Mixing', and Marco A. Martínez Ramírez and Joshua D. Reiss, 'Deep Learning And Intelligent Audio Mixing'). FAST Investigator George Fazekas was also an invited workshop panellist. Both Brecht and Josh were on the organising committee. The fourth workshop is planned for Sept. 2018 at Huddersfield.
Year(s) Of Engagement Activity 2014,2016,2017
URL http://www.aes-uk.org/forthcoming-meetings/wimp2/
 
Description Trip to Shenzen, Hong Kong and Brisbane 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Schools
Results and Impact Bela was presented at the Baoan Maker Alliance and at the Shenzhen DIY Night in Shenzhen. A pop-up Bela workshop was held at the New Interfaces for Musical Expression conference in Brisbane. While not originally in the schedule, the Centre for Digital Music received a last-minute invitation to host the workshop given the interest of many of the participants of the conference. During the workshop, the 20 participants were introduced the basic usage of the platform and their specific questions about Bela as a teaching tool were answered.

Bela, an embedded platform for audio and sensor processing, was developed at C4DM. During this trip Giulio Moro from the Centre for Digital Music got in touch with manufacturers, to plan for future orders and with the community of users, notably gathering interest for it being used as a teaching tools in universities around the world. Meeting manufacturers SeeedStudio and Elecrow in Shenzhen allowed to plan for
the future of the hardware manufacturing of Bela. Useful feedback was gathered from the participants of the workshop in order to improve the ongoing hardware and software development of Bela, which is carried on at C4DM.

The Centre is now close to creating an Augmented Instruments Lab spin-off company with the support of QM Innovation, with the main aim of commercializing Bela and support other projects from the lab.

The Centre for Digital Music gathered interest from teachers at University of Virginia (Travis Thatcher, Peter Bussigel), Griffith University of Brisbane (John Ferguson, Andrew Brown, Toby Gifford), University of West of England (Chris Nash), Staffordshire University (Si Waite) who are interested in the use of Bela as a teaching tool for their classes in sound design and digital musical instruments.
Year(s) Of Engagement Activity 2016
 
Description Workshop on Auditory Neuroscience, Cognition and Modelling 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact The main aim of the organisers (Marcus Pearce, Emmanouil Benetos, Yvonne Blockland) was to organise and host at QMUL a workshop on Auditory Neuroscience and Computation, which took place on Wednesday 17 February 2016, at the Charterhouse Square campus. The workshop brought together auditory neuroscientists, cognitive scientists, as well as researches in music and audio signal processing and related fields. New insights on the cognitive and neural underpinnings of speech, music and sound processing were presented, with a large focus on EEG and MEG data analysis. The workshop focused on academic impact, however it had industry participation (from Audio Analytic, Red Apple Creative). Following an open call for abstracts and selection process., 32 presentations were accepted: 3 keynote talks, 6 oral presentations and 23 poster presentations.

The workshop website includes a PDF book of abstracts and video recorded oral presentations (see website below). Post-workshop summaries are being written by both workshop organizers and attendees, to be published to several venues: Psychomusicology, Young Acousticians Network newsletter, Audio Analytics Lab blog. Future plans: discussions are underway on applying for an EPSRC Research Network grant, for funding future workshops and network activities.

Post-workshop summaries are being written by both workshop organizers and attendees, to be published to several venues: Psychomusicology, Young Acousticians Network newsletter, Audio Analytics Lab blog.

- Psychomusicology report:
o Workshop on Auditory Neuroscience, Cognition and Modeling. Agres, Kat; Sauvé, Sarah Anne Psychomusicology: Music, Mind, and Brain, Vol 26(3), Sep 2016, 288-292. http://dx.doi.org/10.1037/pmu0000151
- Audio Analytic Lab blog post:
http://www.audioanalytic.com/aa-labs-impressions-on-the-workshop-on-auditory-neuroscience-cognition-and-modelling-2016/
- Young Acousticians Network newsletter:
https://euracoustics.org/activities/yan/yan-newsletter-folder/2016/march-2016/at_download/file

Preliminary discussions have been made with Iris Mencke and Elvira Brattico on hosting the next workshop.
Year(s) Of Engagement Activity 2015
URL http://c4dm.eecs.qmul.ac.uk/wancm2016/
 
Description Yang L.'s talk to musicians at the Turkish Music Conservatory 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The talk "Cross-cultural analysis of vibrato: A case study comparing erhu and violin playing styles" was delivered as part of the CompMusic Seminar at the Istanbul Technical University's Turkish Music Conservatory. The seminar focused on Culture specific approaches in music technology.

Practicing musicians and music educators were exposed to technological and scientific methodologies in music research.
Year(s) Of Engagement Activity 2014
URL http://compmusic.upf.edu/tr/node/219