The Commercialisation of Interactive Music

Lead Research Organisation: University of West London
Department Name: London College of Music

Abstract

In 2015, the AHRC funded a research project called Transforming Digital Music under its Digital Transformations call. This project developed a radical new music-release format - variPlay, which packaged an album as mobile iPhone application (an app) with real-time interactive playback capabilities. This innovative technology allows users to manipulate and interact with music in ways that had not previously been possible. In addition, the format packaged rich media such as lyrics, photographs, production credits and song/artist information.

The app offers a number of different modes of interactivity. For example, variPlay allows the listener to decide on and manipulate the genre of the song being played, in a smooth and creative manner. A song consisting of standard pop-rock instrumentation (drums, guitar, synth, strings and vocals) might be manipulated in real-time to instead play as electronic dubstep, a capella, folk-acoustic, or a blend of styles. Equally the listener can manipulate instruments, removing the vocals, isolating the drums, or creating their own preferred mix of the recording. In addition, variPlay can make its own intelligent decisions about how to vary playback, taking the listener on a unique journey through a given song on each play. The listener can experience the music based on their mood or preference, whilst also exploring a deeper creative vision of the artist than can be achieved with conventional playback platforms, which in turn brings a new and innovative commercial proposition.

The Follow-on-Funding project investigators are Prof. Justin Paterson of the University of West London, and Prof. Rob Toulson of the University of Westminster. Both are experienced researchers, music producers and software developers. They have now teamed up with major record label, Warner Music Group (WMG) to launch the app with three high-profile artists from its roster, and WMG will use its global leverage to promote the format. WMG plan to release a special edition variPlay app featuring the works of renowned classical composer Claude Debussy during the delivery of this project, with other high-profile artists from other genres to be confirmed in due course. This new project will therefore maximise the impact of the previous research and reach huge worldwide audiences, bringing them the ability to shape the sound of their favourite artists, whilst simultaneously offering the artists a new mode of deeper engagement with their fans.

The technology also provides the record company with data on how users are responding to, and using the variPlay format. If for instance the bulk of listeners appear to prefer acoustic versions of a given song, future artist development and marketing can react to this in a way never previously possible, thus unlocking career development opportunities for the artist and future monetisation.

The original research concluded with a pilot app-release by the artist Daisy and The Dark; the Red Planet EP. This follow-on project involves taking the technological 'shell' of that app and repopulating it with the music, images, and text of the new WMG artists. The three releases are targeted to different sections of the consumer demographic in order to explore which audiences best respond to such a format, and indeed the associated pricing. This approach will inform future releases.

In addition, the funding is supporting launch events that will draw in the media, and help to publicise the new releases in order to further increase public awareness, and hence impact. The project will culminate in the creation of a sustainable business model for variPlay, attracting further commercial funding and later forming a spin-out company to precipitate further releases. This project will therefore define a transformational digital business model that might reinvigorate the global music industry.

Download the pilot Red Planet EP app for iPhone and iPad here: http://bit.ly/RedPlanetApp

Planned Impact

This project has been carefully designed to maximize impact in a number of strategic pathways. These are to: 1] demonstrate a new delivery format to the global music industry; 2] offer artists and associated stakeholders new modes of expression; 3] open new artist development opportunities to Warner Music Group (WMG); 4] enhance the reputation of the academic project partners; 5] bring a radical new form of 'creative' listening to the general public.

1] The global music industry is desperate to find a more profitable distribution model than the streaming services that currently dominate. So much popular music is now freely available that many sections of consumer society no longer consider paying for it. The variPlay platform offers a unique listening experience unattainable from streaming, radio or any other playback medium. As such, it offers the industry a chance to open a new mode of monetisation based upon a genuine USP. This will cascade back to artists and copyright holders, restoring a degree of their income that is for so many, decimated by the meagre margins offered by streaming platforms.
Whilst WMG are anxious to be early adopters of this initiative and will be the primary beneficiaries, the greater industry will eagerly watch, and when this project goes to its final phase of generating self-sustainability, others will also seek to exploit its commercial potential. The true creative potential of digital music playback technologies has yet to be discovered, offering profound and transformational industry impact.

2] Musicians, songwriters, music producers, mastering engineers and publishers will benefit directly and significantly. Musicians and songwriters will be empowered to realise their wider artistic aspirations by gaining more flexibility for their creative vision, thus engaging with their audience at a deeper level. In order to populate variPlay, music producers will be given the opportunity to implement multiple creative ideas, many of which they are currently forced to jettison in the name of studio editing. Mastering engineers will see new opportunities for preparing audio for interactive songs. Music publishers will benefit from increased revenue through extra sales induced by this technology.

3] variPlay reports listener behaviour back to the developers in far more detail than current streaming platforms. This data then allows labels, artists and their management to better understand how their fan base best enjoys listening to them, and will help to tailor future releases accordingly. It will also inform touring plans - guiding where, when and what to play, thus benefiting all attendant stakeholders.

4] The investigators, their universities, and the AHRC will benefit from this commercialisation, which will bring international recognition and reputation and qualify the research agendum. The investigators themselves will gain traction and see new career opportunities emerge.

5] Lastly, huge numbers of listeners will benefit from this project as WMG uses its global reach to disseminate the original work. variPlay will enrich and extend both the artistic and creative nature of music playback, and hence bring an improved user experience and enhanced listening pleasure. Music consumers generally experience static playback, whereby a track will proceed through its duration in exactly the same way - every time. DJ culture subverts this; the popularity of such a dynamic listening experience has long been evidenced, and remixing is now a long established meme. This project brings a similar excitement to the solo listener, allowing them to transform their listening experience with - for the first time - their favourite music in whatever genre, on demand and in real time, keeping tunes fresh and invigorated much as a live performance by a band might offer. It could be viewed as an enhanced 'quality of life' with respect to listeners' engagement with recorded music.

The potential is enormous.

Publications

10 25 50
 
Title Asympt Man - Chasing Infinity 
Description This is an interactive music app for the UK artist Asympt Man, part of the variPlay family of releases, supported by this AHRC award. A unique automated interface also allows you to step back from the app and listen to the song as it takes an algorithmic journey through different remixes, while instrument stems allow components to be muted, soloed or rebalanced for a personalised experience - so it is possible to solo the drums or remove the vocals and sing along to an instrumental version. The app also includes rich media content, incorporating animations, artwork, lyrics, production credits and social media links, allowing the listener to explore the artist's creative world while listening along to their music. 
Type Of Art Artefact (including digital) 
Year Produced 2018 
Impact This app was the first variPlay release to offer in-app purchases. It achieved numerous downloads in the UK. 
URL https://apps.apple.com/gb/app/asympt-man-chasing-infinity/id1396470545
 
Title Ofenbach variPlay 
Description Ofenbach a French band with 700 million YouTube views. This is an interactive music app for the band, part of the variPlay family of releases with Warner Music Group, supported by this AHRC award. A unique automated interface also allows you to step back from the app and listen to the song as it takes an algorithmic journey through different remixes, while instrument stems allow components to be muted, soloed or rebalanced for a personalised experience - so it is possible to solo the drums or remove the vocals and sing along to an instrumental version. The app also includes rich media content, incorporating animations, artwork, lyrics, production credits and social media links, allowing the listener to explore the artist's creative world while listening along to their music. 
Type Of Art Artefact (including digital) 
Year Produced 2019 
Impact This app was downloaded about 1000 times in France, UK, Russia, Italy and the United States and was number-one music app on the French App Store. The track had more than 20 million plays on Spotify and 8.5 million plays on YouTube. 
URL https://itunes.apple.com/us/app/ofenbach-variplay-rock-it/id1457847537?mt=8
 
Title Plan (something good) variPlay 
Description This is an interactive music app for the band DeFab1 ft. Vanessa Knight, part of the variPlay family of releases supported by this AHRC award. A unique automated interface also allows you to step back from the app and listen to the song as it takes an algorithmic journey through different remixes, while instrument stems allow components to be muted, soloed or rebalanced for a personalised experience - so it is possible to solo the drums or remove the vocals and sing along to an instrumental version. The app also includes rich media content, incorporating animations, artwork, lyrics, production credits and social media links, allowing the listener to explore the artist's creative world while listening along to their music. 
Type Of Art Artefact (including digital) 
Year Produced 2018 
Impact This app was featured as Record of the Day by the British Photographic Institute (BPI) and presented at the BPI Innovation Hub. 
URL https://itunes.apple.com/gb/app/plan-defab1-vanessa-knight/id1413512922?mt=8
 
Title Quanta/Signal/Noise - Langham Research Centre 
Description Interactive iOS app 
Type Of Art Artefact (including digital) 
Year Produced 2020 
Impact Worldwide radio airing, e.g. Resonance FM, Framework Radio, Staalplaat Radio Show, The Institute of Spectra-Sonic Sound. 
URL https://apps.apple.com/gb/app/quanta-signal-noise/id1542394327
 
Title Ximena Sariñana variPlay 
Description Ximena Sariñana is a Grammy-nominated Mexican artist with nearly 1 billion YouTube views. This is an interactive music app for her, part of the variPlay family of releases with Warner Music Group, supported by this AHRC award. A unique automated interface also allows you to step back from the app and listen to the song as it takes an algorithmic journey through different remixes, while instrument stems allow components to be muted, soloed or rebalanced for a personalised experience - so it is possible to solo the drums or remove the vocals and sing along to an instrumental version. The app also includes rich media content, incorporating animations, artwork, lyrics, production credits and social media links, allowing the listener to explore the artist's creative world while listening along to their music. 
Type Of Art Artefact (including digital) 
Year Produced 2018 
Impact This app was downloaded about 1500 times in UK, Mexico, United States, Bolivia, Colombia, Paraguay, Africa, India, and the Middle East. The track had more than 28 million plays on Spotify and 9.5 million plays on YouTube. 
URL https://itunes.apple.com/us/app/ximena-sari%C3%B1ana-variplay/id1435482151?ls=1&mt=8
 
Description Warner Music Group have released two apps in the 'variPlay' format. The first was in Mexico and several other Spanish-speaking countries: Paterson, J. L., Toulson, E. R. and Sariñana, X. (2018) Ximena Sariñana variPlay. Apple App Store. Available at: https://itunes.apple.com/us/app/ximena-sari%C3%B1ana-variplay/id1435482151?ls=1&mt=8 (Accessed: 28 September 2018) There is a further French release: Paterson, J. L., Toulson, E. R. and Ofenbach (2019) Ofenbach variPlay. Apple App Store. Available at: https://itunes.apple.com/us/app/ofenbach-variplay-rock-it/id1457847537?mt=8 (Accessed: 27 April 2019) These apps had several thousand downloads around the world. We also created two independant releases in the UK: Paterson, J. L., Toulson, E. R., et al. (2018) Plan (something good) variPlay. Apple App Store. Available at: https://itunes.apple.com/gb/app/plan-defab1-vanessa-knight/id1413512922?mt=8 (Accessed: 27 September 2018) Asympt Man (2018) Chasing Infinity. Available at: https://apps.apple.com/gb/app/asympt-man-chasing-infinity/id1396470545 (Accessed: 4 July 2019) Collectively, the implementation of new technologies via these apps have exercised both the music industry and also the consumer, exploring acceptance and appetite. Langham Research Centre (2020) Quanta / Signal / Noise. Apple App Store. Available at: https://apps.apple.com/us/app/quanta-signal-noise/id1542394327 (Accessed: 21 December 2020). UPDATE 2023: Rock It has now had 39m plays on Spotify and 12m on YouTube, and Que Tiene 45m Spotify and 12m YouTube.
Sector Creative Economy,Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections
Impact Types Cultural

 
Description HAPPIE- Haptic Authoring Pipeline for the Production of Immersive Experiences
Amount £782,921 (GBP)
Funding ID 105159 
Organisation Innovate UK 
Sector Public
Country United Kingdom
Start 05/2019 
End 12/2020
 
Description Warner Music Group 
Organisation Warner Music Group
Country United States 
Sector Private 
PI Contribution The research team developed interactive music apps for artists from the Warner Music Group roster.
Collaborator Contribution Warner Music Group provided extensive in-kind support for the project. This support included provision of music stems, consultancy, marketing, legal and operational support.
Impact This collaboration was multidisciplinary in that it involved industry expertise from Warner Music Group, plus technical and artistic expertise from the research team.
Start Year 2017
 
Title CROSS-FADING BETWEEN AUDIO FILES 
Description A method is provided for forming cross-fades between audio files, which includes providing a plurality of audio files, each audio file comprising a plurality of respective audio streams pertaining to a single song, and providing a library of different fades, wherein each fade defines a respective characteristic shape of change in playback level over time to fade-in or fade-out an audio stream. The method further includes associating a respective fade, selected from the library of fades, with each audio stream, such that a cross-fade from a first one of the audio files to a second one of the audio files can be formed by applying the respective fades of the audio streams of the first audio file to fade-out the first audio file, and simultaneously applying the respective fades of the audio streams of the second audio file to fade-in the second audio file. These fades might be placed intelligently through real-time analysis of the plurality of audio streams and matching of musical features such as harmony. The method can enhance the ability of a content producer to generate musical content that can be changed as desired by a listener, by means of the cross-fade, to reflect, e.g. different preferences, moods or environments. 
IP Reference WO2017068032 
Protection Patent application published
Year Protection Granted 2017
Licensed No
Impact None.
 
Title variPlay 
Description The original AHRC funding for 'Transforming Digital Music: Investigating Interactive Playback' developed an iOS app system called variPlay which was piloted there. The app facilitated interactive and algorithmic music playback; see the summary page. This project took that code framework and deployed it as a number of commercial releases; see Artistic and Creative Products section. 
Type Of Technology Webtool/Application 
Year Produced 2018 
Impact See the Artistic and Creative Products section. 
 
Description Radio and media exposure 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Paterson discussed variPlay in a June 2019 BBC World Service / Open University programme 'The Future of Music Technology' hosted by Pink Floyd's Nick Mason (BBC estimate: 10 million listeners worldwide).

Langham Research Centre:

https://auralaggravation.com/2020/12/21/langham-research-centre-quanta-signal-noise/

Review of app and EP by Christopher Nosnibor

https://silenceandsound.me/2020/12/15/langham-research-centre/

Review of app and EP by Roland Torres

www.mixcloud.com/spectrasonicsound/the-institute-of-spectra-sonic-sound-121220/

https://kepw.org

Radio broadcast on Spectra-Sonic Sound show, KEPW 97.3 FM Eugene, Oregon, 12.12.2020
https://www.mixcloud.com/Resonance/langham-research-centre-14-december-2020/

https://www.resonancefm.com/

Radio broadcast on Resonance FM, app mentioned, 15.12.2020

https://vimeo.com/494129481

App promo film
https://www.mixcloud.com/kevin-press/the-moderns-ep-136/

https://themoderns.blog/2020/12/13/the-moderns-ep-136/

http://www.radioregent.com/

Radio broadcast on The Moderns, Radio Regent, 13.12.2020
https://www.mixcloud.com/RadioEclectus/radio-eclectus-74/

Radio broadcast on KHUH Hollow Earth Radio, Seattle, 10.12.2020

https://frameworkradio.net/2020/12/737-2020-12-06/
Radio broadcast on Framework Radio, 6.12.2020

http://stream.dfm.nu:8205/dfm_1?fbclid=IwAR0muPNV_TBooT-ZT5Hhjfnl8LzeL9nwGKOnJjv71UrWIKHfQojG9TVSb78

http://dfm.nu
Radio broadcast on Staalplaat Radio Show, DFM, Amsterdam, 9.12.2020

https://www.mixcloud.com/spectrasonicsound/the-institute-of-spectra-sonic-sound-121220/

Radio broadcast on The Institute of Spectra-Sonic Sound, 12.12.2020
Year(s) Of Engagement Activity 2019,2020