The Commercialisation of Interactive Music
Lead Research Organisation:
University of West London
Department Name: London College of Music
Abstract
In 2015, the AHRC funded a research project called Transforming Digital Music under its Digital Transformations call. This project developed a radical new music-release format - variPlay, which packaged an album as mobile iPhone application (an app) with real-time interactive playback capabilities. This innovative technology allows users to manipulate and interact with music in ways that had not previously been possible. In addition, the format packaged rich media such as lyrics, photographs, production credits and song/artist information.
The app offers a number of different modes of interactivity. For example, variPlay allows the listener to decide on and manipulate the genre of the song being played, in a smooth and creative manner. A song consisting of standard pop-rock instrumentation (drums, guitar, synth, strings and vocals) might be manipulated in real-time to instead play as electronic dubstep, a capella, folk-acoustic, or a blend of styles. Equally the listener can manipulate instruments, removing the vocals, isolating the drums, or creating their own preferred mix of the recording. In addition, variPlay can make its own intelligent decisions about how to vary playback, taking the listener on a unique journey through a given song on each play. The listener can experience the music based on their mood or preference, whilst also exploring a deeper creative vision of the artist than can be achieved with conventional playback platforms, which in turn brings a new and innovative commercial proposition.
The Follow-on-Funding project investigators are Prof. Justin Paterson of the University of West London, and Prof. Rob Toulson of the University of Westminster. Both are experienced researchers, music producers and software developers. They have now teamed up with major record label, Warner Music Group (WMG) to launch the app with three high-profile artists from its roster, and WMG will use its global leverage to promote the format. WMG plan to release a special edition variPlay app featuring the works of renowned classical composer Claude Debussy during the delivery of this project, with other high-profile artists from other genres to be confirmed in due course. This new project will therefore maximise the impact of the previous research and reach huge worldwide audiences, bringing them the ability to shape the sound of their favourite artists, whilst simultaneously offering the artists a new mode of deeper engagement with their fans.
The technology also provides the record company with data on how users are responding to, and using the variPlay format. If for instance the bulk of listeners appear to prefer acoustic versions of a given song, future artist development and marketing can react to this in a way never previously possible, thus unlocking career development opportunities for the artist and future monetisation.
The original research concluded with a pilot app-release by the artist Daisy and The Dark; the Red Planet EP. This follow-on project involves taking the technological 'shell' of that app and repopulating it with the music, images, and text of the new WMG artists. The three releases are targeted to different sections of the consumer demographic in order to explore which audiences best respond to such a format, and indeed the associated pricing. This approach will inform future releases.
In addition, the funding is supporting launch events that will draw in the media, and help to publicise the new releases in order to further increase public awareness, and hence impact. The project will culminate in the creation of a sustainable business model for variPlay, attracting further commercial funding and later forming a spin-out company to precipitate further releases. This project will therefore define a transformational digital business model that might reinvigorate the global music industry.
Download the pilot Red Planet EP app for iPhone and iPad here: http://bit.ly/RedPlanetApp
The app offers a number of different modes of interactivity. For example, variPlay allows the listener to decide on and manipulate the genre of the song being played, in a smooth and creative manner. A song consisting of standard pop-rock instrumentation (drums, guitar, synth, strings and vocals) might be manipulated in real-time to instead play as electronic dubstep, a capella, folk-acoustic, or a blend of styles. Equally the listener can manipulate instruments, removing the vocals, isolating the drums, or creating their own preferred mix of the recording. In addition, variPlay can make its own intelligent decisions about how to vary playback, taking the listener on a unique journey through a given song on each play. The listener can experience the music based on their mood or preference, whilst also exploring a deeper creative vision of the artist than can be achieved with conventional playback platforms, which in turn brings a new and innovative commercial proposition.
The Follow-on-Funding project investigators are Prof. Justin Paterson of the University of West London, and Prof. Rob Toulson of the University of Westminster. Both are experienced researchers, music producers and software developers. They have now teamed up with major record label, Warner Music Group (WMG) to launch the app with three high-profile artists from its roster, and WMG will use its global leverage to promote the format. WMG plan to release a special edition variPlay app featuring the works of renowned classical composer Claude Debussy during the delivery of this project, with other high-profile artists from other genres to be confirmed in due course. This new project will therefore maximise the impact of the previous research and reach huge worldwide audiences, bringing them the ability to shape the sound of their favourite artists, whilst simultaneously offering the artists a new mode of deeper engagement with their fans.
The technology also provides the record company with data on how users are responding to, and using the variPlay format. If for instance the bulk of listeners appear to prefer acoustic versions of a given song, future artist development and marketing can react to this in a way never previously possible, thus unlocking career development opportunities for the artist and future monetisation.
The original research concluded with a pilot app-release by the artist Daisy and The Dark; the Red Planet EP. This follow-on project involves taking the technological 'shell' of that app and repopulating it with the music, images, and text of the new WMG artists. The three releases are targeted to different sections of the consumer demographic in order to explore which audiences best respond to such a format, and indeed the associated pricing. This approach will inform future releases.
In addition, the funding is supporting launch events that will draw in the media, and help to publicise the new releases in order to further increase public awareness, and hence impact. The project will culminate in the creation of a sustainable business model for variPlay, attracting further commercial funding and later forming a spin-out company to precipitate further releases. This project will therefore define a transformational digital business model that might reinvigorate the global music industry.
Download the pilot Red Planet EP app for iPhone and iPad here: http://bit.ly/RedPlanetApp
Planned Impact
This project has been carefully designed to maximize impact in a number of strategic pathways. These are to: 1] demonstrate a new delivery format to the global music industry; 2] offer artists and associated stakeholders new modes of expression; 3] open new artist development opportunities to Warner Music Group (WMG); 4] enhance the reputation of the academic project partners; 5] bring a radical new form of 'creative' listening to the general public.
1] The global music industry is desperate to find a more profitable distribution model than the streaming services that currently dominate. So much popular music is now freely available that many sections of consumer society no longer consider paying for it. The variPlay platform offers a unique listening experience unattainable from streaming, radio or any other playback medium. As such, it offers the industry a chance to open a new mode of monetisation based upon a genuine USP. This will cascade back to artists and copyright holders, restoring a degree of their income that is for so many, decimated by the meagre margins offered by streaming platforms.
Whilst WMG are anxious to be early adopters of this initiative and will be the primary beneficiaries, the greater industry will eagerly watch, and when this project goes to its final phase of generating self-sustainability, others will also seek to exploit its commercial potential. The true creative potential of digital music playback technologies has yet to be discovered, offering profound and transformational industry impact.
2] Musicians, songwriters, music producers, mastering engineers and publishers will benefit directly and significantly. Musicians and songwriters will be empowered to realise their wider artistic aspirations by gaining more flexibility for their creative vision, thus engaging with their audience at a deeper level. In order to populate variPlay, music producers will be given the opportunity to implement multiple creative ideas, many of which they are currently forced to jettison in the name of studio editing. Mastering engineers will see new opportunities for preparing audio for interactive songs. Music publishers will benefit from increased revenue through extra sales induced by this technology.
3] variPlay reports listener behaviour back to the developers in far more detail than current streaming platforms. This data then allows labels, artists and their management to better understand how their fan base best enjoys listening to them, and will help to tailor future releases accordingly. It will also inform touring plans - guiding where, when and what to play, thus benefiting all attendant stakeholders.
4] The investigators, their universities, and the AHRC will benefit from this commercialisation, which will bring international recognition and reputation and qualify the research agendum. The investigators themselves will gain traction and see new career opportunities emerge.
5] Lastly, huge numbers of listeners will benefit from this project as WMG uses its global reach to disseminate the original work. variPlay will enrich and extend both the artistic and creative nature of music playback, and hence bring an improved user experience and enhanced listening pleasure. Music consumers generally experience static playback, whereby a track will proceed through its duration in exactly the same way - every time. DJ culture subverts this; the popularity of such a dynamic listening experience has long been evidenced, and remixing is now a long established meme. This project brings a similar excitement to the solo listener, allowing them to transform their listening experience with - for the first time - their favourite music in whatever genre, on demand and in real time, keeping tunes fresh and invigorated much as a live performance by a band might offer. It could be viewed as an enhanced 'quality of life' with respect to listeners' engagement with recorded music.
The potential is enormous.
1] The global music industry is desperate to find a more profitable distribution model than the streaming services that currently dominate. So much popular music is now freely available that many sections of consumer society no longer consider paying for it. The variPlay platform offers a unique listening experience unattainable from streaming, radio or any other playback medium. As such, it offers the industry a chance to open a new mode of monetisation based upon a genuine USP. This will cascade back to artists and copyright holders, restoring a degree of their income that is for so many, decimated by the meagre margins offered by streaming platforms.
Whilst WMG are anxious to be early adopters of this initiative and will be the primary beneficiaries, the greater industry will eagerly watch, and when this project goes to its final phase of generating self-sustainability, others will also seek to exploit its commercial potential. The true creative potential of digital music playback technologies has yet to be discovered, offering profound and transformational industry impact.
2] Musicians, songwriters, music producers, mastering engineers and publishers will benefit directly and significantly. Musicians and songwriters will be empowered to realise their wider artistic aspirations by gaining more flexibility for their creative vision, thus engaging with their audience at a deeper level. In order to populate variPlay, music producers will be given the opportunity to implement multiple creative ideas, many of which they are currently forced to jettison in the name of studio editing. Mastering engineers will see new opportunities for preparing audio for interactive songs. Music publishers will benefit from increased revenue through extra sales induced by this technology.
3] variPlay reports listener behaviour back to the developers in far more detail than current streaming platforms. This data then allows labels, artists and their management to better understand how their fan base best enjoys listening to them, and will help to tailor future releases accordingly. It will also inform touring plans - guiding where, when and what to play, thus benefiting all attendant stakeholders.
4] The investigators, their universities, and the AHRC will benefit from this commercialisation, which will bring international recognition and reputation and qualify the research agendum. The investigators themselves will gain traction and see new career opportunities emerge.
5] Lastly, huge numbers of listeners will benefit from this project as WMG uses its global reach to disseminate the original work. variPlay will enrich and extend both the artistic and creative nature of music playback, and hence bring an improved user experience and enhanced listening pleasure. Music consumers generally experience static playback, whereby a track will proceed through its duration in exactly the same way - every time. DJ culture subverts this; the popularity of such a dynamic listening experience has long been evidenced, and remixing is now a long established meme. This project brings a similar excitement to the solo listener, allowing them to transform their listening experience with - for the first time - their favourite music in whatever genre, on demand and in real time, keeping tunes fresh and invigorated much as a live performance by a band might offer. It could be viewed as an enhanced 'quality of life' with respect to listeners' engagement with recorded music.
The potential is enormous.
Publications
J Paterson
(2019)
Reimagining and performing data
Paterson J
(2018)
Interactive Recorded Music: Past, Present, and Future
Title | Asympt Man - Chasing Infinity |
Description | This is an interactive music app for the UK artist Asympt Man, part of the variPlay family of releases, supported by this AHRC award. A unique automated interface also allows you to step back from the app and listen to the song as it takes an algorithmic journey through different remixes, while instrument stems allow components to be muted, soloed or rebalanced for a personalised experience - so it is possible to solo the drums or remove the vocals and sing along to an instrumental version. The app also includes rich media content, incorporating animations, artwork, lyrics, production credits and social media links, allowing the listener to explore the artist's creative world while listening along to their music. |
Type Of Art | Artefact (including digital) |
Year Produced | 2018 |
Impact | This app was the first variPlay release to offer in-app purchases. It achieved numerous downloads in the UK. |
URL | https://apps.apple.com/gb/app/asympt-man-chasing-infinity/id1396470545 |
Title | Ofenbach variPlay |
Description | Ofenbach a French band with 700 million YouTube views. This is an interactive music app for the band, part of the variPlay family of releases with Warner Music Group, supported by this AHRC award. A unique automated interface also allows you to step back from the app and listen to the song as it takes an algorithmic journey through different remixes, while instrument stems allow components to be muted, soloed or rebalanced for a personalised experience - so it is possible to solo the drums or remove the vocals and sing along to an instrumental version. The app also includes rich media content, incorporating animations, artwork, lyrics, production credits and social media links, allowing the listener to explore the artist's creative world while listening along to their music. |
Type Of Art | Artefact (including digital) |
Year Produced | 2019 |
Impact | This app was downloaded about 1000 times in France, UK, Russia, Italy and the United States and was number-one music app on the French App Store. The track had more than 20 million plays on Spotify and 8.5 million plays on YouTube. |
URL | https://itunes.apple.com/us/app/ofenbach-variplay-rock-it/id1457847537?mt=8 |
Title | Plan (something good) variPlay |
Description | This is an interactive music app for the band DeFab1 ft. Vanessa Knight, part of the variPlay family of releases supported by this AHRC award. A unique automated interface also allows you to step back from the app and listen to the song as it takes an algorithmic journey through different remixes, while instrument stems allow components to be muted, soloed or rebalanced for a personalised experience - so it is possible to solo the drums or remove the vocals and sing along to an instrumental version. The app also includes rich media content, incorporating animations, artwork, lyrics, production credits and social media links, allowing the listener to explore the artist's creative world while listening along to their music. |
Type Of Art | Artefact (including digital) |
Year Produced | 2018 |
Impact | This app was featured as Record of the Day by the British Photographic Institute (BPI) and presented at the BPI Innovation Hub. |
URL | https://itunes.apple.com/gb/app/plan-defab1-vanessa-knight/id1413512922?mt=8 |
Title | Quanta/Signal/Noise - Langham Research Centre |
Description | Interactive iOS app |
Type Of Art | Artefact (including digital) |
Year Produced | 2020 |
Impact | Worldwide radio airing, e.g. Resonance FM, Framework Radio, Staalplaat Radio Show, The Institute of Spectra-Sonic Sound. |
URL | https://apps.apple.com/gb/app/quanta-signal-noise/id1542394327 |
Title | Ximena Sariñana variPlay |
Description | Ximena Sariñana is a Grammy-nominated Mexican artist with nearly 1 billion YouTube views. This is an interactive music app for her, part of the variPlay family of releases with Warner Music Group, supported by this AHRC award. A unique automated interface also allows you to step back from the app and listen to the song as it takes an algorithmic journey through different remixes, while instrument stems allow components to be muted, soloed or rebalanced for a personalised experience - so it is possible to solo the drums or remove the vocals and sing along to an instrumental version. The app also includes rich media content, incorporating animations, artwork, lyrics, production credits and social media links, allowing the listener to explore the artist's creative world while listening along to their music. |
Type Of Art | Artefact (including digital) |
Year Produced | 2018 |
Impact | This app was downloaded about 1500 times in UK, Mexico, United States, Bolivia, Colombia, Paraguay, Africa, India, and the Middle East. The track had more than 28 million plays on Spotify and 9.5 million plays on YouTube. |
URL | https://itunes.apple.com/us/app/ximena-sari%C3%B1ana-variplay/id1435482151?ls=1&mt=8 |
Description | Warner Music Group have released two apps in the 'variPlay' format. The first was in Mexico and several other Spanish-speaking countries: Paterson, J. L., Toulson, E. R. and Sariñana, X. (2018) Ximena Sariñana variPlay. Apple App Store. Available at: https://itunes.apple.com/us/app/ximena-sari%C3%B1ana-variplay/id1435482151?ls=1&mt=8 (Accessed: 28 September 2018) There is a further French release: Paterson, J. L., Toulson, E. R. and Ofenbach (2019) Ofenbach variPlay. Apple App Store. Available at: https://itunes.apple.com/us/app/ofenbach-variplay-rock-it/id1457847537?mt=8 (Accessed: 27 April 2019) These apps had several thousand downloads around the world. We also created two independant releases in the UK: Paterson, J. L., Toulson, E. R., et al. (2018) Plan (something good) variPlay. Apple App Store. Available at: https://itunes.apple.com/gb/app/plan-defab1-vanessa-knight/id1413512922?mt=8 (Accessed: 27 September 2018) Asympt Man (2018) Chasing Infinity. Available at: https://apps.apple.com/gb/app/asympt-man-chasing-infinity/id1396470545 (Accessed: 4 July 2019) Collectively, the implementation of new technologies via these apps have exercised both the music industry and also the consumer, exploring acceptance and appetite. Langham Research Centre (2020) Quanta / Signal / Noise. Apple App Store. Available at: https://apps.apple.com/us/app/quanta-signal-noise/id1542394327 (Accessed: 21 December 2020). UPDATE 2023: Rock It has now had 39m plays on Spotify and 12m on YouTube, and Que Tiene 45m Spotify and 12m YouTube. |
Sector | Creative Economy,Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections |
Impact Types | Cultural |
Description | HAPPIE- Haptic Authoring Pipeline for the Production of Immersive Experiences |
Amount | £782,921 (GBP) |
Funding ID | 105159 |
Organisation | Innovate UK |
Sector | Public |
Country | United Kingdom |
Start | 05/2019 |
End | 12/2020 |
Description | Warner Music Group |
Organisation | Warner Music Group |
Country | United States |
Sector | Private |
PI Contribution | The research team developed interactive music apps for artists from the Warner Music Group roster. |
Collaborator Contribution | Warner Music Group provided extensive in-kind support for the project. This support included provision of music stems, consultancy, marketing, legal and operational support. |
Impact | This collaboration was multidisciplinary in that it involved industry expertise from Warner Music Group, plus technical and artistic expertise from the research team. |
Start Year | 2017 |
Title | CROSS-FADING BETWEEN AUDIO FILES |
Description | A method is provided for forming cross-fades between audio files, which includes providing a plurality of audio files, each audio file comprising a plurality of respective audio streams pertaining to a single song, and providing a library of different fades, wherein each fade defines a respective characteristic shape of change in playback level over time to fade-in or fade-out an audio stream. The method further includes associating a respective fade, selected from the library of fades, with each audio stream, such that a cross-fade from a first one of the audio files to a second one of the audio files can be formed by applying the respective fades of the audio streams of the first audio file to fade-out the first audio file, and simultaneously applying the respective fades of the audio streams of the second audio file to fade-in the second audio file. These fades might be placed intelligently through real-time analysis of the plurality of audio streams and matching of musical features such as harmony. The method can enhance the ability of a content producer to generate musical content that can be changed as desired by a listener, by means of the cross-fade, to reflect, e.g. different preferences, moods or environments. |
IP Reference | WO2017068032 |
Protection | Patent application published |
Year Protection Granted | 2017 |
Licensed | No |
Impact | None. |
Title | variPlay |
Description | The original AHRC funding for 'Transforming Digital Music: Investigating Interactive Playback' developed an iOS app system called variPlay which was piloted there. The app facilitated interactive and algorithmic music playback; see the summary page. This project took that code framework and deployed it as a number of commercial releases; see Artistic and Creative Products section. |
Type Of Technology | Webtool/Application |
Year Produced | 2018 |
Impact | See the Artistic and Creative Products section. |
Description | Radio and media exposure |
Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
Part Of Official Scheme? | No |
Geographic Reach | International |
Primary Audience | Public/other audiences |
Results and Impact | Paterson discussed variPlay in a June 2019 BBC World Service / Open University programme 'The Future of Music Technology' hosted by Pink Floyd's Nick Mason (BBC estimate: 10 million listeners worldwide). Langham Research Centre: https://auralaggravation.com/2020/12/21/langham-research-centre-quanta-signal-noise/ Review of app and EP by Christopher Nosnibor https://silenceandsound.me/2020/12/15/langham-research-centre/ Review of app and EP by Roland Torres www.mixcloud.com/spectrasonicsound/the-institute-of-spectra-sonic-sound-121220/ https://kepw.org Radio broadcast on Spectra-Sonic Sound show, KEPW 97.3 FM Eugene, Oregon, 12.12.2020 https://www.mixcloud.com/Resonance/langham-research-centre-14-december-2020/ https://www.resonancefm.com/ Radio broadcast on Resonance FM, app mentioned, 15.12.2020 https://vimeo.com/494129481 App promo film https://www.mixcloud.com/kevin-press/the-moderns-ep-136/ https://themoderns.blog/2020/12/13/the-moderns-ep-136/ http://www.radioregent.com/ Radio broadcast on The Moderns, Radio Regent, 13.12.2020 https://www.mixcloud.com/RadioEclectus/radio-eclectus-74/ Radio broadcast on KHUH Hollow Earth Radio, Seattle, 10.12.2020 https://frameworkradio.net/2020/12/737-2020-12-06/ Radio broadcast on Framework Radio, 6.12.2020 http://stream.dfm.nu:8205/dfm_1?fbclid=IwAR0muPNV_TBooT-ZT5Hhjfnl8LzeL9nwGKOnJjv71UrWIKHfQojG9TVSb78 http://dfm.nu Radio broadcast on Staalplaat Radio Show, DFM, Amsterdam, 9.12.2020 https://www.mixcloud.com/spectrasonicsound/the-institute-of-spectra-sonic-sound-121220/ Radio broadcast on The Institute of Spectra-Sonic Sound, 12.12.2020 |
Year(s) Of Engagement Activity | 2019,2020 |