Transforming Digital Music: Investigating Interactive Playback

The Transforming Digital Music research project is to develop a unique interactive music album format that could transform the way we listen to and engage with digital music. This innovative technology will allow users to manipulate and interact with music in ways that have not previously been possible, bringing new user experiences and opening a number of artistic and commercial opportunities for music artists and the entire music industry.

Nowadays, music is predominantly consumed through digital platforms; however, the true artistic power of the digital playback medium has never been fully evaluated or exploited. Modern digital playback systems, including mobile devices, computers, tablets and networked hi-fi systems, are extremely over-engineered for simply playing music and audio. By utilising the full power of the modern digital music playback device, we hypothesise that a new and heightened level of engagement can be achieved, releasing greater opportunities of creative expression for the artist and an enhanced user experience for the listener. Exploration of this creative and transformative opportunity is essential in order to consolidate models that could reinvigorate the global music industry, which has experienced considerable turbulence since the introduction of web-based digital music. In particular, we wish to focus our investigation on the opportunities for popular music to be dynamic and interactive, whilst retaining and extrapolating the artistic vision of the musician.

An example of interactive music is in the form of a song that can be listened to by the consumer in a number of different ways. The song might be an upbeat rock genre track including drums, guitars, vocals and maybe a string section. Consider, however, that the listener might want to enjoy to the song in a more relaxed manner - a unique experience based on the listener's mood at that moment. They might prefer to listen to a stripped down version, perhaps with the electric guitars replaced by acoustic versions, the drums become hand percussion and the strings take a more prominent role in the mix. It is still the same song, maintaining the artistry intended by the musician, but it has changed to give the listener a different experience. Similarly the listener might select an electronic version of the song, which dynamically introduces programmed drumbeats and synthesised sounds. Modern digital playback systems have the processing capability to perform the real-time rendering of multiple audio streams required to implement these interactive features.

Additionally, in bringing digital interactive music systems to a wide and far reaching audience, it is necessary to consider the tools required for musicians, music producers and importantly, record labels to develop their own interactive music. Our project therefore explores and develops an operational paradigm and simple content management system to allow them to build bespoke interactive output from their own familiar musical material. In order to exemplify this feature set, an app will be developed. This app will serve as a working model that incorporates all of the above features, and will serve as a profile from which any future commercial development might be derived. Its release (for free) will demonstrate feasibility, and harvest consumer feedback and opinion.

Our project is designed to amplify the NESTA-funded work of Dr Rob Toulson and Script Ltd, which focuses on identifying and evaluating new modes of engagement between popular artists and audiences. Our research objectives are focused on investigating two specific aspects of advanced digital music reproduction, i.e. what are the opportunities and capabilities of interactive music playback for 1)the artist (and record label) and 2)the music consumer or listener. By working in collaboration with a recognised music artist and music industry representatives, we will define and appraise this potential future of music release.

All listeners and consumers of commercially released music will benefit from this research. A main goal of this investigation is to generate outputs that will enhance the artistic and creative nature of audio reproduction systems and hence bring an improved user experience and listening pleasure. All music consumers will benefit from an enhanced 'quality of life' with respect to their engagement with recorded music.

Musicians, songwriters, music producers, mastering engineers and publishers will benefit directly and significantly from this research. In particular, we will investigate new methods for engaging and invigorating the music-buying public, so we are identifying new opportunities for creative artists to develop sustainable careers in the arts. Musicians and songwriters will be empowered to realise their wider artistic aspirations by gaining more flexibility for their creative vision. The technology we propose could act as a major driver in that regard, thus increasing revenue streams to the artists themselves. Music producers will be given the opportunity to implement multiple creative ideas, many of which they are currently forced to jettison in the name of studio editing. With our technology these ideas might be employed to produce different variants of musical material. Mastering engineers will see new opportunities for preparing audio files for interactive songs. Music publishers will benefit from increased revenue, simply through extra sales and synchronisation royalties induced by this technology.

The music industry is at a point of fluxive transition, a situation that has been on-going for a number of years with merciless effect. The industry is desperate for innovative ideas to propel it towards the new business models of the future. Currently ascendant technologies such as streaming provide miniscule revenues both for the industry and artist; the album has always been the primary financial driver in industry sales. Following the demise of the CD, no one knows how to sell albums any more, and new initiatives are sought. The technology we propose could provide one, or possibly even a major component in the rejuvenation of the album.
The true creative potential of digital music playback technologies have not to date been exploited. As a result there is a major opportunity to define and transform new technical approaches, that might have a major impact on creative technical industries such as application developers, digital artists, animators and graphic designers, who might all see growth in their commercial opportunities and demand. Thus, the music and arts industries as a whole will benefit from this research, which aims to justify and promote a higher public value to be associated with commercial music, and so invigorate and enhance the career opportunities for all those working within diverse areas of the industry.

The academic community interested in music production, audio quality, and user experience will see great impact. This study, as well as providing answers to the specific research questions detailed, will ask new questions that should be investigated by evolving and continuing studies. This project therefore acts as an opening of new and exciting strands of research that might continue to be considered by the collaborating institutions and/or by the wider international academic community. The individual universities and associated academics will benefit greatly by this research which will bring internationally disseminated outputs and enhanced recognition and reputation. The collaboration with key project partners will boost the universities' reputation and industrial links, and allow partnerships and collaborations to continue past the completion of this study. The partners themselves will gain momentum and greater justification for their own related work and will see new opportunities and greater audience awareness as a result of this project.

The potential is enormous.