UKRI Centre for Doctoral Training in Artificial Intelligence and Music

Lead Research Organisation: Queen Mary, University of London
Department Name: Sch of Electronic Eng & Computer Science

Abstract

The UKRI Centre for Doctoral Training in Artificial Intelligence and Music (AIM) will train a new generation of researchers who combine state-of-the-art ability in artificial intelligence (AI), machine learning and signal processing with cross-disciplinary sensibility to deliver groundbreaking original research and impact within the UK Creative Industries (CI) and cultural sector.
The CI sector is a substantial part (5.3%) of the UK's economy and employs more than two million people; it is the fastest growing sector and contributes £92bn of GVA. Bazalgette's "Independent Review of the Creative Industries" highlights their "central importance to the UK's productivity and global success," and notes the particular importance of the music industry, where the UK is one of very few countries who are net exporters.
The need for increasing the scale of doctoral training in AI is clear. The UK government's Industrial Strategy White Paper (2017) identifies the development and maintenance of leading research in AI as the first of four Grand Challenges. To address this challenge, Hall and Pesenti's review recommends support for more PhD places in AI, as does the Royal Society, who noted a "critical need for increased training at PhD level", and a "substantial skills shortage" in Machine Learning (2017).
The core area of this CDT is Music Information Research (MIR, also known as Music Informatics), which involves the use of intelligent information processing methodologies to understand and model music, and to develop products and services for creation, distribution, interaction and experience of music and music-related information. The proposed research focus is structured along three themes identified as requiring intensive attention and integration:
1. Music understanding, encompassing machine listening, intelligent signal processing, and data- and knowledge-driven approaches to music content modelling and analysis;
2. Intelligent instruments and interfaces, encompassing embeded intelligence and intelligent sensing for music performance, production, listening and education, and applications of AI to human-computer interaction in creative contexts;
3. Computational creativity, encompassing generative music composition, automated accompaniment systems, and systems for expressive musical performance and assisted production.
Research in each area will be guided by and grounded in real application needs by a unique set of industrial and cultural stakeholders (see support letters), from big players in media entertainment to innovative SMEs and cultural institutions, encompassing a wide spectrum of the digital music world.
The CDT will take a cohort-based approach, drawing on a supervisory team of over 30 academics led by QMUL's Centre for Digital Music (C4DM), a world-leading research group in the area of music and audio technology with a strong track record in doctoral training. The entire training approach, from the strategic focus to the topics of individual PhD projects, will be guided by C4DM's network of industrial and cultural partners, ranging from large companies and high-profile arts venues to a vibrant network of SMEs including several successful QMUL spin-out companies. Each PhD student will undertake a personalised programme of research supported by specialist taught modules, industrial placements, skills training, and opportunities for co-creation with cultural partners.
The AIM CDT benefits from a substantial institutional investment in the EPSRC and AHRC CDT in Media and Arts Technology (2014-22), for which QMUL has already invested over £6M in specialist facilities, including A/V production studios, a performance laboratory with state-of-the-art motion capture equipment, and specialist maker spaces. AIM builds on QMUL's outstanding track record in this interdisciplinary area, while bringing a new focus to the opportunities and challenges of AI in the creative industries.

Planned Impact

The AI and Music CDT holds benefits to several groups beyond the academic community, both in the UK and internationally. These include:

Commercial Private Sector:
* Music technology companies working on: automatic music content analysis, through new technologies for music recommendation, playlisting, fingerprinting and transcription; music interfaces, through creation of smart interfaces which adapt into a user's existing workflows; music performance and education, through new tools for computer-aided composition, performance, and instrument tutoring; music production, through technologies for intelligent sound engineering and audio mixing.
* Musical instrument manufacturers, who will benefit from embedded intelligence technologies towards the creation of smart instruments, and on data-driven tools for musical instrument design and user evaluation.
* Audio archiving companies, through access to scalable data-driven algorithms and methods for music content organisation, indexing and retrieval.
* Computer games companies, through improved and user-responsive game music and audio, computational creativity technologies for generating procedural audio, and through new types of audio or music-based computer games.
* Broadcast media industries, through use of the latest music information research in the design of new technology-based programmes or in new audio production processes.
* Hearing aid and cochlear implant manufacturers, through access to new machine listening research applicable to improving the enjoyment of music.
* Composers and musicians who will benefit from tools for computer-aided music generation and composition, as well as tools enabling efficient annotation and scoring of their creative output.
* Concert venues and promoters, who will benefit from performances involving smart musical instruments, intelligent music interfaces and computer-aided music compositions.

Public and third sector:
* Libraries, museums and public archives hosting, storing and preserving recorded music collections, through new methods for automatic curation, indexing, retrieval and visualisation of their content.
* Educational institutions, including schools, universities and music conservatories, through new digital instruments to promote student interest in music learning and new software tools for music education, including ear training, performance tutoring and music analysis.
* Organisations promoting STEM (science, technology, engineering and maths) to students and underrepresented groups, by using music AI technologies and smart instruments to generate interest in STEM careers.
* Healthcare workers who work with music and audio, such as music therapists, through new music and audio visualisation and navigation tools.
* Arts performance organisations, including not-for-profit orchestras and ensembles, through innovative technology-driven methods for music performance.
* Standards organisations, through access to new research methods on which to base forthcoming audio and music encoding and transmission standards.

Wider Public:
* People interested in exploring music or other audio recordings at home, school, college or university, using the latest research, either for educational or general interest purposes.
* Audiences of creative output involving audio and music, through availability of new creative outputs, new digital instruments, or through technology facilitated by our audio and digital music research.
* Students in schools and universities, enriching their knowledge and educational experience through new digital instruments and software tools for music tutoring and understanding.
* Members and users of public libraries, museums and sound archives, benefiting from an enriched cultural experience through the use of new digital technologies for exploring music and audio collections.

Publications

10 25 50