Learn to Play: Computational Assessment of Musical Playability for Users' Practice

This Follow-On Funding for Impact and Engagement proposal is based on research from the AHRC Digital Transformations project, 'Transforming Musicology' (AH/L006820/1), and the Electronic Corpus of Lute Music project, most recently as 'Lute Music in the Open (ECOLM III)', AH/H037829/1. It explores the concept of 'playability' of music. By developing a system to assess the difficulty of a displayed piece, and then using this system to create on demand a set of practice exercises based on passages within the music judged to be tricky by the system, it will help students learning to play an instrument (flute, guitar, or renaissance lute).

The guitar is the most widespread instrument in the world today, and the internet provides a literally bewildering number of 'tabs' (scores notated in the format known as tablature) requiring no formal knowledge of music notation. Tablature provides instructions about the placement of fingers to form chords or melodies and the sequence in which they should be played. It is a system that has stood the test of time, and has been used for hundreds of years, at least since the 15th century, and is particularly useful for instrumental teaching, especially in the early stages.

There is a vast amount of music available online and the system we create will help musicians find music to suit their skill level. The system will analyse the playability of tablature versions of pieces of music for guitar (classical and other styles) and for renaissance lute (we already have a corpus of c10,000 pieces in ECOLM). Using measures based on hand-stretches and position-shifts indicated in the tablature we'll compute indexes of playability of individual chords and transitions between them.

The flute is another very popular instrument among self-learners and young people, especially in schools; based on figures from the Hackney Music Service, we estimate that over 3,000 non-beginner flute students take lessons in London schools alone. We'll build on earlier work carried out by co-I Fiebrink on the modelling of difficulty in flute music, a very useful starting point, since the simpler texture of the music allows us to focus on its melodic aspects rather than on chords (as on guitar or lute).

We'll then use standard machine-learning techniques to build models of playability to identify difficult passages in unknown flute, guitar and lute pieces. They will also be used to grade pieces (based on the difficulty of the most technically-challenging passages) and the results compared with the grades listed by music publishers in their catalogues. The proof-of-concept demonstrator forming the main output of the project will then use simple algorithms to generate entirely new exercises derived from these passages for practising by a student.

All the above will be evaluated by our user community - players at various levels and flute, guitar and lute teachers.

The music will be presented within a high-quality graphical user interface provided by our music-industry partner, Tido Music. Currently used for a number of educational packages, mostly aimed at amateur pianists, it will be adapted to communicate remotely with the playability estimation and exercise generation back-end developed and maintained by Goldsmiths. This way our models can be tested from the outset with a professional user-interface, and use musical scores from the Tido music library (access restrictions entirely under Tido's control), or from elsewhere, without compromising rights ownership.

The lessons learned will be applied directly in two ways. We shall hold a workshop for professional and amateur musicians, including those involved as beta-testers, to discuss their assessment of the system with its designers and developers. This feedback will then be used as material for a full proposal to Innovate UK for funds to carry out further research and development to take this work beyond proof of concept to a commercially viable product.

The guitar is the most popular instrument in the world today, and hundreds of thousands of players at every level learn and exchange music online. Millions of pieces can be found notated in tablature, an ideal system for learners (e.g., a single site, www.ultimate-guitar.com, has tens of thousands of regular users and offers 800,000 songs in tab), but finding those within a player's capability is a difficult and tedious task. This project aims to make that easier and to provide automatically-generated exercises for tricky passages in chosen pieces.

In musicological research carried out within the Transforming Musicology project and elsewhere, we have found that an important feature of historical repertories not hitherto studied extensively is the 'playability' of instrumental pieces (especially in arrangements), and we developed proof-of-concept measures to locate hard passages within pieces for renaissance lute notated in tablature. These can be applied to the vast modern repertory of music for guitar in tablature found on the internet. Co-I Fiebrink has also developed a method for grading pieces for flute (in normal staff-notation) according to their technical difficulty.

This proposal builds on that work in a system to help self-learning musicians, and their teachers, find music to suit their level of attainment from online resources, and - a feature unprecedented as far as we are aware - which generates a set of original practice exercises based on hard passages identified by the system.

Generating exercises from difficult passages of music will have a definite medium-term impact on the teaching and learning of the flute and the guitar (our principal focuses). Practising short exercises is far more enjoyable than struggling with complete pieces. Automated generation of original exercises based on a piece being studied (as are often written out by a teacher) provides a new and enjoyable experience for beginners and intermediate learners, and will help advanced students (up to conservatory level) make best use of their practice time.

Our music industry partner, Tido Music, have expressed a need to respond to a demand from their users for automated playability measurements of this sort. With them we shall be able to trial the method at an early stage of development, providing user feedback into the design process. We'll develop a demonstrator to show the potential for this approach which will be submitted to Tido's team of beta-testers. This is a basis for rapid development of a working system that will be offered as a service in conjunction with their educational materials. The 'grading' methods we use will enhance the experience of searching for music suitable for their level of ability for online customers of companies distributing such music. Once the proof-of-concept demonstrator produced here is developed to industry standards, and its capabilities extended to other instruments (especially the piano) it is likely to have a wide uptake among instrumental teachers as well as self-learning online customers.

We'll publish academic papers in the normal way throughout the project to ensure immediate academic impact for the project, further promoted by the use of posters and demos at music/computing, music education and musicology conferences. Our project workshop will report on activities, inviting interested communities and stakeholders to try the system and hear about our vision of how it offers a transformative enhancement for instrumental teaching yet preserves traditional standards and values.

This project seizes an opportunity presented by the application of a novel musicological concept developed in T-Mus (that of 'playability' as a determinant of historical repertoire) in a public-facing role with potential as a commercial venture. It thus further contributes to the stated belief of the T-Mus project that "Musicology need not - and should not - be an 'ivory-tower' discipline."