Networking technology and the experience of ensemble music-making

Lead Research Organisation: University of Sussex
Department Name: Sch of Media, Film and Music

Abstract

Could networking technology transform ensemble music-making so that more people can benefit? Research shows that ensemble performance participation raises self-confidence in children and adults. However ensemble music-making grinds to a halt when non-music readers participate - with the result that they feel excluded. Can a system be developed which integrates them?

A vast library of ensemble music exists in infinite archive form: http://imslp.org/ (International Music Score Library Project - 295,000 scores at 21/11/14). This offers everyone the chance to play many culturally valued pieces of ensemble and orchestral music because parts are typically available alongside scores. However these files are often scans of 19th century and 20th century editions, and therefore static, and of variable quality. Could critical and curatorial approaches, including careful preparation of files in a more suitable format (e.g. MIDI) drive a networked music notation system suitable for a wide range of users? And thereby interface between the infinite library resource of the online music score library project, and the real-world possibility of ensemble performance for newcomers to participatory music, facilitated by networking technology?

Libraries such as imslp.org increasingly represent to the public the body of historical and contemporary notated music available to live music ensembles. New beneficiaries of ensemble music through active computer networking in live ensemble performance contexts could be many and multiple, ranging from school children to adult amateurs to people engaging in music therapeutic treatments or recovering from an experience of isolation. On our advisory group we will have representatives from Education (the head of Brighton & Hove Music Service), Health (the research lead from Intensive Care Medicine Research, Brighton and Hove), Cultural organisations (the head of the Brighton Science Festival) and Arts Policy (the learning producer from the Arts Council funded organisation Sound and Music). In addition we will have academic experts in public engagement and digital material culture.

In the first instance we will create a system which dynamically updates to indicate current position in the score. This could be conductor-led (with a 'smart baton' counting beats and so updating position in score) or computer controlled, in the case that a conductor was not available. The scores for orchestral members would be networked and either digitally presented and highlighted on a tablet, or paper scores augmented with digitally controlled LEDs. Once we have a system which can synchronise networked information, then we will also be interested in how this could support other forms of notation for ensemble performance.

During the pilot, we will consider questions about the legibility of tablet displays; the flexibility and future-proof status of the underpinning software; and possible approaches to music notation digital networking that do not involve screens. We will explore how effective technologies can be in bringing mixed ability musicians and complete beginners together.

This work may have exciting implications for non-classical ensembles where synchronised notations could be productive in group work and musical forms requiring coordination hand in hand with creative flexibility in tempos and/or transpositions (e.g. electric guitar ensembles, concert bands, laptop orchestras).

Through this practice project, using Arts and Humanities critical and collaborative approaches, the project team and the advisory group will meet regularly to debate, scope and prepare the ground for a larger project on the future relationship between networked ensemble performance and online infinite archives of notated music, such as imslp.org, by creating a prototype with school children as initial subjects and then publicly debating its potential wider applications with a range of interested parties.

Planned Impact

Through practice-led research, collaborative critical enquiry and innovation using networking technology this pilot will critically mediate the gap between fixed musical notation and the flexibility needed in real-life ensemble performance, especially in mixed ability settings. The model developed will add value and further creative possibilities to existing work (for example the BBC's 'Ten Pieces' project and associated website resources http://www.bbc.co.uk/programmes/p01vs08w). Through its work and discussions with its advisory group, the project will scope a bigger project on the question of public online archives of musicals scores and their relationship to future networked ensembles and orchestras - it thus has the longer-term potential to reach and influence programming policies as well as broad arts and arts education policies. We welcome the presence of the London Symphony Orchestra's Head of Orchestral Artistry on our advisory group.

Which groups will benefit from the pilot project?

- Artists interested in networking technology and its potential transformation of ensemble music of all kinds
- Bands interested in developing new ways of structuring and coordinating their compositions and performances
- Teachers and education professionals interested in using networking technology in order to deliver ensemble performance requirements of the national curriculum in more engaging and effective ways
- Health professionals and patients interested in innovative therapies involving ensemble performance and networking technology
- Music hubs interested in reaching out further to enable young people from disadvantaged backgrounds to experience participation in ensemble music, even though they may lack instrumental training
- Community ensembles and orchestras who bring together mixed ability groups in order to perform the music of today and historical ensemble and orchestral music
- Professional ensembles and orchestras interested in approaches to engaging new audiences through the direct experience of participation
- The general public through enhanced understanding of innovation through critical approaches to technology in relation to both traditional and contemporary musical practices

How will they benefit?

- Artists will be able to share in and understand the findings and innovations of the work through the public events and the website
- Bands (initially at Brighton Institute of Modern Music) will be offered the chance to experiment with the prototype developed after the test as part of the collaborative public demonstration; Mark Irwin, Assistant Principal, BIMM, has expressed willingness to be involved
- Teachers, education professionals and Music Hubs will be drawn in through (a) direct participation and (b) sharing of the project via networks of the Head of Brighton and Hove Music Service, who is on the advisory board
- Health professionals and patients will gain access to the project through the networks of the Head of Intensive Care Medicine Research who is on the advisory board
- Community ensembles will benefit through the participation on the Advisory Board of Dr Adam Swayne, director of Contemporary Music for All, Sussex, who will also be involved in public events post-testing period
- Professional orchestras will learn of this work and the possibilities of deployment via the presence of the Head of Orchestral Artistry from the London Symphony Orchestra on the advisory board
- The general public attending the Brighton Science Festival, will learn of this project through the Festival's networks and marketing, and also through Music Of Our Time's networks; there is a great deal of public and press interest in the nature and culture of live musical practices and so the public roundtable and demonstration on networking technology and the experience of ensemble performance will attract strong attention, which is why the Festival has already committed to programming it in February 2016.

Publications

10 25 50
 
Title Alice in Wonderland 
Description Written for The New Music Players and Orchestra of Sound and Light for concerts in London and Sussex, with funding support from The RVW Trust and Arts Council England. This composition was adapted via MusicXML format for use in an iPads ensemble notation system as part of the AHRC funded practice-led research project 'Networking Technology and the Experience of Ensemble Music-Making'. Alice in Wonderland is a score in three main sections, inspired by a short film of the same name from 1903 by Hepworth and Stow. The film stars May Clark as Alice. In 2010 the BFI restored and released the remaining fragments of the film. It lasts approximately 9'30". The film is fascinating because it is a very early attempt to transfer an extended children's novel to the screen. As is usual in the early days of cinema, it presents a series of vignettes or scenes from a well-known story. It is also very engaging because it uses special effects and trick photography to create the atmosphere of a dream world of fantasies and anxieties. I wanted to work with an early film that would capture the interest of non-specialist audiences including school children and to create a score that could enable them to participate musically. The core ensemble of flute, clarinet, piano, violin and cello establishes the pulse and the main pitch centres. Throughout the first section the pulse is clear and the pitch centre is C. In the second section, there is a more complex approach to rhythm, but throughout the piano and the cello provide the main pulse and rhythm, and the chords will always work with the note E. In the third section, the ideas of the first section are explored and developed. 
Type Of Art Composition/Score 
Year Produced 2015 
Impact Performed with a combined ensemble of classical musicians and a rock band using iPads technology: 22 January 2016 Brighton Institute of Modern Music, Brighton Performed with sixth form specialist music students using iPads technology: 22 January 2016 East Sussex Academy of Music, Lewes Performed for school children in these contexts: 5 February 2016 Southover Primary School, Lewes 5 February 2016 Ore Village Academy, Hastings Performed with experimental open score ensemble (non-professional musicians) using iPads technology: 21 February 2016 Brighton Science Festival, St Nicholas Church, Brighton Performed with mixed ability music students at Dorothy Stringer High School, Brighton using iPads technology: 17 June 2016 Attenborough Centre for Creative Arts, University of Sussex 
URL http://www.uymp.co.uk/composers/ed-hughes/works/alice-in-wonderland
 
Title Brighton Science Festival - NETEM with COMA - 21 February 2016 
Description A performance at the Brighton Science Festival produced by the NETEM team (Networking Technology and the Experience of Ensemble Music Making). The adapted score by Ed Hughes, 'Alice in Wonderland', was performed by COMA Sussex alongside ten members of the public who had limited or no previous experience of ensemble music-making. The members of the public performed handbells and chimes, colour-coded according to the 'Boomwhacker' educational colour coding system, and were cued by symbols with the same colours appearing in their wirelessly networked and synchronised iPad screens. 
Type Of Art Performance (Music, Dance, Drama, etc) 
Year Produced 2016 
Impact In a reflective post-performance discussion, which was documented and transcribed, all participants reflected on the change to their experience and effectiveness of the networking system in an ensemble setting. 
URL http://netem.org.uk/
 
Description Our project, Networking Technology and the Experience of Ensemble Music-Making, began with ethical approval for a six week observation period in November to December 2015 in which we noted the enjoyment and difficulties experienced by members of a school orchestra club aged 9 to 11 in a primary school in Sussex, UK, in 2015. The orchestra was conducted voluntarily by a non-specialist classroom music teacher with strong interests and capabilities in music. Rehearsals were characterised by frequent stops because some children got lost and needed help from their teacher who had to halt the musical performance. Broadly, those children able to keep their performing in time with others and not lose their place enjoyed the experience. However, in contrast to confident students who typically had more experience as well as instrumental lessons, less confident pupils became visibly upset when they lost their place in the music.

To address this, we devised software to display musical score and parts on networked tablets. The software comprises two linked apps (Conductor and Performer). Conductor runs on iPad only, Performer also runs on Android. The software is currently packaged with a selection of classical and contemporary music for beginner and mixed ability ensembles. The software works by running the whole score on the Conductor app, and visually synchronises the play position in all of the Performer apps which are wirelessly networked via bluetooth.

Colleagues from School of Psychology at The University of Sussex led a study on the project which indicated benefits to the pupils with the software that the research team developed. The study found that:
? enjoyment leads to motivation to engage (echoing previous research literature)
? enjoyment also may predict appreciation of technology
? the biggest effect was for those with less perceived ability
Indications were that greatest benefit applied to those already enjoying playing orchestra, but with less perceived ability. Indeed, the authors argued that it is possible that by 'reducing barriers to engagement, including keeping time and synchrony' (Hanrahan, Hughes, Banerjee, Kiefer, Eldridge, 2018, p. 28) technology helped to increase enjoyment and perceived ability in those pupils, leading to higher likelihood of persistence and longer engagement.


Overall the initial project, which led to case studies with community ensembles and members of the public, demonstrated:
? strong value for novice notation readers because the software offers a way in to (or a 'scaffold' for) a form of literacy that potentially gives access to a wider range of musical experiences than they would otherwise have access to?; further research could include focus on individual musical development and how the technology might facilitate practice/preparation behaviour in advance of agreed ensemble music-making sessions
? strong potential in creative use of the software to provide reduced notation, forming ensembles with different cultural backgrounds (e.g. rock/classical) through a shared interest in structured improvisation enabled by technology
? the software in its current state is less likely to be adopted by experienced staff notation readers: this could be addressed by further development and research in annotation (including networked annotation) and improved support for following human tempos and expression

The work and trials were carried on pre-pandemic. But during the pandemic it was reported by a professional publication (Music Teacher, October 2020, p. 11) that the App was suitable for socially distanced performance practice because the networking protocol can be sustained accurately over an area of at least 20m.
Exploitation Route We are very excited about the potential to build on this work. Having focused deliberately on a tightly-knit timetable and target group, we think that feedback points to very strong potential in terms of the benefits of our research to existing and future user groups.

Users reported significant benefits: players feel more confident and relaxed, and can undertake longer and more complex pieces. Teachers noted improved retention rates. In response to feedback from teachers, in 2018 it was adapted to support teaching of ensemble music at GCSE (set-works).

We shared the system with students with limited music literacy at Brighton Institute of Modern Music in a specially convened workshop. These students gave very positive feedback on how the system facilitated their performance both technically and expressively; the sense of security provided by the system interestingly led to higher opportunities for flexible and creative expression. Similarly we shared the results with COMA Sussex at the Brighton Science Festival and via numerous public events (please refer to web site). Here we had a range of feedback but striking comments from novice players included 'I enjoyed feeling part of something bigger' and 'it gave me a sense of belonging to something special'. We think we can build on this towards a larger scale, more sustained study, with a view to examining the contribution networking technology can make to community, health and well-being settings and/or the contribution digital notation could make to changing the experience of composing and performing in groups.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Healthcare,Leisure Activities, including Sports, Recreation and Tourism,Culture, Heritage, Museums and Collections,Retail

URL http://netem.org.uk/
 
Description The Syncphonia app gained attention when the research and first case study were profiled on ITV News Meridian in March 2016, prompting interest in social media amongst teachers, educators including the Association of British Orchestras and other potential users. [Malcolm Shaw, ITV News, 9/3/16 'A unique musical experiment', News Report ITV1 News Meridian]. Since the launch of the app at the British Science Festival in 2017 which produced publicity in the i newspaper (printed edition and online) (Tom Bawden, i News, 26/9/17 'Video: New app brings karaoke to the classroom' https://inews.co.uk/essentials/news/education/new-app-brings-karaoke-classroom/ [accessed 29 /9/17] ), and on ABC (Australian) network radio (Science Programme) ('Keeping the wheels on in orchestra rehearsal', review, Australian Broadcasting Corporation, 23/9/17, http://www.abc.net.au/radionational/programs/scienceshow/keeping-the-wheels-on-in-orchestra-rehearsal/8974978 [accessed 29 /9/17] ) we have seen global interest, noticeably in English speaking regions. (Our website and user guide are currently in English only). To date 5,826 impressions globally (to 5/9/2018); 163 downloads. The app store analytics gives the download break down as 17% UK. 14% USA. 11% Australia. 8% Canada. 4% Netherlands. 46% others. (app store analytics) Syncphonia's partnership manager for 2017-2018, Liz Webb, has initiated numerous partnerships with schools, mainly but not exclusively focused in the south-east. In addition the team demonstrated at trade shows in Kensington Olympia and Manchester. The partnership manager's direct contact with schools has led to trials in Southover School (Lewes), Dorothy Stringer (Brighton), BIMM, East Sussex Academy of Music (Lewes), Access to Music (Brighton), Bolton Music Service, Heathfield Community College (Sussex), St Richards Bexhill, Chyngton Primary, Wallands Primary (Lewes), St James's Primary (Cheltenham). The team worked with the Sussex Innovation Centre to project the brand through a new website: www.syncphonia.co.uk. Through the website the partnerships manager has received direct enquiries from Australia, Quebec and the USA (for example, Lance Hoffmann 'Seriously Fun Singers' Chevy Chase, Maryland Email 8 March 2018). These conversations with remote partners have pointed towards the app's potential to change the experience of ensemble music making through enhanced access to music reading amongst adult ensembles, including choirs, as well as amongst our main target market, school ensembles. Several successes occurred in secondary schools with teachers committed to music and to creative use of technology and therefore able to implement Syncphonia without assistance from the team. For example: Heathfield Community College reported that their ensemble music making changed in two key ways: the app enabled real beginners to join in and experience ensemble music in a setting where orchestra was facing recruitment challenges; and it lessened the sense of personal exposure and risk by enabling participants to discreetly choose their part: We had 30 ipads we set up ourselves. The kids really liked it. Normally there are never enough parts in orchestral sets and it's just so brilliant being able to select your own parts. It's particularly valuable because you used to have to be grade 3 to be in the orchestra, but numbers went down, so I relaxed that requirements and so now it's very mixed with real beginners. You can have beginners joining in - that's really good with Syncphonia. A nice feature is, with beginners they like being able to choose the part to play without others knowing that they are 'basic'. No technical problemns with the app store version - it all worked fine. Colour notes are beautiful. It's the future. (Johnny Marrows, Teacher of Music, Media & Digital Champion. Heathfield Community College. Interview with Liz Webb, 17 February 2018 at 1300) Hastings Academy reported that the app transformed the confidence and therefore the perceived sound of the ensemble (echoing early comments by child participants in the research case study): It was so much fun and I can't wait to use it for future events and projects, if we can sound this good on a first attempt I would be interested to see the results after a few rehearsals. Kym East, Head of Music, Hastings Academy. 28 March 2018. (comment on Twitter) Important conversations initiated by the partnerships manager have been taking place with influential leaders in Music Services across southern England. Music Services play a critical role in ensuring affordable instrumental lessons are offered in state schools across the UK. They also run ensembles outside school time. The partnerships manager initiated contacts with Brighton and Hove, East Sussex, West Sussex and Hertfordshire Music Services. West Sussex Music Service stated an interest in using Syncphonia to support ensemble music making interventions in classroom teaching. These conversations continue and point towards possibility of broader take up of the App through Music Services, if (a) the app can be consolidated in response to feedback and things we have learnt in 2017/18, and (b) the app can be adapted for classroom use through the addition of new repertoires, uploading function, and ideally a deal with a reputable music education publisher to radically expand content: Our teachers were very interested in this new technology and I believe a number have asked to be kept updated on its development and progress. It was a really informative CPD session - one that has got us all thinking about we might utilise it in schools. James Underwood, Chief Executive, West Sussex Music 16/5/2018 I'm progressing the idea of arranging a practical inset with ESM staff to demonstrate Syncphonia. Thanks for our discussions - I was very interested in the practical and confidence building aspects of this initiative. Malcolm Warnes, East Sussex Music Service. 14 June 2018 What a wonderful demo. It's definitely the future Katy Wilmshurst, Hertfordshire Music Service. 3 Sept 2018 Individual music writers and arrangers have contributed to Syncphonia's growing portfolio, and conversations have started with publishers of classical music, who recognise the value of Syncphonia in the rapidly emerging digital market for notated ensemble music. Director of New Music for Edition Peters Group, Katie Tearle MBE commented: "Syncphonia makes music making more accessible for marginalised kids. There's a huge value in terms of music making and communal music-making especially for whole-class learning. It gives students guidelines and a framework and stops them from getting lost and panicking." During the pandemic it was reported that the App's capacity to work over distances of up to 25m meant that it was suitable for socially distanced ensemble music making (Music Teacher magazine, Oct 2020, p. 11).
Sector Digital/Communication/Information Technologies (including Software),Education,Culture, Heritage, Museums and Collections
Impact Types Cultural,Societal

 
Description Cuckmere: portraits in music, sound and images
Amount £15,000 (GBP)
Funding ID GFTA-00034237 
Organisation Arts Council England 
Sector Public
Country United Kingdom
Start 03/2017 
End 10/2017
 
Description Grants for the Arts
Amount £14,975 (GBP)
Funding ID 30215433 
Organisation Arts Council England 
Sector Public
Country United Kingdom
Start 09/2015 
End 06/2016
 
Description University of Sussex Enterprise Funding
Amount £31,634 (GBP)
Organisation University of Sussex 
Sector Academic/University
Country United Kingdom
Start 08/2017 
End 07/2018
 
Title Syncphonia 
Description Pilot version of networked synchronised music notation for musical ensembles, available via app store (free). 
Type Of Technology Software 
Year Produced 2017 
Impact Profile on Australian Broadcasting Corporation (The Science Show). https://soundcloud.com/ed-hughes-4/keeping-the-wheels-on-in-orchestra-rehearsal 
URL https://www.syncphonia.co.uk/
 
Title syncphonia apps 
Description The main outcome of the project was the development of our research prototype into commercial software. This software is now available on the app store, as Syncphonia Conductor and Syncphonia Performer. It was released in sept 2017 
Type Of Technology Software 
Year Produced 2017 
Impact The software has been used successfully in schools, and we are developing it further through our recent enterprise panel funding. 
URL https://www.syncphonia.co.uk/
 
Description School visit (Burgess Hill) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Syncphonia 2.0.
15 GCSE students downloaded the App onto their iPads then performed the Menuetto from Mozart: Eine Kleine Nachtmusik, one of their set works to study at GCSE. Demonstrating value to new group of beneficiaries (older average age group).
16.5.2019
Year(s) Of Engagement Activity 2019
 
Description Six week trial of technology with primary school orchestra (Wallands School, Lewes) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact Use of Syncphonia in Wallands School Orchestra weekly from September 2018. Feedback included:
'the technology proved beneficial immediately. Two younger violinists also found the app very helpful in keeping their place. The response I have had from all the players is that using Syncphonia is very helpful.'
Stephen Giles, music teacher, East Sussex County Council/Brighton Dome Brighton Festival.
Year(s) Of Engagement Activity 2018
URL http://netem.org.uk/
 
Description expolondon 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Schools
Results and Impact We presented Syncphonia at the Music and Drama Education Expo in London. We made contacts with educators, third sector organisations and industry.
Year(s) Of Engagement Activity 2018
 
Description expomanchester 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Schools
Results and Impact We presented Syncphonia at the Music and Drama Education Expo in Manchester. Through this event, we made contacts with schools and third sector music organisations.
Year(s) Of Engagement Activity 2017
 
Description science festival 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Public/other audiences
Results and Impact We ran an event 'orchestra hero' as part of the British Science Festival in Brighton, UK. This event used the Syncphonia system to enable members of the public with no musical experience to take part in an orchestra performance with professional musicians. Approximately 15 members of the public took part, along with a selection of local musicians and a conductor who is an advocate of our software. The event was reported to be successful by participants who clearly enjoyed the opportunity for the new experience. After the session, we had some interesting group discussion and received useful feedback on the design of the software.
Year(s) Of Engagement Activity 2017
URL https://www.britishsciencefestival.org/event/orchestra-hero/
 
Description syncphonia release 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact We held a very well attended press-conference to launch our software, Syncphonia. During the press conference, we demoed the software with a small ensemble, and spent a while taking some very interesting questions from journalists from a range of media outlets. This resulted in several articles in the press, and a radio interview on Australian ABC channel http://www.abc.net.au/radionational/programs/scienceshow/keeping-the-wheels-on-in-orchestra-rehearsal/8974978. Following the launch, we received numerous requests for information.
Year(s) Of Engagement Activity 2017
URL http://www.sussex.ac.uk/broadcast/read/41418