Pet Sounds: Creating Music using Social Media and Mobile Technologies

Lead Research Organisation: Queen's University of Belfast
Department Name: Sch of Arts, English and Languages

Abstract

The proposed project seeks to explore the ways in which the power of social media and social interactions-whether online or in the real world-can be harnessed to create digital music. The core methodology of the project is to develop a mobile app, Pet Sounds, which will be used to study these interactions. Pet Sounds will enable users to create a musical self-portrait or 'musical selfie', a musical representation of the user that reflects social experiences. In order to modify this composition, users must engage in different kinds of social interactions. These interactions might take place via social media like Facebook or Instagram. For example, chatting with another user might result in one kind of musical outcome, while tagging friends in a photo might result in another kind of musical outcome. The composition can evolve only through these kinds of social interactions, as well as by undertaking real-world social activities with friends. The musical selfie logs these interactions via different kinds of compositional transformations. For example, the composition might grow longer or shorter in duration; it might add a new voice or voices; it may change timbral qualities, tempo, rhythms, harmonies, and so on. The sounds and music will be generated almost entirely via social activities. Users will be able to share their musical selfies with each other, and collaborate with other users in developing their compositions.

This study entails an interdisciplinary collaboration that spans musicology, ethnography, and composition and software design. The researchers will develop Pet Sounds in the context of workshops with diverse groups of participants who have varying backgrounds in music, music technologies, and social media use. The app will be designed to appeal to specialist and non-specialist users alike. While the app itself will not be released as an output during the lifespan of the project, its prototyping will provide the basis for studying socially-based musical interactions. These interactions will be studied via critical perspectives in musicology, sound studies, composition, and interaction design. The outputs of the project will include a co-authored journal article, conference paper, public presentations and performances that will examine the many creative, critical/theoretical and technical dimensions of the research.

The research team will be based at the Sonic Arts Research Centre at Queen's University Belfast, the University of Oxford, and the University of Bristol. Workshops will take place at the Pervasive Media Studio at Watershed, Bristol. This non-academic partner will be important in attracting a diverse community of creative practitioners, technologists, and general audiences. At Watershed, the research team will conduct a series of Design and Play Workshops, and a Lunchtime Talk. In Belfast, the research team will partner with the Junior Academy of Music to reach music educators and young people aged 12 to 17 years in the context of workshops on 'App-ifying Music'. Other key elements of impact will include hosting a panel for media artists and technologists at an international media arts centre, and a presentation for creative industry professionals at a major industry event.

The aim of the research is to show how music making can evolve as a social activity using new digital technologies. The proposed project will be playful, collaborative, interactive, participative and fundamentally social. In this way it diverges from projects in music technology that either require a great deal of skill on the part of the performer or expertise on the part of the listener. Pet Sounds will show that 'new music' and digital music cultures can be inclusive, engaging, and friendly. It will further show that users or participants themselves can play a large role in collaboratively creating compositions that are musically interesting and that draw upon users' own experiences in meaningful ways.

Planned Impact

Pet Sounds is a playful and inclusive project that aims to connect with people in ways that reflect everyday life experiences, social activities, and common uses of ubiquitous technologies in creating music. Therefore our impact strategy is developed with this in mind. We aim to reach a variety of groups outside academia, including the general public with interests in digital music and social media, music educators, young musicians, media artists and creative technologists, and creative industry professionals. Each of these groups has a different stake in the potential of our project to transform digital music cultures.

The general public with interests in music, including musicians as well as those who have no prior training in music, can benefit from our project in a number of ways. In the first instance, they can create music and develop a musical identity in the form of a 'musical selfie,' by using the Pet Sounds app. Although a public release of the app is an intended output of the project, we will use pilot versions of the app at workshops with participants who will be engaged at different stages in the project. These participants will have significant input into the design of the Pet Sounds app, which we will seek to release in the future via follow-on funding. Design and Play Workshops will take place at the Pervasive Media Studio at Watershed Bristol, a non-academic partner of the project. At Watershed we will also deliver a Lunchtime Talk, aiming to reach a wide and varied audience of creative technologists and the general public.

We will reach music educators and young musicians in the context of workshops co-delivered with staff at the Junior Academy of Music (JAM), a music programme for children based at Queen's University Belfast. At JAM we will deliver a three-hour workshop to local music educators on using apps for music tuition. These educators will be able to apply the knowledge gained from this workshop in the context of individual and classroom teaching. We will also engage a group of young people aged 12 to 17 years in the context of a daylong workshop co-delivered with staff in JAM FOUR, which is focused on Creative Music Technology. These young people will learn about composing in performing with digital music apps, and they will present a public performance at the end of the day, thereby gaining an opportunity to perform with new digital music technologies.

We will aim to reach an international community of media artists and creative technologists in the context of a panel at a major international centre for arts and technology. This panel will focus on creating music via social media, and will present new perspectives from music, interaction design, and humanities-based research.

Finally, we will aim to present our project to creative industry professionals at the SXSW Interactive Festival. These professionals, representing a wide range of people who work in high-tech creative industries-whether as designers, entrepreneurs, software developers, investors, etc.-will benefit by hearing about the challenges and successes of our project. Since our project represents a new merging of social media technologies, mobile media technologies, and digital music technologies, our presentation-as well as our informal discussions as facilitated by SXSW Interactive Festival activities-will benefit a large international group of potential stakeholders who have interests in these areas. Similarly, we will benefit from this networking opportunity, by meeting potential collaborators and investors who will support the project beyond the lifespan of the grant, when we will aim to produce a public/commercial release of the project through the PI's interactive music label Optophono.

Publications

10 25 50
 
Description The 'Pet Sounds' project explored the possibilities of music making in the age of social media. Social networks are surprisingly inert when it comes to musical composition. On Facebook, for example, music is frequently shared but it is rarely composed. While various other platforms for online composition do exist, they typically follow antiquated models of collaboration, whereby, say, different musicians consecutively add new tracks to existing songs. With 'Pet Sounds' we sought to enable new modes of composition whereby anyone might create music through everyday online social interactions. We further aimed for this music to reflect the various genres born of social media, i.e. the selfie, the tweet, the emoticon. Our research was therefore propelled by questions like 'what might a musical selfie sound like?' and 'how might an audio emoticon extend the language of online conversations'? The research team, which included a musicologist, composer, computer scientist and social scientist, hosted workshops with a wide range of stakeholders: young musicians, music tutors, designers/technologists, and the general public. Together we discovered the value of speculative design in the creation of new musical interfaces. Overall, the project revealed the vast potential for new kinds of music making in today's socially networked world.

The project evolved in a number of ways over the course of the research period. While we had originally planned to prototype only a single smartphone app (on 'musical selfies'), we instead developed several interconnected apps. While our previous software prototypes had focused on how to translate a visual idea (the selfie) into an aural or musical domain, a conference CFP inspired us to re-conceptualise the selfie not only as visual 'image' but as a form of visual 'feedback'. One of our apps, 'Helmholtz', develops this idea by exploring different forms of acoustic resonance and feedback. Thus, the 'Helmholtz' app embodies a different kind of critical design approach than we had initially envisioned for our project.

We delivered six workshops through the project. Our first workshop, at Pervasive Media Studios in Bristol, made it clear to us that speculative design or 'design fictions' - methodologies that have been in critical design practice (Dunne 2005; Dunne and Raby 2013; Auger 2013) but rarely by the music tech community - could become an integral part of our project. We co-authored an online tutorial on music apps for music educators with a serendipitous collaborator; we further published a project website that documented our design studies and provided information on the project (www.echo-snap.org).
Exploitation Route The Digital Transformations Small Grants Call invited applicants to create 'playful, challenging and thought-provoking projects in which practice-based researchers work with colleagues in other disciplinary areas to explore and reflect on how our use of digital technologies is transforming engagement with both past and contemporary cultures and societies'. The scope of this Call enabled us to take risks that would have otherwise been difficult. Our project focused on innovative design strategies within the creation of new 'social music' genres such as 'Musical Tweets' and 'Musical Selfies' - genres that have the potential to profoundly transform social media. Our project embraced both interdisciplinary and critical perspectives that are rare within computer music. In particular, through our project we discovered the vast potential of speculative design to energise and transform research/design methods in NIME, and, more broadly, in the area of digital technology and software design. We probed the potential of speculative design through various workshops with diverse groups. Our workshops with music students, music educators, media artists and the general public positioned participants not as potential 'users' of our software prototypes but as potential collaborators/co-creators/designers. Participants directly informed and influenced the design process in unpredictable ways, thereby demonstrating how the research and design process within NIME might be democratised and more inclusive, incorporating perspectives from beyond the immediate academic/research community.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education

URL http://www.echo-snap.com/
 
Description App-ifying Music Tutorial 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact The research team met with music educators in Belfast to discuss strategies for incorporating apps into music teaching. In collaboration with one of these educators we developed a tutorial on 'App-ifying Music' in collaboration with one of the educators, Dr John D'Arcy. This tutorial, which is published on our project website, can be used by music educators and the general public as a primer on music apps.
Year(s) Of Engagement Activity 2016
URL http://www.echo-snap.com/tutorial/
 
Description Design Workshop at Pervasive Media Studios 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact The research team hosted a day-long design workshop at the Pervasive Media Studios in Bristol. The workshop was aimed at design professionals, new media technologists and the general public. This workshop focused on the idea of the 'musical selfie'. Through hands-on workshop activities and structured discussion we probed the idea of the 'musical selfie', its potential design/implementation, and its potential impact upon social media communities.
Year(s) Of Engagement Activity 2015
URL http://www.echo-snap.com/musical-selfie-workshop-in-bristol/
 
Description Junior Academy of Music workshops 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Schools
Results and Impact The research team hosted two workshops with young students (aged 12-17) enrolled in Music Technology courses at the Junior Academy of Music in Belfast. We invited students to create and present 'A Theme Tune for Yourself'. These workshops focused on the challenges of creating music that represents social identities, and what it would mean to create and share self-representational music via social networks.
Year(s) Of Engagement Activity 2016
 
Description MusicTechFest workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dr Chris Haworth delivered a hands-on workshop exploring music apps with music industry professionals at #MusTechFestResearch in Berlin, Germany. This symposium brought together scholars, industry professionals and musicians in discussing critical issues related to music technology. Dr Haworth also presented a paper as part of a sub-theme titled 'Music as a Site for Collaboration across Divisions'. It focused on the capacity of mobile technologies to bring complex or experimental sonic ideas to younger audiences.
Year(s) Of Engagement Activity 2016
URL http://musictechfest.net/mtfberlin2016/
 
Description Musical Selfies Workshop at ISEA 2016 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The research team held a day-long workshop for an international group of media artists at the annual meeting of the International Society for Electronic Arts (ISEA2016) in Hong Kong. For this workshop, titled 'Musical Selfies: Feedback and Self-Reflection through Mobile Composition', we invited participants to prototype a putative musical selfie app/device. Moving from abstract to concrete, we began with structured discussion; moved onto a 'musical selfie manifesto'; then designed prototype devices that drew out different themes of the discussion, while adhering to the terms of the manifesto; and finally, presented the results to the rest of the group in a final performance. It was a huge success: lively, energetic and creative, with all participants engaging fully and reporting positively on the experience. A more detailed report on the workshop appears on our project website.
Year(s) Of Engagement Activity 2016
URL http://www.echo-snap.com/isea-workshop-report/
 
Description Panel at SXSW 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact We disseminated our research during a panel at South by Southwest (SXSW) in Austin, Texas (USA), a leading networking event for creative industry professionals. The panel, which featured presenters from Pervasive Media Studios in Bristol, where the Co-Investigator is based, was titled 'Subtle Interfaces: Designing for Calm Tech', and explored the use of new technologies in the design of tranquil environments.
Year(s) Of Engagement Activity 2016
 
Description Sonic Arts Research Centre workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Undergraduate students
Results and Impact The research team hosted a workshop for undergraduate-level musicians at the Sonic Arts Research Centre. For this workshop we had invited participants to create 'Musical Tweets': miniature compositions that they might develop or share through social media networks. The workshop focused on various aspects of conceptualising and creating Musical Tweets, from compositional approaches and aesthetics to the implications of sharing and composing Musical Tweets through social media. It generated a great deal of discussion and the students reported a very positive learning experience.
Year(s) Of Engagement Activity 2016