TOPUP: Transform Outcome and Process Utilities and Programs - Upgrading music performance and intervention research equipment for creative processes
Lead Research Organisation:
Anglia Ruskin University
Department Name: Fac of Arts, Humanities & Social Sci
Abstract
Being human means to be creative, to relate to others, to be part of a culture that is developed in places where people produce it as a creative expression of their social identity. Music is one of the cultural assets of regions in which they create a particular expression of their identity that relates to places and ways in which humans interact and create music. In front of the LIPA in Liverpool a sculpture displays suit-, drum-, and guitar cases and from there you have a view on the Mersey river floating towards the ocean, a few doors down you can enter the Jacaranda Club etc.; these places were instrumental for developing creativity that allowed four young men to travel the world and the creative industries to grow into world-leading figures.
At the core of this is the creative process at a certain time and place and the 'chemistry' of humans relating to each other in a creative process that can bring about change in practice and outcomes. The 'right music at the right time' has an impact on the listener, the expressions of emotions and thoughts in a song can represent and symbolise a common denominator of what many may feel and think at a particular time. Music as the language of emotions allows expressing the momentary multimodal complexity of how things relate to each other and can help to discover ourselves in what we feel and think about the societal challenges and how we join-in, to bring people and organisations together to support happier and healthier lives.
The Cambridge Institute for Music Therapy Research (CIMTR) is dedicated to study the creative process to bring about change in people's lives, to initiate processes to do things differently and to overcome the burden experienced when living with disease. CIMTR researches ways in how we can rethink the medical idea of outcomes and treatment processes as we are dealing with interactive creative processes and performances during musical interventions and although performed with a body, the expression of being human is not necessarily identical with the materialistic description of dysfunctions of body parts or physiological processes. To be incorporated in a body is different from its physical functions as we live in our body and being and feeling healthy and creative can also be expressed in an excessive pushing of the body's boundaries for artistic purposes.
In short, CIMTR's research adds a humanistic, narrative perspective of the multimodal human experience to the datasets recorded during creative processes and how change can be described integrating the creative and physiological processes at a certain time and place. This can be a therapy room in the intimacy of a healing contract between a person seeking help and a professional devoting his/her creative sensitivity to the healing process and not to the artistic outcome, it can be a stage and how the performers are is inspired from each other or from the audience, it can be the creation of an audio mix in the studio where shared listening may create a sound that is signified from the people involved, it may be the calibration of audiovisual settings for immersive processes, it may show how a drama therapist mirrors and extends the body language of a person locked in a narrowed regulation of emotion, or how an actors improvising roles of a play to transform the given narratives.
With this bid we aim to expand our methodological repertoire of technological solutions to focus on how the body performs in different settings and how this performance looks like with different lenses of brain research, how people look at things happening around them during creative processes and track their gaze and objects focused upon, how their bodies move through spaces in which they are involved in creative processes and to utilise the language of current technological measures describing what we can observe, track and evoke with paradigms that are adequate to represent the creative process of being human.
At the core of this is the creative process at a certain time and place and the 'chemistry' of humans relating to each other in a creative process that can bring about change in practice and outcomes. The 'right music at the right time' has an impact on the listener, the expressions of emotions and thoughts in a song can represent and symbolise a common denominator of what many may feel and think at a particular time. Music as the language of emotions allows expressing the momentary multimodal complexity of how things relate to each other and can help to discover ourselves in what we feel and think about the societal challenges and how we join-in, to bring people and organisations together to support happier and healthier lives.
The Cambridge Institute for Music Therapy Research (CIMTR) is dedicated to study the creative process to bring about change in people's lives, to initiate processes to do things differently and to overcome the burden experienced when living with disease. CIMTR researches ways in how we can rethink the medical idea of outcomes and treatment processes as we are dealing with interactive creative processes and performances during musical interventions and although performed with a body, the expression of being human is not necessarily identical with the materialistic description of dysfunctions of body parts or physiological processes. To be incorporated in a body is different from its physical functions as we live in our body and being and feeling healthy and creative can also be expressed in an excessive pushing of the body's boundaries for artistic purposes.
In short, CIMTR's research adds a humanistic, narrative perspective of the multimodal human experience to the datasets recorded during creative processes and how change can be described integrating the creative and physiological processes at a certain time and place. This can be a therapy room in the intimacy of a healing contract between a person seeking help and a professional devoting his/her creative sensitivity to the healing process and not to the artistic outcome, it can be a stage and how the performers are is inspired from each other or from the audience, it can be the creation of an audio mix in the studio where shared listening may create a sound that is signified from the people involved, it may be the calibration of audiovisual settings for immersive processes, it may show how a drama therapist mirrors and extends the body language of a person locked in a narrowed regulation of emotion, or how an actors improvising roles of a play to transform the given narratives.
With this bid we aim to expand our methodological repertoire of technological solutions to focus on how the body performs in different settings and how this performance looks like with different lenses of brain research, how people look at things happening around them during creative processes and track their gaze and objects focused upon, how their bodies move through spaces in which they are involved in creative processes and to utilise the language of current technological measures describing what we can observe, track and evoke with paradigms that are adequate to represent the creative process of being human.
Publications
Fachner J
(2023)
Music therapy, neural processing, and craving reduction: an RCT protocol for a mixed methods feasibility study in a Community Substance Misuse Treatment Service.
in Addiction science & clinical practice
Maidhof C
(2023)
Intra- and inter-brain coupling and activity dynamics during improvisational music therapy with a person with dementia: an explorative EEG-hyperscanning single case study.
in Frontiers in psychology
Pasqualitto F
(2023)
Neuroplastic Changes in Addiction Memory-How Music Therapy and Music-Based Intervention May Reduce Craving: A Narrative Review.
in Brain sciences
| Title | In Situ |
| Description | live performnce with voice, guitar and saz along with talk about music, technology and improvisation in a clinical research context. Performnce held at the Churchill college in Cambridge as part of the Science Festival. The performance included a stream of thoughts presentation based on songs from Bob Dylan and d Schubert's Winterreise (Das Wirtshaus) |
| Type Of Art | Performance (Music, Dance, Drama, etc) |
| Year Produced | 2023 |
| Impact | further invitation to performances |
| URL | https://cimacc.org/background/ |
| Description | Objectives from our AHRC application were: 1) Rethink and transform medical paradigms by observing creative practice, 2) Implement mobile time- and place-based research approaches, 3) Integrating multimodal methodologies to investigate the creative process, 4) Understanding how music brings about change during creative interventions. Given our CREsCa win we can now do what we aim for: rethinking medical paradigms for the humanities and investigate the creative process in an interdisciplinary way by respecting and adapting to the creative process -because it is the base of therapeutic work with music and the arts- while gaining data with tools based in medical imaging and behavioural observation technology in order to objectify and materialise data that can be treated with advanced quantitative methods. For that, we upgraded our behaviour observation facilities with state-of-the-art AV and motion capture recording systems. These upgrades were accompanied by building improvements such as temperature control systems, storage solutions, and complex wiring between therapy labs and equipment in recording / control rooms, enabled by internal funding (£195k). In one of our recent projects, we set-up a mobile recording studio in a Community Substance Misuse Treatment centre. This allowed access to participants using the service. The mobile recording studio as a place where music therapy happened in form of music production and creative play became part of the treatment centres offering during the study. No extra visits and barriers for participants were hindering them to take part, and to record and play music about their current situation. In this context the grant further enabled one of our PhD students to analyse biomarkers with a complex clinical analysis software. He was able to show how participants exhibited a frontal activation pattern after music therapy that indicates modulation of processing strong emotions and how craving is affected by being involved in music therapy. It needed a substantial sum to get access to this software that also allows comparisons to normative EEG databases to compare individual brain activity with a neurometric tool used in clinical settings. We have been able to apply complex equipment arrays in our Lab to study intervention sequences in neurorehabilitation including AV, enabling machine learning assisted video analysis, and created a comparable paradigm making use of technical solutions studying defined music intervention sequences by synchronising EEG, MIDI, mocap and AV data to detect subtle improvements in stroke survivors. With the TOPUP equipment this can be done as in other world-class labs that have published in the same field. The multimodal integration of the different data streams is of importance to focus on in-situ sequences that may be evoked or emerge in the therapy process and is of importance when studying interaction in music and musical communication in dyads or ensembles. We were able to upgrade our infrastructure for recording multimodal und multi-person data streams (including audio, video, body and gaze behaviour, brain and heart activity from up to 4 persons at the same time) and analysis of these complex datasets. This will enable ongoing and future research a better understanding how music brings about change during creative interventions. The new AV system in the therapy rooms will substantially support music therapy assessment procedures. The new AV system installed will be used by music and drama therapy students and practitioners recording their sessions with 4 synchronised cameras and three microphones showing the therapeutic interaction from 4 angles. This may reveal details not recognised during the session. The same goes for music and drama students and researchers analysing creative processes in ensemble play, improvisation, and performance. Current and future post-graduate researchers are and can use our new and upgraded infrastructure, including new brain imaging equipment, for a huge variety of different research projects. Similarly, new post-doctoral researchers can implement new projects and we expect significantly more collaboration opportunities in the future. |
| Exploitation Route | Not yet, PhD and data analysis still ongoing, will be presented in peer conferences in more detail and published. |
| Sectors | Creative Economy Education Healthcare Culture Heritage Museums and Collections Pharmaceuticals and Medical Biotechnology |
| Description | Extension feasibility planning RIBA3 + 4 |
| Amount | £18,000 (GBP) |
| Organisation | Anglia Ruskin University |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 03/2024 |
| End | 07/2024 |
| Description | Internal capacity QR funding for facility upgrade and building work, interior and exterior planning and realisation |
| Amount | £190,000 (GBP) |
| Organisation | Anglia Ruskin University |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 04/2023 |
| End | 08/2023 |
| Description | Temporary admission of a Lab Developer |
| Amount | £11,000 (GBP) |
| Organisation | Anglia Ruskin University |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 11/2023 |
| End | 07/2024 |
| Title | AV and HR (pulse rate) recordings of participants listening to preferred music |
| Description | including pulse rate data recorded with OH1 and Samsung Galaxy 4; AV monitoring data from participants (confidential) while listening to thei music and talking with therapist. |
| Type Of Material | Data analysis technique |
| Year Produced | 2022 |
| Provided To Others? | No |
| Impact | documentation and understanding how music influences HR and HRV measures ind PwDs and how it decreases agitation |
| Description | Musical Minds - The Science Museum Group |
| Organisation | Science Museum Group |
| Department | The Science Museum |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | We have been asked to contribute one of the first RadioMe prototyp elements (a fitnes tracker used for the puls rate recordings during music listening) and have lend it to the organisation. |
| Collaborator Contribution | They offer an exhibition space and to keep it in their Musical Mind portfolio to go on national and international tour. 'Musical Mind will showcase new music-making opportunities that science and technology have unlocked and the fresh perspectives they are revealing about how music affects our bodies and minds. Through incredible and intriguing objects, musical commissions and interactive experiences, discover how technological advancements continue to push the limits of music and to make playing music more accessible for everyone. This riotous, hands-on experience will encourage us all to feel, to remember and to reflect on what music means to us and the lives of others. Musical Mind is the headline exhibition for Manchester Science Festival, premiering at the Science and Industry Museum in Manchester before a national and international tour.' |
| Impact | Fachner, J. (2022, Oct 21-30). Turn it up - the power of Music. Paper presented at the Manchester Science Festival, Manchester. |
| Start Year | 2022 |
| Description | Turn it up - The Power of Music - Science Museum London |
| Organisation | Science Museum Group |
| Department | The Science Museum |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | One of our test objects for the RAdioMe project was put under glass for the The power of Music exhibition in Manchester. It has now traveled to the Science Museum in London and became part of the exhibition there. For the opening of the exhibition a panel discussion was organised and I was invited to contribute to the panel discussion. |
| Collaborator Contribution | The SMG organised the exhibition and a series of events and promotion material |
| Impact | On the 19th of October the Turn-it-up exhibition was opened and the panel discussion was part of the event |
| Start Year | 2022 |
| Title | MEASURE (Musical Engagement Analysis Support Using REcordings) |
| Description | I (Richard Parker) developed a software toolkit incorporating ML to automate the annotation of 18 types of behaviour related to pose, movement, interaction, facial expression and audio events. Following testing on artificial examples, I evaluated the software under realistic conditions through two case studies: children on the autism spectrum and adults with disorders of movement. For each, I compared the information yielded by my models with the annotation scheme used by the therapist and applied selected models to video extracts. Both of the studies were viewed as credible by the therapists. I compared and contrasted my tool with related tools and have identified opportunities to build on this research. My main finding is that ML can support video analysis for MT by (1) extracting from MT videos data for human body movement, facial expression and the location of objects of interest and (2) processing those data to detect specific types of client behaviour. This, is, to my knowledge, the first time that the feasibility of automated annotation for videos featuring a range of MT client types has been demonstrated in a single integrated tool. |
| Type Of Technology | Software |
| Year Produced | 2023 |
| Impact | Music Therapy PhDs and computer affine music therapist are interested to make use of the software to analyse their video recordings. The software is also used in current research studies in Neurorehab to study movement smoothness anad other selected measures. |
| Description | BBC World Service Interview for 'People Fixing the World' |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | Richard Kenny, a journalist for this BBC program, visited a couple who are RadioMe participants and interviewed them in their home about music listening and the RadioMe system, he then interviewed Dr Alex Street about the music compilation and listening process, the science behind music listening in dementia and the current and future development of the system. The podcast will be broadcast in April 2024. |
| Year(s) Of Engagement Activity | 2024 |
| Description | Dyadic nonverbal & heart-rate synchrony during music therapy and its relationship to self-reported therapy readiness - Paper presented at 17th World Congress of Music Therapy, Vancouver, Canada in July. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This was a presentation of our PhD student making use of several analysis methods employed in a international research project together with researchers from Austria and Switzerland. TOPUPs focus is on analysis of the interaction in a therapy setting and here we are looking into a particular way of applying a motion energy analysis. |
| Year(s) Of Engagement Activity | 2023 |
| Description | EEG workshop |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | Colleagues and PhD students in CIMTR research institute and in the wider faculty were invited to get introductory training of utilising EEG for their research projects. an after noon of presentation and discussion with the distributorof a professional EEG recording tools was organised to learn about basic concepts and general applications of the imaging tools and analysis software. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Interactive Radio, Biomarker and Music (Therapy) Responses to Reduce Agitation in Dementia |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The World Congress of Music Therapy is held every three years. Music therapy professionals and experts in related fields from around the world gather at the congress to share ideas, experiences, trends, and research outcomes. The World Congress of Music Therapy is hosted by a WFMT organizational member in conjunction with a local host. The site of each congress is determined by a bid process undertaken by the WFMT Council six years in advance. The Federation makes an effort to circulate the location of the congresses among the eight regions. WFMT members who would like to host a World Congress should contact the WFMT Past President to request proposal details. July 24-29, 2023 Dear Friends: I am pleased to offer my warmest greetings to everyone attending the 17th World Congress of Music Therapy (WCMT 2023), hosted by the Canadian Association of Music Therapists (CAMT). This triennial event provides music therapy professionals and specialists associated with the field an important platform to discuss current research and share insights and best practices. I am certain that delegates will make the most of the many educational and networking opportunities planned for the next six days centred on this year's theme: "Music Therapists: Reflecting, Connecting and Innovating in the Global Economy." I would like to commend the CAMT for offering this congress in both in-person and virtual formats and for putting together an informative and rewarding program for everyone involved. I would like to thank the citizens of Vancouver for their warm hospitality. Please accept my best wishes for an engaging experience. Sincerely, The Rt. Hon. Justin P. J. Trudeau, P.C., M.P. Prime Minister of Canada |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.wcmt2023.org/program-at-a-glance/ |
| Description | Intra- and inter-brain coupling during music therapy. Paper presented at RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion, Oslo, Norway. June 2023 |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Other audiences |
| Results and Impact | The MIRAGE Symposium #2: Music, emotions, analysis, therapy ... and computer is an internationa gathering of experts in the field of music information retrieveal and we were presenting a multimodal study integrating brain and MiDI data to analyse music performance during music therapy improvisation with a dementia patient. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.uio.no/ritmo/english/projects/mirage/events/2023/june/mirage-symposium-2.html |
| Description | Music Therapy Process and Social Neuroscience Approaches to Investigate Mechanisms of Change. Panel discussion at the 17th World Congress of Music Therapy, Vancouver, Canada. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This was a panel discussion organised from us with colleagues from the USA and Austria. The panel was presented at the 17th World Congress of Music Therapy, Vancouver, Canada in July. About 1700 delegates were preseent at the conference and this panel saw over 60 attendees. Several plans of further collaborations were discussed. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://issuu.com/presidentwfmt/docs/mt_today_vol.18_no._1 |
| Description | Music and the Brain |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | We know that music has the power to move us in profound ways, but what is happening to the brain when we listen to music, and can we harness an understanding of this interaction for therapeutic uses? Studies have demonstrated that listening to music can have a positive effect on your learning, cognitive function and general wellbeing, and research has shown that listening to and playing music can reduce the stress hormone cortisol. There is also some indication that professional musicians have healthier brains - indeed the renowned Oliver Sacks said, "Anatomists today would be hard put to identify the brain of a visual artist, a writer or a mathematician - but they would recognize the brain of a professional musician without a moment's hesitation." But what is happening to the brain when we listen to music, and can we harness an understanding of this interaction for therapeutic uses? In this discussion, a panel of scientists and musicians come together to examine the fascinating relationship between the human brain and the sound of music. To help answer these questions we are joined by an expert panel including: • Professor Green-aka Stephen Manderson, rapper, singer, songwriter, actor, best-selling author of Lucky, mental health activist and patron for the charity Campaign Against Living Miserably. • Imogen Heap-Grammy and Ivor Novello award-winning composer and recording artist, whose pioneering work in the intersection of music and tech has produced the gestural music-ware MI.MU Gloves system and The Creative Passport, an integrated digital ID solution to empower music makers. • Professor Jörg Fachner-Professor of Music, Health and the Brain at Anglia Ruskin University, and Co-Director of Anglia Ruskin's Cambridge Institute for Music Therapy Research. • Professor Lauren Stewart-Professor in Psychology & Director of MSc in Music Mind and Brain, Goldsmiths University of London. • Samira Ahmed (Chair)-Award-winning journalist and broadcaster, presenter of Radio 4's Front Row and BBC1's Newswatch. Approx. 250-300 attendees are expected. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.ianvisits.co.uk/calendar/2023/10/19/music-and-the-brain-356373 |
| Description | Nachtclub ÜberPop - Podcast Mother's Little Helper produced from NDR |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | This was a podcat produced for a major German Radiostation NDR done from a German journalist (Henning Cordes) after an interview with about music and drug-induced alterations of consciousness and how this may impact artistic creativity. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.ardaudiothek.de/episode/nachtclub-ueberpop/mother-s-little-helper/ndr-blue/12391059/ |
| Description | Noldus VISO training |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Professional Practitioners |
| Results and Impact | This training workshop was part of the training program for the equipment purchased with the TOPUP funding. However the training is also for our music therapist practitioners working with patients in the building and for students in the postgraduate courses of the university. |
| Year(s) Of Engagement Activity | 2024 |
| Description | Shared Moments and Right Periods in Music Therapy in Neurorehabilitation. - 17th World Congress of Music Therapy in Vancouver, Canada |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Joint Paper Presentation at the 17th World Congress of Music Therapy in Vancouver, Canada in July about the results of a research project funded from the Austrian Govt. funding body CDG.. There were 1700 attendees at the conference from professional practitioners from all over the world. over 50 participants attended the presentation. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://www.wcmt2023.org/program-at-a-glance/ |
| Description | Shared music listening and neural processing of emotion during music interventions. Presentation and Performance at the Cambridge Festival; Churchill College, Cambridge University. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | The invited paper and performance presentation took part in an event organised from CIMACC at Churchill Colleage during the public Cambridge Festival. The Cambridge festival (Formall Cambridge Science festival and Festival of Ideas) is an annual event organised from both universities in Cambridge and is addressing the public as an audience, is opening the labs and showcasing research or inviting to panel dicussions, etc. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://cimacc.org/ |