The Kinesemiotic Body: a pragmatic account of the local discourse organisation of dance

Lead Research Organisation: Loughborough University
Department Name: Criminology, Sociology and Social Policy

Abstract

The Kinesemiotic Body project brings together an interdisciplinary group of researchers with the aim of evaluating whether a description of dance discourse informed by multimodal discourse analysis and visualised through enriched videos can capture the way dance communicates through a flow of choreographed sequences in space, and whether this description can support the interpretative process of nonexpert audiences.
Our theoretical framework will be based on an extended dynamic theory called segmented discourse representation theory (SDRT) and on the Functional Grammar of Dance Movement created by PI Maiorani, thus leveraging the work carried out on a previous project funded by Loughborough University Calibre programme for priority research areas, which also included the collaboration of Co-I Zecca and Lock and collaboration with the English National Ballet. The project will also leverage on previous work carried out by Bremen PI Bateman and his team on developing a SDRT-based framework for analysing movement-based communication.
The project will first develop a descriptive framework for analysing how dance creates meaning through structured movement in interaction with the space that surrounds it; the framework will then be used to enrich video materials capturing dance sequences performed by an English National Ballet dancer with a special set of discursive labels; the effectiveness of these video annotations will then be tested on sample audiences of non-experts to understand whether and how the labels support the audiences' interpretation of the dance sequences.
The development work and the testing will be carried out over 24 months by the Loughborough and Bremen teams according to a series of specific workpackages leveraging off the diverse areas of disciplinary strength of the team members as well as those members strong experience in interdisciplinary cooperative research.
International dissemination and outreach will be maintained via a website and the publication of papers at a disciplinarily-diverse range of international conferences, together with a final workshop at the London campus of Loughborough University involving international invited speakers and the involvement of prospective non-academic partners to explore further research developments and collaborations.
Our long-term goal is to develop an interdisciplinary area of research focusing on movement-based communication that can extend beyond the study of dance to other movement-based forms of communication and performance and inform foster the creation of partnerships between the academia and the institutions that host and promote such disciplines.

Publications

10 25 50
 
Title Ballerinas in sensor suits 
Description Short documentary about The Kinesemiotic Body project with the participation of the English National Ballet 
Type Of Art Film/Video/Animation 
Year Produced 2023 
Impact Requests from other universities for sharing datasets and models of analysis. BBC request for an interview during news broadcats East Midlands Today. 
URL https://youtu.be/HPwLpGzIe3g
 
Description We have been able to complete a full segmentation of the way dance, and specifically ballet, is structured in multi-level units that can be assembled to create movement-based communication.
We have created a Controlled Vocabulary for using the Functional Grammar of Dance in ELAN to annotate dance as a form of movement-based communication, and this can be used by non-experts who are not familiar with the technical terminology used by dance specialists or professionals. The model has already been used by other scholars to be applied in circus studies, performance studies of music ensemble conduction, community-based experiences in real-life environments, fish population movement across corla reefs.
Using our new digital annotation method, we have created datasets and deposited them in the Loughborough University Repository for perusals by scholars who are inetersted in this new approach and need a practical way to annotate dance-based performances without any previous knowledge of dance-specific terminology.
We have also created prototype videos of visualisations that are automatically generated by a software reading our annotations to facilitate the dance meaning understanding of viewers from a non-specialistic audience. These videos have been tested on groups of volunteers both in the uK and in Germany and are still been used in Bremen for reception studies.
We have also fund out that through our annotation we can capture a dancer's specific balance management during a performance, especially in terms of legwork and footwork, which can be very useful also for rehabilitation.
Exploitation Route The English National Ballet is interested in participating in the creation of a website-based tool that will use our findings to create a choreographer-help tool.
We have also spread our fndings through outputs and presentations internationally to create an interdisciplinary community of scholars in the area of Kinesemiotics, which proved to be very successfyl with our symposium. New research projects are being initiated thanks to the effect of the symposium.
A collaborative project has been planned with the Royal Armoury museum in Leeds to create a movement-based experience for visitors: a cross-research council grant application has been submitted and has gone to the second phase of selection.
A collaboration with the Digital Humanities Lab at Brock University, Canada, has been initiated to apply the knowledge and tools developed in the project to the study and improvement of video game characters.
Sectors Creative Economy

Digital/Communication/Information Technologies (including Software)

Education

Healthcare

Culture

Heritage

Museums and Collections

URL https://www.uni-bremen.de/en/fb-10/forschung/institute/bitt/forschung-und-lehre/multimodalitaetsforschung-in-bremen/projekte/kinesemiotic-body
 
Description The English National Ballet is engaged in the search for funding for developing a website-based tool that draws on our research and that will offer a practical help to their choreographers and to dancers who learn new roles. The plan is also to make this tool available for use to other companies. The project has allowed for sufficient networking and exchange activity, fuelled also by our outputs, to create the nucleus of Kinesemiotics as an area of studies. As a matter of fact, Kinesemiotics will be one of the focal points of a forthcoming Research Topic by Frontiers in Communication, co-edited by Arianna Maiorani and John Bateman: "Movement, Embodiment, Kinesemiotics: Interdisciplinary Approaches to Movement-Based Communication". A collaboration with the Royal Armoury Museum in Leeds has initiated with the intent of creating a digital movement-based experience for visitors using the knowledge and tools developed during the project. A collaboration with the Brock University Digital Humanities Lab (CA) and some indie video game developers has been initiated with the intent of improving video game characters movement.
First Year Of Impact 2023
Sector Culture, Heritage, Museums and Collections
Impact Types Cultural

 
Title Dance discourse annotation using ELAN 
Description We have developed a novel method to annotate dance discourse following the Functional Grammar of Dance (FGD) developed by project investigator Dr Arianna Maiorani. The data annotation is conducted with ELAN software (https://archive.mpi.nl/tla/elan) developed by Max Planck Institute for Psycholinguistics. The annotation method we have developed showcases the translation of all aspects of the FGD into data annotation at different syntactic and semantic levels. Moreover, we have created controlled vocabularies based on the FGD. Controlled vocabularies contain lists of predefined values that code different aspects of dance discourse, and users can select from these lists when annotating. 
Type Of Material Data analysis technique 
Year Produced 2021 
Provided To Others? No  
Impact The notable impacts include 1) controlled vocabularies we have created can be reused and replicated to annotate dance discourse in general, and 2) data annotation provides researchers invaluable insights that may not be immediately noticeable through simple observation of dance sequences. 
 
Title The Kinesemiotic Body 
Description Experimental data in BHV (Biovision hierarchical data) format. The data collected relates to experiments with ballet dancers using Noitom Perception Neuron IMU based sensor suits. Data annotations in ELAN file with specifically created CV based on the Functional Grammar of Dance. The experiments were conducted with professional ballet dancers from the English National Ballet at their main premises in London. The experiments were carried out as part of the international collaborative research project "The Kinesemiotic Body: a pragmatic account of the local discourse organisation of dance" funded by the AHRC in the UK and the DFG in Germany. The annotations are made on some of the data collected during the experiments. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact This is a sample of the first batch of movement data captured through the use of the Functional Grammar of Dance. 
URL https://repository.lboro.ac.uk/articles/dataset/The_Kinesemiotic_Body/21207842
 
Title The Kinesemiotic Body 
Description The Kinesemiotic Body project brings together an interdisciplinary group of researchers with the aim of evaluating whether a description of dance discourse informed by multimodal discourse analysis and visualised through enriched videos can capture the way dance communicates through a flow of choreographed sequences in space, and whether this description can support the interpretative process of nonexpert audiences. Our theoretical framework will be based on an extended dynamic theory called segmented discourse representation theory (SDRT) and on the Functional Grammar of Dance Movement created by PI Maiorani, thus leveraging the work carried out on a previous project funded by Loughborough University Calibre programme for priority research areas, which also included the collaboration of Co-I Zecca and Lock and collaboration with the English National Ballet. The project will also leverage on previous work carried out by Bremen PI Bateman and his team on developing a SDRT-based framework for analysing movement-based communication. The project will first develop a descriptive framework for analysing how dance creates meaning through structured movement in interaction with the space that surrounds it; the framework will then be used to enrich video materials capturing dance sequences performed by an English National Ballet dancer with a special set of discursive labels; the effectiveness of these video annotations will then be tested on sample audiences of non-experts to understand whether and how the labels support the audiences' interpretation of the dance sequences. The development work and the testing will be carried out over 24 months by the Loughborough and Bremen teams according to a series of specific workpackages leveraging off the diverse areas of disciplinary strength of the team members as well as those members strong experience in interdisciplinary cooperative research. International dissemination and outreach will be maintained via a website and the publication of papers at a disciplinarily-diverse range of international conferences, together with a final workshop at the London campus of Loughborough University involving international invited speakers and the involvement of prospective non-academic partners to explore further research developments and collaborations. Our long-term goal is to develop an interdisciplinary area of research focusing on movement-based communication that can extend beyond the study of dance to other movement-based forms of communication and performance and inform foster the creation of partnerships between the academia and the institutions that host and promote such disciplines. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact Multimodal recognition of body position in live performance. Creation of a completely new Elan Controlled Vocabulary for dance analysis based on the Functional Grammar of Dance. 
URL https://repository.lboro.ac.uk/collections/The_Kinesemiotic_Body/6230502/1
 
Description The Department of Linguistics at the University of Bremen (Germany) 
Organisation University of Bremen
Country Germany 
Sector Academic/University 
PI Contribution The Kinesemiotic Body project is jointly funded by the AHRC and the DFG and is led by Dr Arianna Maiorani at Loughborough University (UK) and Professor John Bateman at Bremen University. The contributions made by the team at Loughborough University to this partnership include 1) empirical data collection with professional dancers from the English National Ballet, which in turn will be used for audience experiments by the team at Bremen University, and 2) data elaboration and theory development integrating Dr Maiorani's Functional Grammar of Dance.
Collaborator Contribution The contributions made by the team at Bremen University to this partnership include 1) the observation and evaluation of data collection with professional dancers from the English National Ballet for audience experiments purpose; 2) the setting up of audience experiments sessions, and 3) the creation and update of the project website.
Impact A project website has been created (http://www.kinesemiotic-body.com). The collaboration is multi-disciplinary. Desciplines involved include Multimodality, Applied Linguistics, Engineering and Computer Science.
Start Year 2021
 
Description An radio interview with BBC Radio Leicester 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact The interview was included in the regular show "Curious Kids" and included a presentation of the project's work done with the English National Ballet.
Year(s) Of Engagement Activity 2023
 
Description Article for The Conversation 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact We were contacted by The Conversation to publish a piece on The Kinesemiotic Body Project because they picked on our academic publications and the fact that we were working with an interbational ballet company. They were interested in informing a general audience of how dance is being studied as a form of communication and how this can potentially help in making this form of art more accessible to non experts.
Arianna Maiorani and Chun Liu pitched a piece and it is going to be published as an article for a wide general audience on The Conversation entitled: " Ballet dancers in sensor suits: new research explores how dance is used as a form of communication" in the week commencing on the 13th of May 2023. The article also features visual research material.
Year(s) Of Engagement Activity 2023
URL https://theconversation.com/profiles/arianna-maiorani-1422056/dashboard
 
Description BBC Radio interview 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact A radio interview where I also spoke about the research carried out through The Kinesemiotic Body project
Year(s) Of Engagement Activity 2021
 
Description Interview with the BBC broadcast on East Midlands Today on 26/12/2023 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact This interview with the BBC was the result of a press release with video launced through media and social media by Loughborough University. It was deemed to be of public interest and broadcast among the news on East Midlands Today. The recording of the broadcast is in the following folder, the link was provided by the BBC: https://www.dropbox.com/t/tU85f6fO6MFR4BOe
Year(s) Of Engagement Activity 2023
 
Description Press release with video 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact Mini-documentary presemted by a press release that appeared on the university news and was distributed to media and social media with an immediate response from the BBC.
Year(s) Of Engagement Activity 2023
URL https://www.lboro.ac.uk/news-events/news/2023/december/ballerinasaresteppingintosenorsuits/
 
Description The Kinesemiotic Body website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Study participants or study members
Results and Impact The Kinesemiotic Body website is still running and reporting on our research activity and outcomes. It is a point of referenc for scholars and information channels.
Year(s) Of Engagement Activity 2021,2022,2023,2024
URL https://www.uni-bremen.de/en/fb-10/forschung/institute/bitt/forschung-und-lehre/multimodalitaetsfors...