Augmented feedback to enhance motor and artistic learning during social distancing

Lead Research Organisation: Queen Mary University of London
Department Name: Sch of Biological & Behavioural Sciences

Abstract

Social distancing has produced an indefinite shift to remote teaching. This shift poses unaddressed challenges in delivering classes and providing feedback remotely for physical activities such as performing arts or rehabilitation. Digital technologies can facilitate or enhance learning but little is known about remote motor skills teaching, hence there is an urgent need to identify the best practices for remote teaching of physical and artistic disciplines. The lessons learnt in this project will be transferable to any motor skills teaching, such as rehabilitative exercise or practical skills.
We will build on (a) Queen Mary University of London facilities and expertise including an online teaching platform, experts in psychology, dance medicine, physiotherapy, cognition and learning, artificial intelligence and multimedial signal processing, and (b) a collaboration with established dance partners (e.g. English National Ballet school, Laban Conservatory, FloorBarreTM) and Barts Health NHS Trust.
We will validate the efficacy of different remote teaching methods and tools on physical and artistic training including artificial intelligence augmented feedback. By focusing on remote training for dance we will not only shed light not on a specific popular discipline but on the remote training of a broad range of traits that include physical skills (from flexibility, muscular endurance and power, joint range of motion), psychological state (mental health, concentration, motivation and their relation to this pandemic), artistic development (presence, rhythm, creativity). Human expert assessment will provide immediate results useful to plan teaching, training data for machine learning for behavioural tracking for personalised feedback. Project outcomes include best practice identification and new tools to mitigate social distancing effects on physical, psychological, artistic, and cognitive outcomes. This will be applicable to performing artists, patients with injury, and ultimately the general public.

Publications

10 25 50

publication icon
Färnqvist K (2021) Factors associated with outcome following exercise interventions for Achilles tendinopathy: A systematic review. in Physiotherapy research international : the journal for researchers and clinicians in physical therapy

publication icon
Hebda-Boon A (2023) The impact of instrumented gait analysis on decision-making in the interprofessional management of cerebral palsy: A scoping review. in European journal of paediatric neurology : EJPN : official journal of the European Paediatric Neurology Society

 
Title Enhanced: a dance-enahnced project 
Description Enhanced: A dance-enhanced project", a Prepared Minds Lab production, directed by Elisabetta Versace (Queen Mary University of London): @PreparedLab. The movie is inspired by the ESRC research project "Augmented Feedback to Enhance Motor and Artistic Learning During Social Distancing" awarded to Dr Elisabetta Versace and co-investigators, in collaboration with The English National Ballet School, The Royal Ballet School, The Trinity Laban Conservatoire for Music and Dance, and the composer Philip Glass. In this project, researchers from different disciplines, dancers, choreographers, teachers, physiotherapists and coaches teamed up to provide answers to the need of artists at a difficult time for arts. The artistic outcome s the first choreography danced collaboratively by artists from different vocational dance schools: The English National Ballet School, The Royal Ballet School, The Trinity Laban Conservatoire for Music and Dance. Research outcomes include open source tools that can be used for perfecting dance training in person and remotely, such as MotionPerfection https://www.sciencedirect.com/science/article/pii/S2352711022001650 . More tools are under development by PreparedMindsLab https://www.preparedmindslab.org/ Stay tuned and enjoy this first production! 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2022 
Impact the trailers have been released in 2022, the previous has been displayed at the Festival of communities 2022, the documentary will be released in 2023 to the general public. Teh trailers and documentary contribute to the dissemination of project outcomes, including the closer connection between researchers and artists in their route of research. 
URL https://www.youtube.com/watch?v=86d0z9EsFvI
 
Description We have developed tools for the remote asssessment and feedback of motion-based activities such as dance, sports, rehabilitation.
We have collaborated with dancers, dance teachers, physiotherapists and embedded their feedback in the development of our tools.
Exploitation Route to further develop assessment of motor activities and feedback
Sectors Digital/Communication/Information Technologies (including Software)

Education

Healthcare

Culture

Heritage

Museums and Collections

 
Description We have started to solve a fundamental challenge in the standardisation of assessment of motor artistic activities. We have worked closely with dance institutions (and NHS), identifying mutual needs for development that will be addressed in future projects focused on best practice for motor learning/teaching and mental health for adolescents. This will in turn support outcomes of motor teaching/learning and mental health in adolescents.
First Year Of Impact 2022
Sector Creative Economy,Digital/Communication/Information Technologies (including Software),Education
Impact Types Cultural

Societal

 
Description A best practice guide for managing patellofemoral pain: a synthesis of systematic review, expert clinical and patient values
Amount £30,000 (GBP)
Organisation Private Physiotherapy Educational Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2020 
End 04/2022
 
Description Analysis of hip muscle activation during rehabilitation exercise for individuals with knee cap pain
Amount £44,880 (GBP)
Funding ID M629 
Organisation Rosetrees Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 01/2017 
End 12/2019
 
Description Annual Fund
Amount £0 (GBP)
Funding ID Making Sense of Movement with the Virtual Prof: Development of a Cross-Faculty Set of Online Tutorials (PI) 
Organisation Queen Mary University of London 
Sector Academic/University
Country United Kingdom
Start 08/2014 
End 09/2015
 
Description Augmented feedback to enhance motor and artistic learning during social distancing
Amount £235,667 (GBP)
Funding ID ES/V015354/1 
Organisation Economic and Social Research Council 
Sector Public
Country United Kingdom
Start 01/2021 
End 12/2022
 
Description Barts Chairty: Cap-AI
Amount £100 (GBP)
Funding ID Barts Chairty: Cap-AI 
Organisation Barts Charity 
Sector Charity/Non Profit
Country United Kingdom
Start 02/2018 
End 08/2020
 
Description Bayesian Decision Support for the management of low back pain: Development, exploration and feasibility
Amount £289,760 (GBP)
Organisation Barts Charity 
Sector Charity/Non Profit
Country United Kingdom
Start 08/2021 
End 08/2025
 
Description Collaborate to Innovate
Amount £94,361 (GBP)
Organisation MedCity 
Sector Public
Country United Kingdom
Start 04/2017 
End 05/2018
 
Description EPSRC
Amount £1,923,122 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 06/2017 
End 07/2020
 
Description Efficacy of high volume injections compared to placebo following failure of conservative therapy for Achilles tendinopathy
Amount $574,320 (AUD)
Funding ID APP1164268 
Organisation National Health and Medical Research Council 
Sector Public
Country Australia
Start 01/2019 
End 12/2021
 
Description Exercise Training as a Novel Primary Therapy for Men with Localised Prostate Cancer: The PANTERA Trial (Prostate cAncer Novel ThERApy)
Amount £3,000 (GBP)
Funding ID 17309 
Organisation Cancer Research UK 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2014 
End 03/2017
 
Description Exercise therapy for the treatment of tendinopathies: A mixed methods evidence synthesis
Amount £243,894 (GBP)
Funding ID NIHR129388 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 03/2020 
End 08/2021
 
Description HBLB
Amount £97,119 (GBP)
Organisation Horserace Betting Levy Board 
Sector Charity/Non Profit
Country United Kingdom
Start 04/2016 
End 05/2017
 
Description HEE/NIHR ICA Pre-doctoral Clinical Academic Fellowship 2018
Amount £64,337 (GBP)
Funding ID 385689 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 08/2019 
End 09/2022
 
Description Innovation Fund (EPSRC)
Amount £5,000 (GBP)
Funding ID Musculo-Skeletal Decision Making: Demonstrating Clinical Feasibility 
Organisation Queen Mary University of London 
Sector Academic/University
Country United Kingdom
Start 04/2015 
End 09/2015
 
Description Innovation Vouchers - Path Feel- Validation of hepatic feedback from smart embedded equipment to improve the gait of people with problematic feet
Amount £5,000 (GBP)
Organisation European Commission 
Department European Regional Development Fund (ERDF)
Sector Public
Country Belgium
Start 08/2015 
End 05/2016
 
Description Interdisciplinary Research Links for Medical AI: Management of Musculoskeletal Injury
Amount £196,000 (GBP)
Organisation Newton Fund 
Sector Public
Country United Kingdom
Start 02/2018 
End 02/2020
 
Description Meeting award
Amount £2,000 (GBP)
Organisation Company of Biologists 
Sector Charity/Non Profit
Country United Kingdom
Start 03/2016 
End 04/2016
 
Description NIHR Programme Grant
Amount £15,000 (GBP)
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 08/2018 
End 09/2023
 
Description PMG
Amount £0 (GBP)
Funding ID PMG - Understanding Propulsive Shoulder Forces - Can We Seat Wheelchair Users Better? 
Organisation PMG Research 
Sector Private
Country United States
Start 08/2015 
End 09/2016
 
Description PPEF
Amount £30,000 (GBP)
Organisation Private Physiotherapy Educational Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 08/2015 
End 09/2016
 
Description PPEF
Amount £30,000 (GBP)
Organisation Private Physiotherapy Educational Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 08/2015 
End 09/2016
 
Description Participatory research QMUL - Research England
Amount £10,000 (GBP)
Organisation Queen Mary University of London 
Sector Academic/University
Country United Kingdom
Start 05/2023 
End 07/2023
 
Description PhD Studentship Scheme
Amount £75,000 (GBP)
Funding ID Halime Gulle 
Organisation Government of Turkey 
Sector Public
Country Turkey
Start 08/2017 
End 09/2021
 
Description PhD Studentship Scheme
Amount £75,000 (GBP)
Funding ID Mehmet Delen 
Organisation Government of Turkey 
Sector Public
Country Turkey
Start 08/2017 
End 09/2021
 
Description PhD Studentship Scheme
Amount £75,000 (GBP)
Funding ID Abdulhamit Tayfur 
Organisation Government of Turkey 
Sector Public
Country Turkey
Start 08/2017 
End 09/2021
 
Description Preparing QMUL student team for Cybathlon 2020 bionic Olympics
Amount £15 (GBP)
Funding ID Westfield fund WF1728 
Organisation Grant Westfield 
Sector Private
Country United Kingdom
Start 01/2018 
End 01/2019
 
Description Principal's Collaborative Studentship with Mathematics
Amount £18,000 (GBP)
Funding ID Functional Data Analysis in Orthogonal Designs with Applications to Gait Patterns 
Organisation Queen Mary University of London 
Sector Academic/University
Country United Kingdom
Start 01/2014 
End 01/2018
 
Description QM impact fund (IAA) I-Move engaging tools for motor learning and musculoskeletal rehabilitation
Amount £50,000 (GBP)
Organisation Queen Mary University of London 
Sector Academic/University
Country United Kingdom
Start 06/2022 
End 01/2024
 
Description QMUL Annual Fund
Amount £5,000 (GBP)
Funding ID GAMERS 
Organisation Queen Mary University of London 
Sector Academic/University
Country United Kingdom
Start 08/2017 
End 09/2018
 
Description QMUL Innovation fund (EPSRC)
Amount £10,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 04/2015 
End 09/2015
 
Description Rosetrees
Amount £30,000 (GBP)
Organisation Rosetrees Trust 
Sector Charity/Non Profit
Country United Kingdom
Start 01/2017 
End 06/2018
 
Description Royal Society Leverhulme Trust fellowship SRF\R1\21000155
Amount £65,000 (GBP)
Funding ID SRF\R1\21000155 
Organisation The Royal Society 
Sector Charity/Non Profit
Country United Kingdom
Start 08/2021 
End 10/2023
 
Description Scheme A1
Amount £14,917 (GBP)
Organisation Private Physiotherapy Educational Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 11/2014 
End 11/2015
 
Description Scheme A2
Amount £29,544 (GBP)
Organisation Private Physiotherapy Educational Foundation 
Sector Charity/Non Profit
Country United Kingdom
Start 11/2014 
End 11/2015
 
Description Supported exercise TrAining for Men with prostate caNcer on Androgen deprivation therapy - STAMINA
Amount £2,497,723 (GBP)
Funding ID Rp-PG-1016-20007 
Organisation National Institute of Health Research and Development (NIHRD) 
Sector Public
Country Indonesia
Start 08/2018 
End 09/2023
 
Description Tendinopathy: Treatment Effects and Mechanism (TEAM)
Amount £249,000 (GBP)
Funding ID CAT SCL-2013-04-003 
Organisation HEE/NIHR Integrated Clinical Academic Programme 
Sector Public
Country United Kingdom
Start 03/2014 
End 04/2019
 
Description Tendinopathy: Treatment Effects and Mechanisms (TEAM) CRN Support
Amount £60,000 (GBP)
Funding ID CAT SCL-2013-04-003 
Organisation HEE/NIHR Integrated Clinical Academic Programme 
Sector Public
Country United Kingdom
Start 03/2014 
End 04/2019
 
Description Tendinopathy: Treatment effects and mechanisms (Tendinopathy TEAM).
Amount £275,068 (GBP)
Funding ID CAT SCL-2013-04-003 
Organisation National Institute for Health Research 
Sector Public
Country United Kingdom
Start 03/2014 
End 03/2019
 
Description Uclp biobank
Amount £0 (GBP)
Funding ID UCLP Biobank Tendinopathy Genetics 
Organisation University College London 
Sector Academic/University
Country United Kingdom
Start 08/2015 
End 09/2016
 
Description WF1728
Amount £15,000 (GBP)
Funding ID WF1728 
Organisation Queen Mary University of London 
Sector Academic/University
Country United Kingdom
Start 01/2018 
End 01/2019
 
Title MotionPerfection 
Description Instructional experts and practitioners in sports, dance and rehabilitation need to assess and provide feedback on motor practice. To visualise, analyse and annotate motor practice, we have developed the user-friendly application MotionPerfection. Using just a single camera video recordings and a computer, MotionPerfection enables users to compare a focal user with a model reference, to assess and annotate the performance. We illustrate the software structure and functionality from video acquisition to feedback annotation. Researchers, instructors, clinicians and practitioners can use MotionPerfection to simplify, automate and standardise the analysis and feedback of motor routines in remote and face-to-face settings, in the laboratory and in the field, without the need of specialised equipment. 
Type Of Material Computer model/algorithm 
Year Produced 2022 
Provided To Others? Yes  
Impact dancers and dance teachers and dance schools, as well as physotehrapists and sports specialists can use this model for their practice 
URL https://www.sciencedirect.com/science/article/pii/S2352711022001650
 
Description Collaboration with English National Ballet School 
Organisation Royal Ballet School
Country United Kingdom 
Sector Academic/University 
PI Contribution we have provided expertise in psychology, dance science, dance medicine, digital technologies, AI experts in ballet
Collaborator Contribution experts in ballet, helped with recruitment, provided feedback
Impact recruitment of participants, feedback
Start Year 2021
 
Description Collaboration with Royal Ballet School 
Organisation Royal Ballet School
Country United Kingdom 
Sector Academic/University 
PI Contribution dance science, psychology, digital technologies, AI
Collaborator Contribution expertise in ballet, help with recruitment, feedback
Impact participant recruitment, feedback
Start Year 2021
 
Description Collaboration with Trinity Laban Conservatoire for music and dance 
Organisation Trinity Laban Conservatoire of Music and Dance
Country United Kingdom 
Sector Academic/University 
PI Contribution expertise in dance science, dance medicine, psychology, gitial technology, AI
Collaborator Contribution expertise in dance and dance teaching, feedback , help with recruitment
Impact recruitment, app development
Start Year 2021
 
Description Professorial secondment to University of Canberra 
Organisation University of Canberra
Country Australia 
Sector Academic/University 
PI Contribution only external at research retreat
Collaborator Contribution Organised research retreat
Impact Two papers One visit One more planned
Start Year 2022
 
Title MotionPerfection 
Description Instructional experts and practitioners in sports, dance and rehabilitation need to assess and provide feedback on motor practice. To visualise, analyse and annotate motor practice, we have developed the user-friendly application MotionPerfection. Using just a single camera video recordings and a computer, MotionPerfection enables users to compare a focal user with a model reference, to assess and annotate the performance. We illustrate the software structure and functionality from video acquisition to feedback annotation. Researchers, instructors, clinicians and practitioners can use MotionPerfection to simplify, automate and standardise the analysis and feedback of motor routines in remote and face-to-face settings, in the laboratory and in the field, without the need of specialised equipment. 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact use for motion assessment in dance, sports, rehabilitation 
URL https://www.sciencedirect.com/science/article/pii/S2352711022001650
 
Title weMove 
Description web app for motor assessment and feedback 
Type Of Technology Webtool/Application 
Year Produced 2023 
Impact professionals and users can use weMOve for their learning and teaching and reflections 
URL https://www.wemovetogether.uk/
 
Description Conversations event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Conversation between scientists and choreographer
Year(s) Of Engagement Activity 2021
URL https://conversations.qmul.ac.uk/conversations-week/conversing-bodies-in-motion/
 
Description Festival communities 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Public/other audiences
Results and Impact Festival of communities with a workshop to use digital tools for dance learning and teaching
Year(s) Of Engagement Activity 2022
 
Description PPI Workshop - weMove for fitness (6 December 2023) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Patients, carers and/or patient groups
Results and Impact promote the augmented feedback app and use of digital techonlogies for sports
Get feedback from users
Year(s) Of Engagement Activity 2023
 
Description Presentation to Dissecting Aerial conference 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact workshop on weMove digital app
Year(s) Of Engagement Activity 2024
URL https://www.gravity-levity.net/events/dissecting-aerial-2024-march-23rd-24th/
 
Description Workshop warmup 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Schools
Results and Impact Workshop with dancers related to best èractice in warmup and our project
Year(s) Of Engagement Activity 2021