Capturing Stillness: visualisations of dance through motion capture

Lead Research Organisation: Coventry University
Department Name: Art and Design

Abstract

This project aims to bring together motion capture technology with established movement practices which cultivate attention on bodily awareness, and in particular Skinner Releasing Technique (SRT), to explore how sensorial awareness can be captured and what visualisations might emerge. SRT is a pioneering approach to dance which has evolved from the simple principle that when we are letting go of habitual holding patterns we can move more freely, articulately and powerfully. Joan Skinner, the American choreographer, dance improvisation pioneer and former dancer with the Martha Graham and Merce Cunningham companies, created the technique from early experiments in the 1960s. SRT utilizes image-guided floorwork to ease tension and promote an effortless kind of moving, integrated with alignment of the whole self. Tactile exercises (partner 'graphics') are used to give the imagery immediate kinaesthetic effect: spontaneous movement is evoked by imagery and guided movement studies. Motion capture is a generic name for the techniques which obtain motion data from human performers for a wide range of applications including biomechanics, sports science, ergonomics, advertising, TV and 3D computer games. Regarded as a significant component of the film and games industries, motion capture is also incorporated into arts events but it demands a level of expertise to use it effectively that is not readily achieved by artists.

The project will explore how the methods and principles which characterise the practice of SRT and related practices can 'translate' to the production of visualisations for virtual environments, to shed light on contemporary ideas of interface design and display methods, and to understand more about SRT as a movement discipline. The aim is to enhance the interaction experience for audiences to produce new art experiences which are intellectually and socially engaging and can, through a re-engagement with the complexity of the moving body, generate new understandings about our relationship with our own body within the world. The project is therefore unique in that it will map a specific dance practice, embed this mapping within a game engine and test a variety of avatar visualisations in real time allowing audiences to interact and be directly involved in the process. Finally this interaction will be augmented by using these findings to immerse the player/performer into the action to create new performance spaces and experiences.

Planned Impact

The project will benefit a wide range of external users, organisations and industry experts. It builds on longstanding relationships with companies, venues, organisations, arts professionals and science experts: it is a particular strength of the proposal that it is actively supported by technology companies in the commercial private sector, as well as by arts organisations and HEIs. The project will develop these relationships; it will contribute to industry knowhow and profitability and enhance cultural and educational organisations' own public engagement and learning activities; and it is likely also to be of value to engineering, biomechanical and biomedical science. The project's outputs will thus have economic, social, cultural and educational impact in the UK and overseas, both within and beyond the academic community.
Arts/museums and industry/technology partners have a stake - cultural, financial - in ensuring wide public take-up of the project's deliverables, whether through participation in the artistic work or by use/purchase of the technological applications. HEI partners, whose interest in immersive and 3D technologies ranges from visual/media/performing arts to games to biosurgery, also have an interest in testing and trialling research outputs with wider constituencies in their own fields.
The project's cultural, social and educational impact will be delivered through a range of public events. With administrative and marketing support from Coventry University's Institute for Creative Enterprise (ICE) and the University's communications team, the project leader will work in close partnership with UK-based and international venues, individuals and organisations who will exhibit/show work in progress and completed work. Public engagement will be via the exhibition/installation/show itself and via associated events, planned with host organisations in response to their own outreach requirements and including talks, demonstrations and workshops which will give audiences 'hands on' experience of the new technologies. The work in the Dome/Planetarium, in particular, will attract audiences interested in 3D technologies who will encounter and experience dance in different ways, experience interactive art and provide the project with valuable information about user engagement in new environments. Materials generated through the project, particularly those relating to teaching ideas and software developments, will be disseminated as widely as possible in accessible and attractive ways to reach wide audiences.
The technological, economic and educational impact of the project will be through knowledge exchange with partners who have committed resources (expert advice, equipment) to the research programme. Via seminars, workshops, conferences and publications the project will provide outputs to academic researchers across the cited disciplines. These outputs will also have considerable value for training providers, in both the formal education and the commercial sectors, which use animation, performance, programming and 3D immersive technologies. Importantly, the research will support the commercial partners' development of wider uses for data capture technologies and muscle-modelling software, for entertainment, gaming, medical and other applications; and will enable market expansion both in the UK and overseas. The project leader, supported by the University's Business Development Team, will explore both commercial and non-commercial exploitation opportunities in collaboration with partners and other Coventry University faculties and institutes including the Serious Games Institute and the Health Design Technology Institute.

Publications

10 25 50
 
Title MESSENGER 
Description Model Gardeners present a series of new works that subtly place the viewer in a somewhat neurotic state of calm. Gibson/Martelli refine motion within the manipulated moving image to alter our perceptions, building on the idea that movement can solicit sensory awareness. The artists work with video game engines & performance animation to create unique landscape environments. 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2011 
Impact Exhibitions: 2011 Model Gardeners Gallery, London 
URL http://gibsonmartelli.com/Messenger.html
 
Title VISITOR 
Description Inspired by the artists' travels to the snow-driven mountains of the Canadian Rockies VISITOR was developed following research at The Banff Arts Centre.Vermilion Lake comprises a full-scale replica of a trappers cabin housing an interactive virtual environment. A companion moving image piece, where the bears are sleeping, depicts monochromatic imagery of glaciers, forests and frozen lakes. In both works, either a friendly or malevolent force is suggested, evoking the hunter being hunted, the tracker being tracked. Employing techniques used in video games, bringing exterior virtual space into the physical gallery space the exhibition plays with our apprehension of different forms of reality. 'The worlds they create are total simulacrums: there is no separation between the invented & the real, the site & the represented, the local & the imagined. It is a territory that is cohesive & singular in language, yet simultaneously is forever folding in on itself. Their work engages the particular of the site while undermining its place, the original & point of departure become one in a conceptual unravelling.' Richard Ducker - Curator, Fieldgate Gallery 2010 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2011 
Impact Exhibitions 2012 Towner Art Gallery, Eastbourne 2011 Aberystwyth Art Centre, Wales 2011 James Taylor Gallery, London 2011 Djanogly Gallery, Lakeside Arts, Nottingham 2011 Apthorp Gallery, artsdepot, London 
URL http://gibsonmartelli.com/Visitor.html
 
Description 17 dancers from across the globe were motion captured in the Bugatti Lab at Coventry University and motion.lab at Deakin University, Melbourne. Visualisations of their movement data were incorporated into a series of new Augmented Reality and Stereoscopic Virtual Environments for CAVE & Virtual Reality Headset. The project resulted in international exhibitions, conference contributions, multimedia publications, didactic materials and various other events and public engagement activities including:

Palais de Danse I, II & III a series of Research Salons with presentations, discussions and works-in-progress reflecting on the project where experienced dancers involved in the project and members of the public came together to share ideas and thoughts about its outcomes.

The development of MocApp, a mobile App which allows users to download and view sequences of motion capture. By using this tool, the performers can easily view their own sequences without needing access to a high-end pc and specialised software.
Exploitation Route Skinner Release Technique as a driver for developing virtual environments
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Culture, Heritage, Museums and Collections

URL https://gibsonmartelli.com/SpacePlace/2015/07/30/capturing-stillness/
 
Description Development of software and exhibitions
First Year Of Impact 2012
Sector Creative Economy,Digital/Communication/Information Technologies (including Software)
Impact Types Cultural

 
Description Residency at OCADU Toronto Canada 
Organisation Ontario College of Art and Design University
Country Canada 
Sector Academic/University 
PI Contribution Digital Futures Initiatives Programme
Start Year 2013
 
Description 'Figure & Landscape' 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Undergraduate students
Results and Impact Public Talk at OCADU & University of Toronto, Canada

April 1 2013

AHRC Creative Fellowship Research & Past Exhibitions & Artistic / Academic Practice
Year(s) Of Engagement Activity 2013
 
Description 'kinosphir - she's lost control' 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact 'as yet impossible' futurology lecture series Media City / Salford University
Year(s) Of Engagement Activity 2013
 
Description 'kinosphir- she's lost control' 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Computer Arts Society Lecture Series
Year(s) Of Engagement Activity 2013
 
Description Media Design, School New Zealand - Game Development 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Figure & Landscape' - game worlds talk to faculty
Year(s) Of Engagement Activity 2012
 
Description Palais de Danse Salon I 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Local
Primary Audience Professional Practitioners
Results and Impact An evening of discussion and debate - a get together of experienced SRT dancers involved in the Capturing Stillness Project with conversation about recent projects, mocap viewing and relationship of visualisations of dance imagined and real.
Year(s) Of Engagement Activity 2012
 
Description Palais de Danse Salon II 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Capturing Stillness uses performance capture & computer game worlds to create transformative experiences derived from Skinner Releasing Technique and its poetics. For this presentation Ruth Gibson & Bruno Martelli will describe the project's background and context, commenting on their current artistic practice & their future residency at CAFKA/Christie Digital Systems in Canada - where they will be developing immersive stereo 3d environments with the aim of creating a kinaesthetic code.



The event will be in the form of a drop-in Salon, opening up the projects advances in the motion capture studio - including a demonstration of MocApp - an AR iphone motion capture viewer app developed with Alex Woolner (SGI), Daniel Skolvi (motion.lab) & Melbourne based programmer Scott Ashton who has previously worked with media art pioneer Jeffrey Shaw.
Year(s) Of Engagement Activity 2013
 
Description University of Auckland, National Institute of Creative Arts, Speaking Performance Series 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact 'and the stillness is dancing' talk - AHRC research project
Year(s) Of Engagement Activity 2012