ARray - An augmented reality and pre-visualisation tool for the costume industry
Lead Research Organisation:
Guildhall School of Music and Drama
Department Name: Research Department
Abstract
In 2022, the Production Arts Department at the Guildhall School of Music & Drama successfully applied to the AHRC's Creative Capabilities & Research Fund (CResCa), securing over £900,000 to purchase cutting-edge digital equipment, namely SmartStage Pro XR Stage, Holosys Volumetric Capture Suite, and BlackTrax Realtime tracking. This equipment has enabled the Department to develop a credible XR (Extended Reality) studio that includes a high-quality volumetric capture system with fully integrated hardware and software necessary to allow for motion capture of people and objects in 'six degrees of freedom' (6DoF) ready for use in augmented and virtual reality (AR and VR) and virtual production. As a result, the Department has become increasingly active in technology-driven and collaborative practice-based work across the commercial and research sectors. Ongoing work facilitated by CResCa includes 'A Mid-journey Night's Dream', a project that investigates the potential for AI scenography in the Shakespearian canon, using existing mainstream platforms and bespoke machine learning systems to design stages and costumes that are built in both physical and digital processes to create an immersive XR production.
Whilst the benefits of digital technologies that allow broad and rapid ideation, sharing of ideas and the ability to prepare and visualise work for those of us working at the mise en scène end of production arts is clear, the use of digital tools for costume design is, at present, limited. During our work on 'A Mid-journey Night's Dream', for example, the School's costume lecturers have struggled to adapt to digital workflows due to a paucity of digital tools that can adequately reflect the physical demands of costume design e.g. the tactile nature of fabric, the importance of working with human bodies, the influence of lighting, and so on. Where online cataloguing of stage costumes or traditional physical costume archives exist, they tend to be basic and photographic, not capturing the 3D nature of the originals or their dynamic quality when they move. Moreover, whilst there are some similar tools on the market that provide digital solutions for costume design, such as 'Virtual Wardrobe' by Weta Digital (a software tool used to create digital costumes for films) and 'ClosetSpace' by Stylitics (a digital wardrobe organiser for personal use), there is it seems no tool that specifically addresses the needs of costume houses in streamlining the costume selection process and providing a digital platform for collaboration and experimentation. There is, therefore, a commercial need for a digital tool that can successfully integrate established working practices of costume designers with the digital workflows increasingly found elsewhere in the performing arts sector, not to mention other digital spectacle-based entertainments.
This project has been conceived to support activities enhancing self-sustaining commercial impact on the UK's costume design industry arising from work funded by the Guildhall School of Music & Drama's successful application to the AHRC's Creative Research Capabilities Strand 1 (CResCa). Using cutting-edge technologies that enable motion capture of people and objects in 6DoF ready for use in AR and VR, Guildhall School researchers will collaborate with Angels Costumes (a leading UK-based costume design house that boasts the largest collection of theatre, television and film costumes in the world) to develop a prototype subscription-based digital costume library, initially of 100 items. This digital library will enable costume designers to view manipulate costumes virtually , without the need for the physical prototypes on which the costume design industry currently relies. The development of this digital library will address a significant gap in the commercial market - no equivalent resource currently exists - and will enhance the competitiveness of the UK's costume design industry.
Whilst the benefits of digital technologies that allow broad and rapid ideation, sharing of ideas and the ability to prepare and visualise work for those of us working at the mise en scène end of production arts is clear, the use of digital tools for costume design is, at present, limited. During our work on 'A Mid-journey Night's Dream', for example, the School's costume lecturers have struggled to adapt to digital workflows due to a paucity of digital tools that can adequately reflect the physical demands of costume design e.g. the tactile nature of fabric, the importance of working with human bodies, the influence of lighting, and so on. Where online cataloguing of stage costumes or traditional physical costume archives exist, they tend to be basic and photographic, not capturing the 3D nature of the originals or their dynamic quality when they move. Moreover, whilst there are some similar tools on the market that provide digital solutions for costume design, such as 'Virtual Wardrobe' by Weta Digital (a software tool used to create digital costumes for films) and 'ClosetSpace' by Stylitics (a digital wardrobe organiser for personal use), there is it seems no tool that specifically addresses the needs of costume houses in streamlining the costume selection process and providing a digital platform for collaboration and experimentation. There is, therefore, a commercial need for a digital tool that can successfully integrate established working practices of costume designers with the digital workflows increasingly found elsewhere in the performing arts sector, not to mention other digital spectacle-based entertainments.
This project has been conceived to support activities enhancing self-sustaining commercial impact on the UK's costume design industry arising from work funded by the Guildhall School of Music & Drama's successful application to the AHRC's Creative Research Capabilities Strand 1 (CResCa). Using cutting-edge technologies that enable motion capture of people and objects in 6DoF ready for use in AR and VR, Guildhall School researchers will collaborate with Angels Costumes (a leading UK-based costume design house that boasts the largest collection of theatre, television and film costumes in the world) to develop a prototype subscription-based digital costume library, initially of 100 items. This digital library will enable costume designers to view manipulate costumes virtually , without the need for the physical prototypes on which the costume design industry currently relies. The development of this digital library will address a significant gap in the commercial market - no equivalent resource currently exists - and will enhance the competitiveness of the UK's costume design industry.
Publications
Shorten, D
(2024)
'Guildhall Production Studio: ARray - Turning up the volume in the costume department'
in National Centre for Academic and Cultural Exchange
| Title | Prelight3D (ARray) |
| Description | Prototype platform for a digital costume library viewing and editing tool that uses volumetric capture technology to create high-fidelity 3D representations of theatrical costumes that can be viewed, manipulated, and integrated into virtual stage environments. It allows users to visualise garments in motion from multiple angles, experiment with different outfit configurations, simulate stage lighting conditions, and evaluate how costumes will appear in performance contexts-all through an intuitive interface built on the Unreal Engine platform. |
| Type Of Art | Artefact (including digital) |
| Year Produced | 2023 |
| Impact | To date the prototype has only been used for alpha/beta testing with a selection of industry professionals. One additional commercial partnership is in early stages of negotiation. |
| URL | https://guildhallproductionstudio.gsmd.ac.uk/services/__trashed-2/ |
| Description | The project has illustrated that volumetric capture provides a suitable means for sharing images and information concerning costume. Volumetric capture describes a system that produces a digital realisation of an individual and what they are wearing. A viewer can in effect move round the individual and see them from all angles, and you can watch the individual move (and therefore see how the clothes that they are wearing move). This is valuable for costume stores, hire houses, and designers in theatre, film, television and other media and artforms, since those interested can review a costume and its specific features (for example, its flow, its translucent qualities, its apparent texture) without needing to visit the costume house or have costumes transported to view. The qualities of volumetric capture allow for individual characteristics and (we might even say) 'essence' to be preserved. As Shorten and Deere suggest, 'the project has illustrated that it is possible to evolve practices without eroding the analogue, tactile and embodied essence. ARray's digital approach to costume design can streamline some aspects of the production process but also opens up new avenues for creativity, innovation and collaboration.' (Shorten and Deere, 2024) This is in part because the Holosys volumetric capture system represents the actual person and the actual costume, as opposed to an optical- or marker-based tracking system which provides an avatar to which the digital producer then applies a design. |
| Exploitation Route | The findings might be taken forward by costume houses and hire companies who wish to create a digital archive of their costumes, to share with film/theatre/opera directors and other clients. They might also be taken forward by companies seeking to present an inventory of character 'looks' for purposes of game development and other kinds of entertainment outputs, where it is useful for designers to scan through options for inspiration and to help narrow selection. |
| Sectors | Creative Economy Digital/Communication/Information Technologies (including Software) Leisure Activities including Sports Recreation and Tourism Culture Heritage Museums and Collections |
| Title | Combined volumetric capture and motion tracking studio. |
| Description | Motion capture (MoCap) involves recording the movement of objects or individuals using sensors or markers to capture data that can be mapped onto digital models, enabling the animation of these models in a realistic manner. Volumetric capture (VolCap), on the other hand, captures an object or person from multiple angles simultaneously, creating a three-dimensional model that can be viewed from any direction. A studio equipped with both MoCap and VolCap technologies offers a comprehensive suite for producing detailed and lifelike animations, enabling creators to merge precise movement data with fully rendered 3D environments. This dual capability significantly enhances the realism and interactivity of digital content, making it an advanced solution for industries such as film, gaming, and virtual reality. |
| Type Of Technology | New/Improved Technique/Technology |
| Year Produced | 2024 |
| Impact | The studio was only completed in February 2024. Projects commenced in March 2024. |
| Description | AHRC CResCa Cohort event |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Other audiences |
| Results and Impact | AHRC CResCa Cohort event, Guildhall School of Music & Drama, London, 12 November 20-24. Andy Lavender, Provost & Vice Principal (Academic), was one of the facilitators of the event on behalf of Guildhall School. Dan Shorten, Creative Director of Guildhall Production Studio, outlined the School's use of the CResCa infrastructure, including work on the ARray project. The event included a visit to Target3D's studios to see part of the CResCa infrastructure, along with a demo of work in progress on the ARray project and other initiatives. |
| Year(s) Of Engagement Activity | 2024 |
| Description | Launch of Guildhall Production Studio |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Professional Practitioners |
| Results and Impact | Launch of Guildhall Production Studio, Target3D studios, 3 October 2024. The event was hosted by industry partner Target3D (London E1). Andy Lavender and Dan Shorten spoke, and Leslie Deere gave a presentation including information on the ARray project. Visitors were given a guided introduction to the volumetric capture system and set-up. |
| Year(s) Of Engagement Activity | 2024 |