The Design Generator

Lead Research Organisation: Birkbeck, University of London
Department Name: Film, Media and Cultural Studies

Abstract

The Design Generator project is a collaboration between the Vasari Research Centre for Art and Technology, the Birkbeck Institute for Data Analytics (BIDA), and the V&A Museum. This public engagement initiative will involve extensive experimentation with current Machine Learning algorithms and Neural Network technology to develop an interactive tool for exploring the styles and characteristics of objects in the V&A's collection. The primary planned outcome of the project will be a "Design Generator" tool that allows members of the public to combine key terms from the V&A's collection categories (periods, styles, materials, techniques, etc.) in order to generate images of new, imagined museum objects. For example, the user of the tool might choose to combine terms such as "art nouveau" and "stained glass," or "surrealism" and "tapestry" to produce new, perhaps even improbable objects. The AI tool aims to encourage creative learning and exploration of the defining characteristics of the key periods, styles and materials around which the design museum is organized.

The project actively seeks the help of the public and will culminate in a public-engagement workshop involving a pilot panel made up of 8 members of the V&A public. The hands-on
workshop will introduce participants to the capabilities of AI technologies and their current and potential use within the arts and the museum sector. The workshop will also provide participants with an opportunity to trial the prototype Design Generator application and provide feedback on its development.

Publications

10 25 50
 
Description The Design Generator was an experiment in using a small museum dataset (the V&A's digitized collection) to train an AI model, in this case a text to image generator. The project used a process of pre-training using an existing publicly available model (ru-DALL-E) and then refining that model using the V&A's much smaller dataset (a process referred to as transfer learning). This experiment proved successful and provides a model for future investigations of generative AI and museum collections. The project also used a method which is considerably more sustainable and less resource intensive than most AI training methods.

This public engagement project also provided an opportunity to introduce members of the public to the potential and possible risks of fast developing AI technologies within the contexts of museums and the cultural sector more widely.
Exploitation Route The AI model/tool we produced is publicly available on the code repository GitHub and can be utilized as a starting point for other projects experimenting with the use of generative AI in the context of museum collections. We will be presenting our project and findings at the upcoming New Directions in Museum Analytics at King's College where we hope to generate interest amongst other museum and data researchers.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections
URL http://www7.bbk.ac.uk/vasari/2022/09/24/the-design-generator/
 
Description As a public engagement project, the Design Generator reached a significant non-academic audience during the 2022 V&A Digital Design Weekend. We engaged with approximately 300-400 members of the museum public at our permanent stand and approximately 70 participants in our 3 dedicated workshops. Encountering the project was, for most members of the public, an introduction to the potential uses of AI technology as tool for engaging with museum collections. The positive public response to the project during the Digital Design Weekend has encouraged V&A Senior Digital Curator Marc Barto to explore the potential of adapting the Design Generator tool in the creation of a public engagement experience connected to an upcoming V&A exhibition. We are currently in discussions about this possibility. The development of the Design Generator has allowed us to successfully bid for an artist in residence grant from the Experimental Humanities Collaborative Network (funded by the Open Society Foundations). New York-based artist Aarati Akkapeddi will be an artist in residence at the Vasari Research Centre in May and June where they will work with the Design Generator and the Vasari's Capabilities for Collections-funded Deep Learning Workstation in creating an AI artwork exploring the V&A collection. https://thephotographersgallery.org.uk/whats-on/akin-aarati-akkapeddi
First Year Of Impact 2022
Sector Creative Economy,Culture, Heritage, Museums and Collections
Impact Types Cultural
 
Description Experimental Humanities Collaborative Network - AI Ethics Teaching Unit (Birkbeck/Al-Quds Bard)
Amount $10,000 (USD)
Organisation Open Society Foundations 
Sector Charity/Non Profit
Country United States
Start 01/2023 
End 04/2023
 
Description Experimental Humanities Collaborative Network - Artist Residency Grant
Amount $9,000 (USD)
Organisation Open Society Foundations 
Sector Charity/Non Profit
Country United States
Start 05/2023 
End 06/2023
 
Description Experimental Humanities Collaborative Network - Generative AI and Pedagogy Workshop
Amount $5,000 (USD)
Organisation Open Society Foundations 
Sector Charity/Non Profit
Country United States
Start 04/2023 
End 04/2023
 
Title Design Generator AI Model/Tool 
Description The Design Generator is an AI tool that allows members of the public to combine key terms from the V&A's collection categories (periods, styles, materials, techniques, etc.) in order to generate images of new, imagined museum objects. For example, the user of the tool might choose to combine terms such as "art nouveau" and "stained glass," or "surrealism" and "tapestry" to produce new, perhaps even improbable objects. The AI tool aims to encourage creative learning and exploration of the defining characteristics of the key periods, styles and materials around which the design museum is organized. The publicly available AI model involves a process of "pre-training" and "transfer learning." The model is pre-trained on the open ruDALL-E text to image generator and refined using the V&A collection data through a process of transfer learning. 
Type Of Material Improvements to research infrastructure 
Year Produced 2022 
Provided To Others? Yes  
Impact The model can be used as a starting model for future experiments in working with generative AI and museum collection data. We are currently exploring the potential of future use. 
URL https://github.com/ale66/VA-design-generator
 
Title Design Generator AI Model/Tool 
Description The Design Generator is an AI tool that allows members of the public to combine key terms from the V&A's collection categories (periods, styles, materials, techniques, etc.) in order to generate images of new, imagined museum objects. For example, the user of the tool might choose to combine terms such as "art nouveau" and "stained glass," or "surrealism" and "tapestry" to produce new, perhaps even improbable objects. The AI tool aims to encourage creative learning and exploration of the defining characteristics of the key periods, styles and materials around which the design museum is organized. The publicly available AI model involves a process of "pre-training" and "transfer learning." The model is pre-trained on the open ruDALL-E text to image generator and refined using the V&A collection data through a process of transfer learning. 
Type Of Material Computer model/algorithm 
Year Produced 2022 
Provided To Others? Yes  
Impact The model can be used as a starting model for future experiments in working with generative AI and museum collection data. We are currently exploring the potential of future use. 
URL https://github.com/ale66/VA-design-generator
 
Description CapCo Public Engagement Project Collaboration 
Organisation Victoria and Albert Museum
Country United Kingdom 
Sector Public 
PI Contribution The Vasari Research Centre equipment Upgrade facilitated a successful collaborative funding bid between the Vasari, the Birkbeck Institute for Data Analytics, and the V&A Museum. The AHRC CapCo Public Engagement award was granted for the completion of a project entitled "The Design Generator" - an extensive experimentation with current Machine Learning algorithms to develop an interactive tool for exploring the styles and characteristics of objects in the V&A's collection. As the Principal Investigator, I coordinated the project, organized the data transfer from the V&A, and co-supervised the work of two hired Research Associates (specialists in machine learning). The final outcome of the Design Generator was an AI model, a text to image generator, trained on selected categories of the V&A collection. The RAs worked in the Vasari Research Centre, making use of the equipment purchased through the Vasari Equipment Upgrade grant, including the newly acquired Deep Learning Workstation. The Design Generator model was publicly showcased at the 2023 V&A Digital Design Weekend (23-25 September) through a permanent stand and a series of three public workshops. We continued to be in discussions with the V&A's senior digital producer Marc Barto on using the Design Generator as a public engagement tool for an upcoming V&A exhibition and a special event on generative AI.
Collaborator Contribution Dr. Alessandro Provetti, the director of the Birkbeck Institute for Data Analytics and Reader in Data Analysis at Birkbeck, provided computer science, machine learning, and data analysis expertise on the project. Dr. Provetti co-supervised the hiring and work of the Research Associates. The V&A Research Institute will provided access to V&A image collections and relevant data. V&A Senior Digital Producer Marc Barto facilitated the inclusion of the Design Generator in the V&A Digital Design Weekend.
Impact The collaboration has resulted in a successful funding bid and a completed public engagement project. The project has produced a publicly available AI model and dataset. The reseach team has submitted a journal article to the journal Leonardo detailing the outcomes of the project (under review). The project will be presented in April through a Birkbeck's Artsweek public engagement workshop entitled "Generative AI and the Museum" and as a presentation at the "New Directions in Museum Analytics" conference at King's College London in May. This is a multi-disciplinary collaboration involving computer science (machine learning), museum collections, and digital media studies.
Start Year 2021
 
Description V&A Digital Design Weekend 2022 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact We introduced our Design Generator project to the museum public during the three day V&A Digital Design Weekend (23-25 September 2022). The Design Generator, a Birkbeck-based public engagement project, is an AI model trained on the V&A collection that automatically generates images of fictional design objects from text prompts. The AI tool aims to encourage creative learning and exploration of the defining characteristics of the key periods, styles and materials around which the V&A's collection is organized.

We operated a permanent interactivev stand showcasing the Design Generator during the three days of the event, engaging with approximately 300 to 400 attendees during the weekend. We also ran three dedicate workshops during the Design Weekend, attended by approximately 70 participants in total. These workshops introduced participants to the wider developments of AI technologies being used in the museum context and allowed them to interact with our Design Generator model.

We collected 27 survey responses from workshop participants, with 90% agreeing that they found the workshop very enjoyable and stimulating. We received a number of useful audience comments regarding the potential of generative AI tools for museum collections such as, "Could be a starting point for innovation in design across all sectors...." and "You can (potentially) see all the different categories of the collection in one place rather than walking around the whole area. Provides different experience from connecting with the archives and museum objects, and could reach a different audience from traditional museum goers."
Year(s) Of Engagement Activity 2022
URL https://www.vam.ac.uk/event/YgbjJRV1V5/design-generator