ARTICT | Art Through the ICT Lens: Big Data Processing Tools to Support the Technical Study, Preservation and Conservation of Old Master Paintings

Lead Research Organisation: University College London
Department Name: Electronic and Electrical Engineering

Abstract

The heritage science sector is experiencing a digital revolution linked to the emergence and increasing adoption of cutting-edge non-invasive analytical imaging techniques generating large volumes of multidimensional data from cultural heritage objects. These include macro X-Ray fluorescence (MA-XRF) scanning and hyperspectral imaging (HSI). Combining MA-XRF and HSI data - providing elemental and molecular information - offers huge potential for improved identification, characterization, and visualisation of the materials and features of interest in a painting, including in sub-surface layers of the painting. However, it is increasingly recognised that the wealth of data associated with these modalities cannot be fully exploited through traditional primarily manual approaches to interrogating heritage science data.

The aim of this research - bringing together ICT and Heritage Science researchers to enable the cross-pollination of ideas and expertise - is to co-create new automatic signal analysis and processing tools that are able to 'fuse' MA-XRF and HSI data to support the technical study, conservation and preservation artwork.

In particular, the proposed tools will provide the means to identify, characterize and visualize materials present within a painting, thereby leading to new insights relevant for the conservation and preservation of Old Master paintings or to new ways to engage the public with cultural heritage, science and ICT. For example, the proposed tools will also help visualise in a more integrated and accessible form the features of interest for art-historical study and conservation, such as underdrawing, pentimenti, concealed designs, losses or non-original materials.

To develop such tools, an ambitious research programme is proposed that includes development of: 1) multidimensional multimodal heritage science datasets, 2) multimodal signal processing algorithms for data correction, alignment, registration and mosaicking; and 3) new multimodal signal analysis algorithms capable of inferring material distributions in a painting from MA-XRF and HSI data. In addition, this research programme also envisions a number of significant advances in the area of signal processing - including new sparsity-driven nonlinear unmixing algorithms - that address the specific challenges arising in art investigation that do not arise in other application domains of signal processing, such as the need to identity mixtures of (aged) materials in superimposed layers using combined MA-XRF and HSI datasets.

The research programme also includes a number of case studies on National Gallery paintings that will assess the validity, potential and relevance of the proposed tools to the wider heritage sector.

The other aim of this research is to champion and sustain co-creation activities across the ICT and Heritage Science sectors. The activities planned during and beyond the project include: a) training of researchers, doctoral students, and undergraduate students; b) dedicated courses, workshops and events; c) outreach and public engagement activities; and d) a UK-wide network in the area of "ICT for Art Investigation". These aim to augment the project's co-creation and cross-disciplinary ethos and catalyse further research.

The ideas and tools conceived throughout the research will lead to impact across various levels and communities: a) the ICT sector will be exposed to new challenges stimulating developments in a new area of signal processing for art investigation; b) the Heritage Science sector will benefit from new automated, accessible, robust, user-friendly tools to aid the work of heritage scientists, art historians, and conservators. Finally, the tools and the images and insights they create will provide galleries with new innovative means to interpret and present their collections to the general public.

Planned Impact

Heritage science plays a key role in promoting access to cultural heritage, its interpretation, conservation and management, and this research will offer a range of new multimodal, multidimensional data analysis and processing tools with potential to lead to direct impact on this sector. Further, in view of culture's importance to UK life and tourism, the research will have indirect impact on the UK's economy and society.

Of particular relevance is the current interest in the cultural heritage sector - stimulated by the emergence of new non-invasive imaging techniques and a desire to minimise removal of physical samples for analysis - in the development of automated data processing/analysis tools to support the technical study, preservation and conservation of artworks. The immediate impact of this research on the activities of the National Gallery (NG) or other cultural heritage institutions in the UK and overseas will be as follows:

1 The proposed tools will enhance the ability of heritage scientists and art historians to study aspects of an artwork's creation, including the artist's palette, materials and technique, the artwork's provenance, or changing appearance over time, influencing how artworks are interpreted and presented.

2 The proposed tools will also enhance the understanding of heritage scientists and conservators of an artwork's conservation history, e.g. characterising non-original materials, areas of damage, etc., impacting on conservation approaches to unique (national) collections.

3 The availability of multimodal data analysis technology offers heritage scientists the means to readily combine, interpret, and re-interrogate the existing and expanding datasets that many cultural heritage institutions hold for their collections, maximising the potential of such bodies of data and the understanding of the associated artworks. Further, the tools also offer the potential to make complex scientific data more accessible by presenting a combined interpretation based on data from multiple modalities in a single image.

The visual outputs from the tools - and insights they will create - also provide innovative new ways to present paintings or other artworks within the cultural heritage sector (e.g. art historians&conservators) and to a wider audience. Via presentation on modern digital devices this could range from new views about an artwork's creation and conservation history to new ways to interact digitally with artwork using augmented reality technology that could be the basis for new exhibitions in museums or galleries. The NG's experience with exhibitions and web features based on technical examination of paintings suggests this highly visual area of applied science is an effective way to engage with the public and can increase public understanding of science and ICT.

The research will also promote new avenues of enquiry in the area of signal processing - notably, "Signal Processing for Art Investigation" - as well as a co-creation and cross-disciplinary culture within the wider area of "ICT for Art Investigation". It is anticipated that by demonstrating the power of ICT and signal processing research to solve real challenges in another sector, this project will influence both new research directions within the ICT community along with further co-created, cross-disciplinary ICT research in the cultural heritage sector, and within the Arts&Humanities more generally.

The reputations of the applicant organisations in their respective communities, and their wide international academic and professional networks, will ensure the impact of this research within the ICT sector, the Heritage Science sectors and with heritage end-users. Of particular relevance here is the NG's involvement in developing the European Research Infrastructure on Heritage Science (E-RIHS, www.e-rihs.eu). The experience of the applicants in public engagement activities will ensure the wider impact of the research is realised

Publications

10 25 50