Unlocking our Digital Past with Artificial Intelligence (LUSTRE)

More and more government data are created in digital form. Emails have replaced letters, PDFs and Word documents have replaced paper memos, and audio/visual files are stored in governmental internal archives and in various systems. Yet just a small proportion of these data is transferred to The National Archives and other archival repositories for long-term preservation, access and use.

The LUSTRE project aims to unlock these data by connecting government professionals with Computer Scientists, Digital Humanists and archivists in cultural heritage organisations. It will focus on the application of Artificial Intelligence (AI) to digital archival records in order to make them more accessible. Indeed, AI can be used for sensitivity review (i.e., to identify sensitive documents in a mass of data), making it possible to release records that are not confidential. AI can also be used to search vast amounts of data. But it is crucial to avoid biases in the selection and processing of data, which could discriminate against certain groups and even impact the collective memory. This requires policy makers to engage with algorithms rather than treating AI as a "black box."

The problem of inaccessible governmental records has become particularly acute following the digital revolution. Rigorous filing systems used to organise paper records. However, these systems are not well adapted to the digital age. In 2017, the report Better Information for Better Government (co-authored by the Cabinet Office and The National Archives) identified issues with the management of born-digital records within government - including poorly organised records, scattered across different systems and almost impossible to search effectively. This lack of organisation leads to difficulties in finding information and giving access to records that users need.

The scale of born-digital records also makes it extremely complicated to search for information, particularly when data are scattered on multiple devices and systems. These data could contain confidential and sensitive materials, including materials that could potentially be useful to terrorists and other adversaries. In order to limit risk, data is often locked away and inaccessible to users - including historians, social scientists, journalists and third sector professionals.

Archives are meant to be used, not locked away. Inaccessible government records lead to a lack of accountability in the short term, and risk impacting the cultural memory in the long term. How can we improve access to government archival records in digital form? The LUSTRE project aims to unlock these data by delivering the following outputs:
_4 lunchtime talks at the Cabinet Office;
_a total of 4 face-to-face workshops, including three workshops in London (Cabinet Office and Science Museum) and one workshop in Belfast (hosted by Public Records Office of Northern Ireland);
_online survey and 50 semi-structured interviews;
_open-access report and journal special issue, including one article co-authored by the PI and postdoc;
_cross-sector network on born-digital archives, connecting government professionals with academics and GLAM professionals.

A website, associated social media, and a dedicated LUSTRE list-serv will help us connect with interested parties - in government, academia, archival institutions and beyond.