Unlocking our Digital Past with Artificial Intelligence (LUSTRE)
Lead Research Organisation:
Loughborough University
Department Name: Communication and Media
Abstract
More and more government data are created in digital form. Emails have replaced letters, PDFs and Word documents have replaced paper memos, and audio/visual files are stored in governmental internal archives and in various systems. Yet just a small proportion of these data is transferred to The National Archives and other archival repositories for long-term preservation, access and use.
The LUSTRE project aims to unlock these data by connecting government professionals with Computer Scientists, Digital Humanists and archivists in cultural heritage organisations. It will focus on the application of Artificial Intelligence (AI) to digital archival records in order to make them more accessible. Indeed, AI can be used for sensitivity review (i.e., to identify sensitive documents in a mass of data), making it possible to release records that are not confidential. AI can also be used to search vast amounts of data. But it is crucial to avoid biases in the selection and processing of data, which could discriminate against certain groups and even impact the collective memory. This requires policy makers to engage with algorithms rather than treating AI as a "black box."
The problem of inaccessible governmental records has become particularly acute following the digital revolution. Rigorous filing systems used to organise paper records. However, these systems are not well adapted to the digital age. In 2017, the report Better Information for Better Government (co-authored by the Cabinet Office and The National Archives) identified issues with the management of born-digital records within government - including poorly organised records, scattered across different systems and almost impossible to search effectively. This lack of organisation leads to difficulties in finding information and giving access to records that users need.
The scale of born-digital records also makes it extremely complicated to search for information, particularly when data are scattered on multiple devices and systems. These data could contain confidential and sensitive materials, including materials that could potentially be useful to terrorists and other adversaries. In order to limit risk, data is often locked away and inaccessible to users - including historians, social scientists, journalists and third sector professionals.
Archives are meant to be used, not locked away. Inaccessible government records lead to a lack of accountability in the short term, and risk impacting the cultural memory in the long term. How can we improve access to government archival records in digital form? The LUSTRE project aims to unlock these data by delivering the following outputs:
_4 lunchtime talks at the Cabinet Office;
_a total of 4 face-to-face workshops, including three workshops in London (Cabinet Office and Science Museum) and one workshop in Belfast (hosted by Public Records Office of Northern Ireland);
_online survey and 50 semi-structured interviews;
_open-access report and journal special issue, including one article co-authored by the PI and postdoc;
_cross-sector network on born-digital archives, connecting government professionals with academics and GLAM professionals.
A website, associated social media, and a dedicated LUSTRE list-serv will help us connect with interested parties - in government, academia, archival institutions and beyond.
The LUSTRE project aims to unlock these data by connecting government professionals with Computer Scientists, Digital Humanists and archivists in cultural heritage organisations. It will focus on the application of Artificial Intelligence (AI) to digital archival records in order to make them more accessible. Indeed, AI can be used for sensitivity review (i.e., to identify sensitive documents in a mass of data), making it possible to release records that are not confidential. AI can also be used to search vast amounts of data. But it is crucial to avoid biases in the selection and processing of data, which could discriminate against certain groups and even impact the collective memory. This requires policy makers to engage with algorithms rather than treating AI as a "black box."
The problem of inaccessible governmental records has become particularly acute following the digital revolution. Rigorous filing systems used to organise paper records. However, these systems are not well adapted to the digital age. In 2017, the report Better Information for Better Government (co-authored by the Cabinet Office and The National Archives) identified issues with the management of born-digital records within government - including poorly organised records, scattered across different systems and almost impossible to search effectively. This lack of organisation leads to difficulties in finding information and giving access to records that users need.
The scale of born-digital records also makes it extremely complicated to search for information, particularly when data are scattered on multiple devices and systems. These data could contain confidential and sensitive materials, including materials that could potentially be useful to terrorists and other adversaries. In order to limit risk, data is often locked away and inaccessible to users - including historians, social scientists, journalists and third sector professionals.
Archives are meant to be used, not locked away. Inaccessible government records lead to a lack of accountability in the short term, and risk impacting the cultural memory in the long term. How can we improve access to government archival records in digital form? The LUSTRE project aims to unlock these data by delivering the following outputs:
_4 lunchtime talks at the Cabinet Office;
_a total of 4 face-to-face workshops, including three workshops in London (Cabinet Office and Science Museum) and one workshop in Belfast (hosted by Public Records Office of Northern Ireland);
_online survey and 50 semi-structured interviews;
_open-access report and journal special issue, including one article co-authored by the PI and postdoc;
_cross-sector network on born-digital archives, connecting government professionals with academics and GLAM professionals.
A website, associated social media, and a dedicated LUSTRE list-serv will help us connect with interested parties - in government, academia, archival institutions and beyond.
Organisations
- Loughborough University (Lead Research Organisation)
- Cabinet Office (Collaboration)
- Archives and Records Association (Collaboration)
- The National Archives (Collaboration)
- Public Record Office of Northern Ireland (Collaboration)
- Science Museum (Collaboration)
- Chartered Institute of Library and Information Professionals (Collaboration)
Publications
Jaillant L
(2024)
Are Users of Digital Archives Ready for the AI Era? Obstacles to the Application of Computational Research Methods and New Opportunities
in Journal on Computing and Cultural Heritage
| Description | The LUSTRE project team has collaborated with the Cabinet Office and other project partners to explore the application of Artificial Intelligence to government archives. We have so far delivered 4 lunchtime talks (in December 2022; March, April and June 2023) and three workshop (in January, May and July 2023) in London and Belfast, as well as 50 interviews with policy makers, industry professionals and academics. A list-serv complements the events organised as part of the project, and offers a platform for the LUSTRE community to receive regular updates. A special issue is forthcoming in AI & Society (full papers due in September 2024) and an open-access report will be published later in 2024. |
| Exploitation Route | LUSTRE will lead to impact in three key ways. Firstly, through its events and final report, the project will contribute to changes in the ways government departments prepare their digital records ahead of transfer to archival institutions. Secondly, LUSTRE will lead to changes in the professional practices of archivists who work with digital archives. Thirdly, the project will unlock digital archival materials, making them accessible to a wide range of general users and researchers across multiple academic disciplines. |
| Sectors | Digital/Communication/Information Technologies (including Software) Education Government Democracy and Justice Culture Heritage Museums and Collections |
| URL | https://lustre-network.net/outputs/ |
| Description | The LUSTRE project has impacted on professionals in government, the GLAM sector (Galleries, Libraries, Archives and Museums) and academia. As part of the project, we organised 4 lunchtime talks and 3 one-day workshops, bringing together 800+ participants in total. The two first workshops were hybrid, organised at the Cabinet Office in London and online. The third workshop was organised by the Public Record Office of Northern Ireland (PRONI) in Belfast. The workshops benefited attendees by putting them in touch with professionals with different experience and expertise on AI applied to digital government records. It also allowed the PI to widen her networks of contacts - leading the Home Office, the Department for Education, the Intellectual Property Office, and the Independent Office for Police Conduct to offer letters of support for further grant applications. We have asked participants, of whom over 75 percent on average are non-academics, to provide feedback on their experience. Many participants commented on the value of bringing together digital humanities, computer scientists, GLAM professionals and government professionals, to explore the application of AI to archives. Since its launch in 2022, the LUSTRE website has had 13,909 views from 6,295 users from the UK, US, Canada, Australia and the Netherlands, with the workshop recordings and slides being the most visited content. A special issue on "When Data Turns into Archives: Making Digital Records More Accessible with AI" was accepted in the journal AI & Society, and we have already received expressions of interest from government professionals, GLAM professionals and academics. Full papers will be submitted by September 2024. An open-access report offering guidelines on AI applied to government archives will be published later in 2024. |
| First Year Of Impact | 2022 |
| Sector | Digital/Communication/Information Technologies (including Software),Government, Democracy and Justice,Culture, Heritage, Museums and Collections |
| Impact Types | Cultural Policy & public services |
| Description | Influence on professional practice of professionals in government and archives |
| Geographic Reach | National |
| Policy Influence Type | Contribution to new or improved professional practice |
| Impact | The project is contributing to improved access to public archives. |
| Description | Partnership with Archives and Records Association (ARA) |
| Organisation | Archives and Records Association |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | Expertise in Digital Humanities and academic project management |
| Collaborator Contribution | Participation in project events |
| Impact | Participation in further grant applications and publications |
| Start Year | 2022 |
| Description | Partnership with Chartered Institute of Library and Information Professionals (CILIP) |
| Organisation | Chartered Institute of Library and Information Professionals |
| Country | United Kingdom |
| Sector | Charity/Non Profit |
| PI Contribution | Expertise in Digital Humanities and academic project management |
| Collaborator Contribution | Participation in project events |
| Impact | Participation in further grant applications and publications |
| Start Year | 2022 |
| Description | Partnership with Public Record Office of Northern Ireland (PRONI) |
| Organisation | Public Record Office of Northern Ireland |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | Expertise in Digital Humanities and academic project management |
| Collaborator Contribution | Participation in project events; Organisation of LUSTRE Workshop 3 |
| Impact | Participation in further grant applications and publications |
| Start Year | 2022 |
| Description | Partnership with Science Museum Group |
| Organisation | Science Museum Group |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | Expertise in Digital Humanities and academic project management |
| Collaborator Contribution | Participation in project events |
| Impact | Participation in further grant applications and publications |
| Start Year | 2022 |
| Description | Partnership with The National Archives UK |
| Organisation | The National Archives |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | Expertise in Digital Humanities and academic project management |
| Collaborator Contribution | Participation in project events (including talk given by John Sheridan, TNA's Digital Director, at LUSTRE Workshop 2). |
| Impact | Participation in further grant applications and publications |
| Start Year | 2022 |
| Description | Partnership with the Cabinet Office |
| Organisation | Cabinet Office |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | Expertise in Digital Humanities and academic project management |
| Collaborator Contribution | Participation in project events; hosted LUSTRE Workshop 1 and 2 in London |
| Impact | Participation in further grant applications and publications |
| Start Year | 2022 |
| Description | Blog Post by Cecilia Ghidotti (LUSTRE Research Associate) reflecting on LUSTRE Workshop 1 and 2 |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | In this blog post, Cecilia Ghidotti (LUSTRE Research Associate) reflected on LUSTRE Workshop 1 and 2, and the impact of these events outside academia. The blog post enabled further engagement with our key audiences outside academia (professionals in archives and government). |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://lustre-network.net/2023/06/06/report-from-lustre-workshops-1-2/ |
| Description | LUSTRE Workshop 1: "AI and Born-Digital Archives: Challenges and Opportunities," Cabinet Office, London, Jan. 2023 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | The workshop was a hybrid event held on 26 January 2023: in-person and hosted by the Cabinet Office at The White Chapel Building in London, and online via MS Teams. Choosing this format allowed us to reach a worldwide audience. Over 220 participants attended the one-day workshop, 187 of whom attended online and 30 in person. Most attendees were from the UK, but we also had participants from the US, Netherlands, Australia, New Zeland, Switzerland, Austria, Canada, Italy, Estonia, Czech Republic, Sweden, Denmark, Finland, Germany, Belgium, Spain, France, Greece, Turkey, Israel, Egypt, United Arab Emirates, Russia, South Africa, India, China, Japan, Singapore, Philippines, Brazil and Trinidad and Tobago. The workshop, the first in the series, delved into the challenges and opportunities that Artificial Intelligence (AI) offers to the management and use of born-digital archives. The workshop which started at 10:30 and ended at 15:30 consisted of three talks followed by Q&A sessions and a roundtable discussion around the topic of AI Ethics for government archives. Speakers explored the topic from different angles in the three talks: "Finding light in dark archives: Using AI to connect context and content in email" by Prof. Stephanie Decker and Dr. Adam Nix from the University of Birmingham; Dr. Jenny Bunn from University College London and The National Archives presented "Putting principle into practice: Transparency, recordkeeping and AI"; and "Searching, fast and slow: rethinking the query builder paradigm" by Dr. Tony Russell-Rose from Goldsmiths University. ***AUDIENCE*** Among those who responded to our survey 33% are professionals in the government sector, 52% are professionals and civil servants in the GLAM sector (Galleries, Libraries, Archives and Museums), and 19% are academics. Participants ranged from different levels of experience, and according to our survey responses 28% of them were early career professionals or academics. When asked if the workshop was useful, 66% of survey participants said it was very useful, 24% said it was useful, and 9% that it was extremely useful. ***ORGANISATION*** We received very positive feedback with regards to the organisation of the event, 57% survey participants thought it was very well organised and 38% well organised. Survey comments confirmed that the workshop and hybrid meeting experience was "excellent." A participant from Australia stated: "I attended online and found the online experience was excellent. Well done on managing a hybrid session so well. I am in Australia so didn't stay online for the whole workshop (too late in the evening) but listened to the first presentation and Q&A, which was very relevant to me. I look forward to reviewing the other presentations when they are available." Similar feedback was given other online attendees in locations such as the US: "I liked that there were three presentations only, with the roundtable (which I would have liked to sit in on as an online guest). This allowed speakers to fully address their topic and have plenty of time to answer questions. The longish breaks were great--they allowed me, watching from home in the US, to check work email or make a cup of tea--that way, there were no distractions as I sat down to enjoy the next presentation." ***CONTENT*** The large attendance and engaged discussions of this workshop revealed the relevance of the project and the need to address the implications of using AI technologies to unlock born-digital government records. Survey respondents highlighted that the workshop introduced "thought provoking," "fantastic" discussions relevant to their practice or research: "Very interesting presentations. I particularly liked the range of perspectives on offer, both from the speakers and the delegates;" "The work by Stephanie Decker and Adam Nix is very informative for my PhD thesis topic;" "Some of the insights and understandings I gained from the workshop will likely inform my decision-making about how and when we consider AI in to help manage our born-digital material." Presentation slides and recordings were uploaded on our website after the event to maximise impact. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://lustre-network.net/event/workshop1-ai-born-digital-archives/ |
| Description | LUSTRE Workshop 2: "AI and Born-Digital Archives in the Government Sector and Beyond," Cabinet Office, London, May 2023 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | This second workshop was a hybrid event held on 4 May 2023: in-person and hosted by the Cabinet Office at The White Chapel Building in London, and online via MS Teams. Choosing this format allowed us to reach a worldwide audience. Over 200 participants attended the 1-day workshop, 180 of whom attended online and 25 in person. Most attendees were from the UK, but we also had participants from Belgium, Canada, Ireland, Israel, Italy, Lebanon, Netherlands, Norway, Poland, Portugal, Sweden and the USA. ***Speakers*** The second LUSTRE workshop continued the discussion on the challenges and opportunities that Artificial Intelligence (AI) offers to the management and use of digital born archives. The workshop which started at 10:30 and ended at 15:30 consisted of five talks followed by Q&A sessions. Five speakers explored the topic from different angles in the five talks. Dr Keegan McBride from the Oxford Internet Institute talked about "Artificial Intelligence in the Public Sector: Separating Myth from Reality," James Lappin (Loughborough University) discussed "The Impact of AI on Records Management." Dr Lise Jaillant, PI of the LUSTRE project, presented a talk on "AI and Archives from a Researcher's Viewpoint." In the afternoon the Digital Director of The National Archives John Sheridan discussed the topic with a talk titled "Navigating the maelstrom (looking for lighthouses)," and finally Professor Jason R. Baron (University of Maryland, USA) presented a talk on the case of the US National Archives and Records Aministration and its implications: "Is NARA Ready Yet? The Newly Extended 2024 Start Date for Accessioning Records into the US National Archives Only in Electronic and Digital Formats, and What That Means." ***Attendance and feedback*** The hybrid format allowed active engagement from both online and in-person attendees. Networking opportunities were given to in-person attendees during different breaks during the day. Among those who responded to our survey, 66% were Professionals and Civil Servants in the GLAM sector (Galleries, Libraries, Archives and Museums), 19% were professionals in the government sector, 19% were from the academic sector (with a 11% who identified as Early Career Academics). When asked how useful the LUSTRE workshop was, 77% said it was "Extremely useful" and "Very Useful," 19% chose "Useful." The event was evaluated as "Very well organised" and "Well organised" by all respondents (for a total of 100%). Survey comments confirmed that the workshop and hybrid meeting experience worked well. A participant stated that the option of joining remotely "made it possible for me to fit attendance at the talks around other work commitments." Others expressed their interest in taking part in future initiatives because they had a positive experience: "I found the hybrid event to work very well and would be interested in participating both online and in person again in the future." The large attendance and engaged discussions of this workshop revealed the relevance of the project and the need to address the implications of using AI technologies to unlock digital records. Survey respondents found the workshop timely and thought provoking: _"It was very illuminating in the sense of born-digital archives and the change to these from paper records, as well as about chat bots and the possible changes to how we work, in the sense of computer programs possibly replacing public servants roles in some instances in the future (scary!)." _"It dealt with 'hot' issues that concerns me and my professional community." They also appreciated the variety of the speakers, the level of their presentation and the engagement with the participants: _"The speakers were very engaging, leading to an interesting discussion amongst the participants and attendees." _"Relevant presentations of good quality, both content and presentation. Active participants with good reflections and questions." _"Relevant content. Concrete knowledge. Good presenters. Good activity in the audience." ***Impact on future work*** There was a consensus that the workshop will have a positive impact on participants' future work. Participants said their organisations are facing challenges related to the use of AI which benefit from the knowledge shared during the workshop and they plan to bring the discussion back to their organisations: _"The workshop content covered some current gov digital archives issues and possible approaches to solving them using AI. My current work is in the digital archives area and we are looking at similar issues. So the workshop will contribute to our internal discussions." _"Yes, compiling a report this year about how AI can be used in our work at the National Museum of the Royal Navy. Notes from this session will provide a great foundation for this." The workshop was useful not only for those who already work with AI but also for those are still exploring the use of AI: _"It challenged me to think about practical ways in which AI can be applied to the work of our team. I am currently working on a feasibility study exploring the use of AI in our organisation." Presentation slides and recordings were uploaded on the LUSTRE website after the event to maximise its impact outside academia. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://lustre-network.net/event/workshop-2-4th-of-may/ |
| Description | LUSTRE Workshop 3: "AI, Archives and Public Sector Recordkeeping," Public Record Office of Northern Ireland (PRONI), Belfast, July 2023 |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | LUSTRE Workshop 3 was an in-person event at the Public Record Office of Northern Ireland (PRONI) in Belfast on 6 July 2023. It continued the discussion on AI applied to born-digital archives and recordkeeping in the public sector. The format of the workshop enabled focused discussions with key stakeholders in the government and archives sector, many of them based in Northern Ireland - a part of the UK often neglected in research projects. ***Invited speakers*** (1) James Lappin (Loughborough University) discussed two alternative pathways for the application of AI for recordkeeping purposes in live systems; (2) Michaela Black (Ulster University) talked about responsible uses of Artificial Intelligence; (3) David Brown (Virtual Treasury of Ireland) presented the perspective of an historian on Artificial Intelligence; (4) Angela Campbell (PRONI) discussed how to use AI to improve Excel skills; (4) Alister Pearson (Information Commissioner's Office) discussed how to apply the data protection principles to the use of information in AI systems. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://lustre-network.net/event/lustre-workshop-3-ai-archives-and-public-sector-recordkeeping/ |
| Description | Lunchtime Talk 1 by Giovanni Colavizza, Cabinet Office, London, Dec. 2022 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Computer scientist Dr Giovanni Colavizza gave a presentation on "Towards AI-supported records management" at the Cabinet Office (December 2022), for an audience of professionals in government and the archives sector. After the talk, presentation slides were uploaded on the LUSTRE website to enable further impact. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://lustre-network.net/event/lunchtime-talk-1-with-giovanni-colavizza-towards-ai-supported-recor... |
| Description | Lunchtime Talk 2 by Mhairi Aitken, online, March 2023 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Dr Mhairi Aitken (Ethics Research Fellow in the Public Policy Programme at The Alan Turing Institute) gave the second LUSTRE lunchtime talk, which reached an audience of c. 100 professionals in government and in libraries/ archives. The title of her talk was "Establishing a Social Licence for AI: Addressing ethical considerations through trustworthy practice." A recording of the presentation was uploaded on our LUSTRE website to enable further impact. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://lustre-network.net/event/lunchtime-talk-2-8th-of-march-2023/ |
| Description | Lunchtime Talk 3 by Rob Bath, online, April 2023 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Rob Bath (Digital Director at IRMS - Information and Records Management Society) gave the third LUSTRE online talk on 5 April 2023. The title of his talk was "AI for Information Governance in Microsoft365." The talk reached c. 120 people, mostly coming from the government and archives sectors. The recording of his talk was uploaded on our website to maximise impact. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://lustre-network.net/event/lunchtime-talk-3-with-robert-bath-on-ai-for-managing-records-in-mic... |
| Description | Lunchtime Talk 4 by Helen McCarthy, online, June 2023 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Professor Helen McCarthy (University of Cambridge) gave the fourth LUSTRE online talk on "Exploring the Digital Record of Everyday Life in Covid-era Britain" in June 2023. The talk reached c. 80 professionals in the GLAM sector, government, and academia. The recording of the talk was then made available on the LUSTRE website. |
| Year(s) Of Engagement Activity | 2023 |
| URL | https://lustre-network.net/event/online-lunchtime-talk-4-with-helen-mccarthy-exploring-the-digital-r... |