The Matrix: connecting and re-using digital records and archives of archaeological investigations

Lead Research Organisation: Historic Bldgs & Mnts Commis for England
Department Name: Research Group

Abstract

This project will investigate how digital data from archaeological excavations can be made more useful and interesting to a range of users and audiences. It will produce a plan and methods to get such data more consistently recorded, analysed, disseminated and archived in a way that is Findable, Accessible, Interoperable and Re-useable (FAIR).

The project has four objectives 1) Digital Standards; 2) Heritage Data; 3) Stratigraphy Standards; 4) Search Tools; that aim to address 2 research areas:
1. How can we encourage the sharing, linking and interoperability of archaeological data and information, particularly information derived from the commercial sector in order to maximise public value and enhance the research potential of archaeological data?
2. How can we ensure the consistent development, application and enforcement of existing technical information and data
standards and their promotion to others?

Over the last 20 years the Heritage sector and others have concentrated on how the digital data created and stored on computers can be preserved to the same degree that museum objects, like the Rosetta Stone in the British Museum, can be kept for the benefit of future generations. This interest in digital preservation has been especially strong in the archaeological world where excavation data sets are being increasingly gathered as "born digital" data, using the latest computer technologies, i.e. data created on, and only preservable on a computer. Archaeologists, are particularly concerned to make sure that digital records of excavations are safely digitally archived and preserved for future research, because an archaeological excavation cannot be repeated. This focus has helped address the principle concerns about "how do we keep this stuff digitally". But although there are now well established digital archives, such as The Archaeology Data Service, their next challenge is whether the data are preserved and accessible in a way that makes that data most useful to others beyond the archives and the archivists?

This project will address the current problems caused by the lack of standardized approaches to digital archiving of archaeological data using the particular case study of stratigraphic and phasing data. Stratigraphic data form the backbone of all the related archaeological records from each excavated site and are essential for integrated analysis, wider synthesis and accessible archiving of the growing body of archaeological data and reports generated through the commercial archaeological sector in the UK and internationally. The stratigraphic record, usually in the form of a stratigraphic matrix, with associated relationships and data, acts as a primary, if not the primary piece of 'Evidence' for how, and in what order, the site was excavated. As such the stratigraphic matrix is the key mechanism that enables anyone less familiar with the site, to re-visit the excavation records, understand what data is most relevant for any particular research questions, or problems encountered, and piece together the underlying details of how the interpretations by the excavator(s) were actually arrived at. However, such records are often only held on paper or scanned copies of matrix diagrams that cannot easily be re-used with associated data. Often the key phasing data needed for synthesis work and interpretive understanding is not well documented or archived consistently, if at all, in written reports. This results in key records being unsearchable or remaining unconnected to other data and at best usually requires lengthy and wasteful re-keying if any one wishes to work with the archives from such sites. The focus of digital archives and museums is now switching from simply providing better access to digital archives, to questions of how are users in commercial units, curatorial organizations and academia, along with the general public, going to make best use of this growing body of digital information and data.

Planned Impact

One major group who will benefit from this research will be the commercial archaeology contractors searching for and working with digital archives. The importance of this group has been recently recognized in responses to the Mendoza review of Museums and Archives (Mendoza 2017): "This development of the archaeological profession has effectively removed from the taxpayer the burden of funding archaeological recording in advance of development, as was the case prior to 1990, while ensuring important archaeological information and finds are safeguarded and new historical stories contribute to our national identity. The commercial archaeological sector therefore provides a significant benefit to the public". In the last 20-30 years there has been a huge growth in the amount of archaeological information resulting from the requirement in the UK for archaeological work to be undertaken in response to planning applications and the development control process. As such the results of The Matrix project will improve the outcomes from the planning process, help inform policy development, and enable a more targeted approach to understanding and identification of new research topics.

As a result there is a growing body of archaeological reports (via OASIS and Historic Environment Records) but a much less consistent approach to digital archiving of the data that is associated with those reports. However due to the dispersed nature of commercial archaeological work it is not within the remit of any single commercial organization to address these issues at a national scale. There has been a growing concern at a professional level, e.g. the Chartered Institute for Archaeologists, about the lack of a standardized approach to digital archiving of the results of archaeological project work. For commercial bodies that will be digitally archiving data it will enable them to build consistent data management planning for stratigraphic and related data into their organisational workflows. This would streamline their project management activities and thus should improve the actual use and accessibility of the data thereby providing potential cost savings on how they manage their data across multiple projects.

Another group to benefit from the project outcomes will be independent researchers, such as those who work in local society archaeological groups who are seeking information about archaeological projects. Independent researchers will benefit from the improved sign-posting to information available in the archives. In addition should the local societies be undertaking any fieldwork investigations they will be in a position to archive any results in the most accessible manner for others. The stratigraphic data in particular needs to be consistently archived to enable researchers in the commercial organisations or local society groups that may be undertaking to investigate new sites, to find information that is currently inaccessible, about work undertaken by other organisations in the vicinity.

A number of different projects have been developed in the last 10 years to try and synthesize the results of multiple commercial investigations, such as The Roman Rural Settlement project (Fulford ). One major issue identified in such work was a lack of standardized approaches to archiving of the stratigraphic records. For those wanting to synthesize data from across different excavations, it is critical to be able to search across and compare data from similar types of stratigraphic features and phases. The Fellowship will identify ways to improve the better standardization for structuring data, will making it much more feasible for results from different sites to be brought together and aid those who collate data from multiple interventions, enabling new analytical methods to be applied and interpretations derived from extant data.

Publications

10 25 50
 
Description The Matrix project has successfully established the archaeological disciplines additional requirements for guidance on digital archiving and re-use of stratigraphic data from archaeological excavation projects. A new high-level process model for archaeological stratigraphic analysis has been devised, reflecting common practice in archaeological data analysis and data-flows. The process model has been effectively 'road-tested' during consultations, online workshops and seminars with a cross-section of key archaeological practitioners and managers from across the archaeological sector, including major archaeological contracting organizations working on development control sites and those undertaking research funded excavations in universities. A number of fruitful workshops have been held with archaeological practitioners to review and improve the process modelling and the prototype software.
A project website has been created at https://stratigraphic.github.io/matrix/ and this will continue to be used after the lifetime of the project to promote outputs from the project and pursue further research opportunities.
The "Phaser" prototype stratigraphic online research tool has been developed. https://stratigraphic.github.io/phaser-app/ and this will continue to be used after the lifetime of the project to promote outputs from the project and pursue further research opportunities with a consortium of archaeological practitioners (see AG2GP-Handbook project mentioned below and at https://gtr.ukri.org/projects?ref=AH%2FX006735%2F1 ).

The project has identified significant use cases, and made recommendations, for better sign-posting of specific content in archaeological digital archives to help potential researchers find useful data for re-use.

The project has developed an innovative enhanced matrix analysis methodology to enable the improved analysis of explicit spatiotemporal and implicit temporal relationships in the digital stratigraphic records of archaeological investigations. The Matrix methods have shown how the Harris Matrix can be extended to incorporate additional temporal relationships (e.g. overlaps in time) and phasing relationships (e.g. occurs during; meets in time) that incorporates Allen temporal operators (Allen 1984), which are already implicitly held, but not explicitly represented, in the stratigraphic records (matrix visualization diagrams) created by archaeologists. This will help improve the accuracy of Bayesian chronological modelling techniques and tools along with increasing the interoperability and synthesis of archaeological data.

Following consultations and workshops held during the project with key practitioners to enable incorporation of best practice, the development of the prototype online digital tool supports current uses and processes for recording stratigraphic phases & matrix analysis of digital records. In particular, the prototype tool "Phaser" (https://stratigraphic.github.io/phaser-app/ ) enhances the analysis of related dating evidence alongside primary stratigraphic data. It thereby aids overall synthesis and interpretation of the archaeological data and, if used effectively, it should improve the process for digital publication and enhance the quality of stratigraphic data from archaeological investigations deposited in archives.

A 'Follow-on Funding for Impact and Engagement' project has been UKRI grant funded that will develop an online Handbook to encapsulate best practice recommendations and some key outcomes from The Matrix project. The online handbook aims to enshrine and share best practice and thereby improve good practice for stratigraphic analysis across the archaeological discipline. Through a consortium of archaeological practitioners, the AG2GP-Handbook project will further investigate and aim to deliver a sustainable plan for tackling several of the additional challenges identified by The Matrix project R&D.
Exploitation Route Further work could be undertaken with stakeholders across the Heritage sector, and particularly the major contracting archaeological organizations, to develop shared good practice documentation in the form of an online handbook. This online resource could be piloted initially for stratigraphic analysis and related practices but could potentially be developed for a wider set of related post-ex analysis practices, including submission and format of specialist analysis data. This in turn would help improve the practices for sharing, interoperability and re-use (FAIRness) of data deposited in resulting archaeological archives.

The final project workshop showed that participants felt the project had delivered highly useful and innovative software for stratigraphic analysis of key archaeological matrix data by practitioners. The next challenge was to consider how such software, if it was to be made most cost-effective for longer-term usage, could be made sustainable by a consortium of archaeological practitioners and organizations across the discipline.

Further work could be undertaken in the archaeological discipline to exploit the advances made possible by the successful development in The Matrix project of an enhanced temporal representation methodology. The Phaser stratigraphic analysis software tool demonstrates the practical implementation of an analytical extension of the Harris Matrix method. This method explicitly uses the Allen temporal relationships (Allen 1983) in a new method of data analysis for archaeology. This opens further research possibilities for improving stratigraphic analysis and related chronological reasoning, including better treatment of uncertainty in archaeological dating evidence, potentially using Bayesian chronological modelling methods. There is potential to show further cost-benefits in the explicit recording and analysis by commercial and academic archaeological practitioners, of the additional Allen temporal operators which are already implicitly held, but not explicitly represented or documented, in the stratigraphic records currently created by archaeologists.
Sectors Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections

URL https://stratigraphic.github.io/matrix/
 
Description As a result of The Matrix project it has been recommended that any DMP produced for an archaeological project should be deposited as part of the digital archive and the most recent upgrade to the OASIS system (https://oasis.ac.uk/ ) for reporting archaeological investigations in England and across the UK, now includes the functionality for the DMP to be included as part of the OASIS submission. The adoption and promotion of Data Management Planning is still in its relative infancy amongst archaeological practitioners. DMPs are currently only produced for archaeological research projects that are undertaken as part of a research project funded by the UKRI research bodies. The Chartered Institute for Archaeologists has recommended that "archaeological projects should include a DMP as part of the archaeological project's WSI or project design, and then maintained throughout project delivery". The Matrix project has highlighted that there are additional Data Management Planning requirements for activities undertaken at different stages of an archaeological project. The project has been useful in providing a number of case studies for the SEADDA EU funded COST action on reuse of archaeological data, especially Working Group 4 on "Use and Reuse of archaeological data", see https://www.seadda.eu/?page_id=1103 . The project has also increased the interdisciplinary cooperation between archaeologists and scientific chronological modellers through collaborative work with Sheffield University and Prof Caitlin Bucks pioneering research on Bayesian Chronological Modelling.
First Year Of Impact 2022
Sector Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections
Impact Types Cultural,Policy & public services

 
Description Heritage Information Access Strategy (HIAS) policy updates
Geographic Reach National 
Policy Influence Type Contribution to new or improved professional practice
Impact Recommendation that Data Management Plans be a requirement in all grant aided projects funded by Historic England that undertake data creation according to FAIR principles.
URL https://historicengland.org.uk/research/support-and-collaboration/heritage-information-access-simpli...
 
Description The Archaeologist's Guide To Good Practice - Handbook
Amount £80,277 (GBP)
Funding ID AH/X006735/1 
Organisation Arts & Humanities Research Council (AHRC) 
Sector Public
Country United Kingdom
Start 05/2023 
End 04/2024
 
Title "Phaser" Archaeological Stratigraphic Analysis software tool 
Description The Phaser prototype enables creation, editing, analysis, import and export of archaeological stratigraphy data. Traditional stratigraphic matrices can be implemented as directed graph structures - the archaeological contexts being the vertices (nodes) of the graph, and the stratigraphic relationships (above, below, equals) being typed directional arcs (edges) between nodes, with a cardinality of many-to-many. The prototype produced is an open-source responsive single page application written using the Vue reactive framework, with no server-side dependencies. This allows the PHASER application to run on a wide variety of devices using a modern web browser, with no additional installation or configuration required on the part of the user. Hosting of the application (https://stratigraphic.github.io/phaser-app/) and associated documentation is via a GitHub Pages site (https://github.com/stratigraphic/phaser-app), facilitating the ongoing availability of a documented working deployment of the prototype to persist beyond the official end of the funded project. 
Type Of Material Improvements to research infrastructure 
Year Produced 2023 
Provided To Others? Yes  
Impact The PHASER prototype enables creation, editing, analysis, import and export of archaeological stratigraphy data. One immediate positive outcome from the Phaser development was that it enabled the import of legacy data (such as XSM10 data originally in Stratify .LST format on ADS) and subsequent export of that data in .CSV & JSON format for sustainability of these digitally archived data on the Archaeology Data Service. 
URL https://stratigraphic.github.io/phaser-app/
 
Title Phaser - archaeological spatiotemporal and temporal data analysis method. 
Description The application uses a combination of the entered data for stratigraphic relationships between contexts and the entered hierarchical grouping information (interpretation) to derive additional stratigraphic data relationships between contexts and groups, or between groups (including construction of a Group Matrix diagram). The inherited year range data for these elements can then be used to derive temporal relationships between relevant entities, to indicate alignment or discrepancy with the recorded stratigraphy. The cross-checking that is enabled between the stratigraphic relationships and the temporal dating evidence highlights where there is key evidence from the finds data to focus attention on and it highlights where there is greater evidence for the chronological sequencing. 
Type Of Material Data analysis technique 
Year Produced 2023 
Provided To Others? Yes  
Impact Adding the Allen operators to express the complexity of the temporal relationships at a human interaction level has added a considerable degree of additional information for the archaeological user to work with. This enables the archaeologist to see more explicitly where there may be discrepancies between the stratigraphic relationships recorded for stratigraphic units and the dating evidence recorded for the objects retrieved from those stratigraphic units. We found, as we developed the Phaser application, that this additional information, although highly useful for checking dating correlations, needed some management through the user interface, to enable the nature of any discrepancies to be mediated and highlighted most helpfully to the user. 
URL https://github.com/stratigraphic/phaser-app
 
Description Museum of London Archaeology Research Department 
Organisation Museum of London Archaeology
Country United Kingdom 
Sector Charity/Non Profit 
PI Contribution Matrix project team (Keith May & James Taylor) presented an online research seminar for the Matrix project with around 40 MOLA archaeological staff, giving early feedback on project progress.
Collaborator Contribution Museum of London Archaeology (MOLA) made some datasets and internal documentation available and hosted an online research seminar for the Matrix project with around 40 MOLA archaeological staff, giving early feedback on project progress.
Impact A video of the research seminar was recorded by the MOLA partner and subsequently made available to the project with agreement to publish as an online video on the Matrix project YouTube site - https://www.youtube.com/watch?v=ffYAx60ILoU
Start Year 2021
 
Description SEADDA - Working Group 4: Use and Re-Use of Archaeological Data 
Organisation University of York
Department Archaeology Data Service (ADS)
Country United Kingdom 
Sector Academic/University 
PI Contribution I act as a co-chair of the Working Group 4 for SEADDA COST action based on my experiences from re-use of archaeological stratigraphic and chronological data in the Matrix project. I am therefore also on the SEADDA project management group. Presented initial findings from The Matrix to SEADDA workshop on "Practical aspects of the use and re-use of archaeological data" for International archaeologists in academic and commercial archaeology. Presentation on "Archaeological fieldwork and re-use" available. Provided Case Study material for SEADDA WG4 investigations of developing standards documentation for measuring quantitative and qualitative re-use of digital archaeological data, particularly for stratigraphic and chronological digital data.
Collaborator Contribution ADS are the lead body for the SEADDA EU funded COST action. This COST Action will foster knowledge exchange around innovation within the domain, using the FAIR principles as an exploratory framework for innovation, and connecting with relevant projects
Impact Organise an exploratory workshop on current challenges around the use and re-use of archaeological data Organise a discussion workshop on optimising archaeological data for re-use.
Start Year 2021
 
Description SEADDA - Working Group 4: Use and Re-Use of Archaeological Data 
Organisation University of York
Department Archaeology Data Service (ADS)
Country United Kingdom 
Sector Academic/University 
PI Contribution I act as a co-chair of the Working Group 4 for SEADDA COST action based on my experiences from re-use of archaeological stratigraphic and chronological data in the Matrix project. I am therefore also on the SEADDA project management group. Presented initial findings from The Matrix to SEADDA workshop on "Practical aspects of the use and re-use of archaeological data" for International archaeologists in academic and commercial archaeology. Presentation on "Archaeological fieldwork and re-use" available. Provided Case Study material for SEADDA WG4 investigations of developing standards documentation for measuring quantitative and qualitative re-use of digital archaeological data, particularly for stratigraphic and chronological digital data.
Collaborator Contribution ADS are the lead body for the SEADDA EU funded COST action. This COST Action will foster knowledge exchange around innovation within the domain, using the FAIR principles as an exploratory framework for innovation, and connecting with relevant projects
Impact Organise an exploratory workshop on current challenges around the use and re-use of archaeological data Organise a discussion workshop on optimising archaeological data for re-use.
Start Year 2021
 
Title Phaser - archaeological stratigraphic phasing and temporal analysis tool 
Description An archaeological stratigraphic phasing and temporal analysis tool. Enables archaeologists, both professional and non-professional or academic to import excavation stratigraphic data to the online browser based software, construct stratigraphic matrix sequence diagrams along with archaeological dating records and undertake temporal reasoning and analysis and revise phasing and sequencing for excavated sites. The tool also enables export of resulting analysis outputs as digital files in standardized digital archivable formats recommended by the Archaeology Data Service. 
Type Of Technology Webtool/Application 
Year Produced 2021 
Open Source License? Yes  
Impact Archaeologists can import either digital archive records which may be downloaded from online archives such as the Archaeology Data Service or input new data from ongoing excavations. 
URL https://stratigraphic.github.io/matrix/
 
Description Exploratory workshop meeting(s) 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Professional Practitioners
Results and Impact Initial exploratory workshop meeting with archaeologists at Museum of London Archaeology (MOLA). Discussions with professional archaeologists about post-excavation practices, documentation and exploration of potential case study datasets for Matrix project WP1 Data and Process Review.
Year(s) Of Engagement Activity 2020
 
Description Matrix Workshop 1 - user requirements workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact User requirements workshop - 20 archaeological practitioners and computer programming staff from archaeological organizations across England Scotland and Wales took part in a workshop to explore their uses of archaeological software, current practice and to demonstrate prototype software and gather feedback on potential user requirements
Year(s) Of Engagement Activity 2021
URL https://www.eventbrite.com/e/the-matrix-project-workshop1-tickets-157248655963?utm_source=eventbrite...
 
Description Matrix Workshop 2 - user feedback workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This online Workshop was for an invited audience of professional practitioner and academic archaeologists from the UK and USA to review and give feedback on The Matrix project outcomes, including the prototype software developed by the project and discuss planned next steps.
Year(s) Of Engagement Activity 2022
URL https://www.eventbrite.co.uk/e/the-matrix-project-workshop-2-tickets-320525178437
 
Description Matrix project website and YouTube channel 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Website - https://stratigraphic.github.io/matrix/
to publicise project activities and illicit feedback for participatory design purposes about the Matrix project prototype software developments.

YouTube Channel s - https://www.youtube.com/watch?v=Q3JSwzFT6zM
for workshop and research seminar outputs.
Year(s) Of Engagement Activity 2021,2022
URL https://stratigraphic.github.io/matrix/
 
Description Presentation to the UK Federation of Archaeological Managers and Employers (FAME) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Presentation at FAME Forum summer meeting 17/06/2022 to FAME members in York.
See FAME website - https://famearchaeology.co.uk/fame-forum-presentations/
A glitch in The Matrix? presented by Keith May - AHRC Leadership Fellow & Historic England
Year(s) Of Engagement Activity 2022
URL https://www.youtube.com/watch?v=CCqjHSDMUyY