Laymen To The Help Of Experts: Crowdsourcing To Aid The Reassembly Of Ancient Frescoes

Reconstruction of fragmented objects is of great interest in archaeology, where artefacts are often found in a fractured state, and the effort of manually putting the broken pieces together represents a significant amount of time and resources. Representative of this problem is our work on documenting and reconstructing fragments of Late-Bronze-Age wall paintings from the site of Akrotiri on the volcanic island of Thera (modern-day Santorini, Greece). As is common in archaeological finds, the wall paintings are excavated as tens of thousands of pieces, and searching for matches is a daunting manual process that dwarfs the available resources.

In a previous project we have contributed methods to acquire object fragments efficiently, identify pairwise matches between fragments, interactively refine automatically suggested matches and automatically assemble them into larger clusters. However, unavoidable error sources in acquisition and processing, but also physical influences, such a erosion and other damage over the last 3,500 years, lead to uncertainty in identifying matches between fragments. We find that humans are required to sift the automated match suggestions and that without massively parallelising and without introducing redundancy in the manual sifting, there will always be a substantial number of matches go unnoticed.

This work proposes a crowdsourcing solution to obtain this goal. We believe that a key novelty lies in the fact that we are trying to let non-expert solve a problem in the digital domain that normally requires experts to operate in the physical domain. We believe that we are excellently positioned to turn this into a high-impact project with generalisable outcomes.

The project will be conducted in close collaboration with the Akrotiri excavation (who provides space, expert time and access to frescoes) and with Microtask Oy, a very successful startup company specialised on commercial crowdsourcing (Microtask will consult the principal investigator free of charge); Microtask have received significant attention for their digitisation project with the National Library of Finland (http://www.microtask.com/media). From the beginning on, this will ensure that the scope of the project will stay relevant and that the execution is technically sound.

While there are currently no plans for a direct commercialisation during the two-year run time of this project, Microtask have indicated to be interested in helping with a commercialisation beyond the run time of this project.

The Akrotiri Excavation is hoping to be able to continue using our system as a key element of upcoming assembly phases at the site.

In addition, to identify applicable problem domains beyond fresco reassembly as such, we will:-

* Actively seek contact with conservators and archaeologists who face assembly problems as well. In fact, we previously already received requests from curators that work with various broken artefacts, ranging from flint stone figurines, over human bones, to plaster statuettes, tomb stones, papyri, and even fragments of entire buildings. While these objects are typically hard to digitise with our current acquisition pipeline, human help in the match finding task may pose weaker requirements on the digitisation phase, so that our crowdsourcing approach could allow for the assembly of artefacts that otherwise would remain fragmented, be it due to time and money constraints, or because of the scale of the problem.

* Conduct mini-workshops at the upcoming Griphos Seminars at Akrotiri, to brainstorm with conservators and archaeologists on additional opportunities.

* Follow traditional scientific publication strategies to advertise our work.

The larger consortium of participants in the Griphos project will continue exploring additional funding sources, in Greece, the UK, Europe and the US. The Akrotiri excavation has already identified a fresco (from building Xeste 3, room 7) that will be put aside exclusively for our research. Provided sufficient funding is found, Akrotiri would dedicate space and two conservators to work with us on improving our system and to study the assembly procedure under fully controlled conditions, rather than having to conduct our research in parallel to the conservative assembly procedure. If this plan succeeds (earliest in summer 2012), it would offer an additional testbed for the research planned in this proposal.

With respect to a broader audience, the Griphos project has regularly attracted attention in the international press, with articles on various web portals, but also general newspapers, including the Philadelphia Inquirer (a two-page feature), EPOC (a German science magazine which is part of Scientific American) and various Greek newspapers. We will continue to capitalise on the general attractiveness of the subject, in the hope to not only inform a broader audience, but also to make new connections to problem domains previously not thought to be in reach for computer technologies. We also expect that in advertising the crowdsourcing app to visitors of the Akrotiri excavation, we will gain additional visibility for crowdsourcing solutions.

On the educational side, we believe that both the RA and one of my research students who works on fresco assembly will benefit from having received training in a really exceptional problem domain.

More generally, while the immediate scope of our project is to develop a system that is geared toward the assembly of wall paintings, we intend to pursue this development with generality in mind, seeking general applications in the humanities.