From crowdsourcing to digitally-enabled participation: the state of the art in collaboration, access, and inclusion for cultural heritage institutions

Lead Research Organisation: British Library The
Department Name: Digital Scholarship

Abstract

Crowdsourcing has helped to provide a framework for online participation with, and around, cultural heritage collections for several years. As more cultural institutions ask members of the public to undertake tasks such as transcription, classification and description of digitised collections, expectations about the ability of these tasks to engage audiences more deeply with cultural heritage are rising. Co-creation and developing new models of ownership and authorship present significant challenges for institutions used to framing interactions with collections on their own terms. As crowdsourcing moves from an experimental activity to something that requires closer and more systematic integration with physical and other online outreach activities, institutions are grappling with the associated challenges to their organisational practices.

This project brings together practitioners and researchers representing a range of approaches to digitally-enabled participation in cultural heritage to write a book in a week through a 'book sprint'. In the spirit of digitally-enabled participation, the book will not be considered 'final' until it has been through a period of public comment. The book will be freely available as an open access publication. The project will also assemble leaders in the field for a workshop that captures expert views on the most pressing issues to address through future funding calls, with the results published in a white paper written by the PI and Co-Is.

This project builds on the expertise of three innovators in the field, each at a world-leading institution, who are setting the standards for crowdsourcing and digitally-enabled participation around cultural heritage collections in the UK and US. We have invited a range of expert practitioners and researchers from different fields to participate in our events and publications, enhancing our ability to address challenges from working with marginalised and disenfranchised communities to enhancing access across multiple channels and challenging organisational practice.

In collecting, digesting and sharing best practice in crowdsourcing and digitally-enabled participation, this project will help maximise the value of existing platforms and activities. The support network created through participation in the book sprint, workshop and online discussion will help organisations stay audience-focused as they respond to technical and cultural changes in the field.

This project primarily addresses the 'digitally-enabled participation' theme of this call, but it also touches on Machine Learning and AI and the automatic creation of data based on collections. The focus of the cross-cutting theme 'Contemporary challenges addressed by digital tools and methods' on ethical frameworks to address inequalities in digital engagement, sustaining relationships and developing activities with marginalised groups and cooperative working models are particularly important for this project. Questions of building digital capacity and supporting leaders of digital transformation from the 'Leadership and skills development in digital scholarship' theme are also crucial to the long-term viability of this field.

Planned Impact

This project will frame the current state of crowdsourcing in cultural heritage institutions by convening a group of experts to discuss best practices for designing, managing and integrating crowdsourcing activities, and to look ahead to future challenges and unresolved issues that could be addressed by longer-term collaborations.

It will directly impact crowdsourcing practitioners who work in cultural institutions, as well as those who wish to gain experience with crowdsourcing. By helping those designing and running crowdsourcing and digital engagement projects to maximise the value of their efforts and reduce barriers to participation, this project will help both cultural heritage practitioners and the publics they seek to engage.

The impact on participants in crowdsourcing and digital engagement projects will be transformative, as this work seeks to not only ensure that the publics engaged in this type of project are not exploited, but indeed that they are seen as collaborators on high-quality outcomes. The public will be directly engaged with our research outputs, with participants in crowdsourcing projects encouraged to read and comment on the draft book. This will improve the quality of the final publication, and ensure that participant needs are addressed.

Finally, this work will impact stakeholders at institutions participating in crowdsourcing and digital engagement projects, including colleagues in collections management, copyright and licencing, metadata standards, learning and public engagement.

The listed book sprint attendees are extremely well-suited to deliver this impact as each links a considerable networks of experts in digitally-enabled participation as well as extensive volunteer communities on popular projects into the network. Committed participants have helped to create crowdsourcing projects such as By the People (https://crowd.loc.gov/), the Colored Conventions Project (http://coloredconventions.org/), LibCrowds (https://www.libcrowds.com/), MicroPasts (https://crowdsourced.micropasts.org/) and Zooniverse (https://www.zooniverse.org). These projects have cumulatively engaged with more than 2.5 million volunteers from around the world.

We will disseminate findings in a number of ways: 1) Through knowledge exchange and skills development among the PI, Co-Is, and other attendees at the book sprint; wider engagement of sprint attendees with those providing feedback on the first book draft; and with additional attendees of the follow-up workshop, including those joining remotely. 2) Through the outputs generated from the two events: an open access book and a white paper on priority areas for future research grants. 3) Through a newly-catalysed community of practice that will ensure that the impact of this effort will continue in the long term.

Each method above will be impactful in its own right. The book will provide accessible and practical insights into innovative and effective practice. The follow-up workshop will produce a white paper about the futures of crowdsourcing and digitally-enabled participation. The network will create opportunities for collaborative bids for further funding to address the issues identified through this project. We believe that, once all of the above activities and output are completed, the result will be increased efficiency of public services, and enhanced quality of experience for participants.

We will measure impact in a number of ways, including the successful production of a book draft at the end of the week-long book sprint; the number and quality of comments on the first edition of the book, and the number of comments included in the edits for the second edition; book downloads and sales; white paper downloads and views; and the number of conference papers and blog posts from individual participants.

The timelines and methods for delivering this impact are discussed in the 'Pathways to Impact' attachment.

Publications

10 25 50
 
Description While our award is still in progress, we have made significant progress on our goal of fostering an international community of practice in crowdsourcing, citizen research and online volunteering in cultural heritage. This community includes both to book sprint participants and the wider community of practice made up of crowdsourcing practitioners and volunteers.

To select book sprint participants for our in-person book sprint (planned for Baltimore in April 2020), we wrote and shared a public call for participation in early 2020. This resulted in our group of 15 sprint participants (including the PI and Co-Is). However, the COVID-19 pandemic significantly impacted our planned activities for this project. Once it became clear that the pandemic would prevent us from doing so safely, we decided to delay the book sprint activities until 2021 so we could re-evaluate our options. In the interim, the book sprints facilitation team developed an online alternative, and we chose to move forward with virtual planning.

In response to the COVID-19 pandemic, we modified our one-week in-person book sprint, coordinating with the book sprint facilitators to create two sequential online book sprints. We worked with the facilitation team to determine a structure for the virtual sprint that was in line with our values statement, particularly in regard to mitigating Zoom fatigue, recognising the impact on the household of long working meetings, and ensuring support was in place for sprint members who are caregivers. We redirected our travel budget towards a daily stipend for sprint participants, and are helping with childcare support in special circumstances.

Once the new book sprint plan was established, we contacted each participant originally selected to see if they were still able to participate given our new format and timelines. Once the participant group was re-established, we set up an internal online discussion list for our book sprint participants. This created a space where they could get to know each other and begin to share expertise, while simplifying the dissemination of our planning and logistical messages. This community is further supported with a Slack workspace for discussion during the book sprint process. We expect this will underpin upcoming work on future research agendas.

Externally, we established a public forum on the Humanities Commons website to raise the visibility of our work and provide accessible entry points for newcomers to the field. We also created a project website to communicate the project aims and objectives, and to share early results. We devised a Twitter hashtag - #CollectiveWisdomAHRC - to raise awareness of our project and funder, and to allow people to follow our progress on social media.

We are poised to realise our second objective, as our book sprints will be held from mid-March to early April. Our third objective is currently partially met, as we discuss common questions on our Humanities Commons forum and internal mailing list, and we anticipate generating shared understandings of issues in need of further research through our workshop and white paper later in the year.
Exploitation Route We have published some early survey findings for use by others, but the bulk of our public-facing outcomes will be delivered in the coming months.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Leisure Activities, including Sports, Recreation and Tourism,Culture, Heritage, Museums and Collections

URL https://collectivewisdomproject.org.uk/
 
Description Perhaps ironically, crowdsourcing and online volunteering activity rose sharply in the first weeks and months of lockdown. While this demonstrated the need for our work, the work required to keep new material flowing into projects while responding to media and volunteer enquiries kept the leadership team and our potential booksprint participants very busy. In addition to delaying our book by a year while we worked with the Book Sprint facilitators to turn the physical 'sprint' into an online one, COVID-19 impacted our plans for soliciting feedback from the wider community on our book draft. Our original plan for including a wider range of diverse perspectives in the book was to invite comments from stakeholders and volunteers on the draft of the book, then incorporate them into the final edit. However, the virtual restructuring made that impossible, and so we had to design an alternative way of ensuring that the wider community had a chance to participate in some way. We designed, piloted, adjusted then ran two small surveys designed to collect information that represented a variety of views and perspectives. The surveys asked project stakeholders and volunteers two key questions, with additional questions for stakeholders to provide the basis for possible case studies. In addition to sharing the results with participants in our book sprint, we published the responses to those questions on our website on the day the surveys closed. Our goal was to contribute information back to people who shared their insights, and provide direct access to stakeholder and volunteer perspectives to those outside our usual peer group. As the survey only closed in early March, we have not yet found any evidence of impact for these activities. However, as the surveys provide direct access to the volunteer and stakeholder voice, we expect that they will supplement statistical work published elsewhere and help crowdsourcing and digital participation projects understand volunteer and organisational needs.
First Year Of Impact 2021
Sector Digital/Communication/Information Technologies (including Software),Leisure Activities, including Sports, Recreation and Tourism,Culture, Heritage, Museums and Collections
Impact Types Societal

 
Description Research collaboration with two US institutions for management of project 
Organisation Adler Planetarium
Country United States 
Sector Charity/Non Profit 
PI Contribution Contribution of PI effort to this project by Ridge. Organising Book Sprint, including soliciting and reviewing applications and making offers of participation. Re-organising Book Sprint for virtual participation as a result of COVID-19 travel restrictions and safety measures. Setting up and using tools for project management (Trello, Google Docs). Setting up and using tools for communication with full Sprint team (JISC email listserv, Slack). Writing and soliciting survey responses for additional input via non-Sprint participants. Anonymizing and sharing responses to survey via project website. Administrative support from BL as well as finance support for arranging book sprint stipends, replacing logistical support for travel.
Collaborator Contribution Contribution of Co-I effort to this project by Ferriter and Blickhan. Organising Book Sprint, including soliciting and reviewing applications and making offers of participation. Re-organising Book Sprint for virtual participation as a result of COVID-19 travel restrictions and safety measures. Setting up and using tools for project management (Trello, Google Docs). Setting up and using tools for communication with full Sprint team (JISC email listserv, Slack). Writing and soliciting survey responses for additional input via non-Sprint participants. Anonymizing and sharing responses to survey via project website.
Impact - Organisation of project, including Book Sprint and public outputs listed below - Re-organisation of project due to COVID-19 safety restrictions - Announcement of project via public blog posts on BL, Library of Congress, and Zooniverse blogs - Creation of Project Website, including project overview information and Values Statement - Public Call for Participants in Book Sprint - Public surveys (2) for soliciting input from non-sprint participants, including practitioners and volunteers on cultural heritage crowdsourcing projects
Start Year 2020
 
Description Research collaboration with two US institutions for management of project 
Organisation Library of Congress
Country United States 
Sector Public 
PI Contribution Contribution of PI effort to this project by Ridge. Organising Book Sprint, including soliciting and reviewing applications and making offers of participation. Re-organising Book Sprint for virtual participation as a result of COVID-19 travel restrictions and safety measures. Setting up and using tools for project management (Trello, Google Docs). Setting up and using tools for communication with full Sprint team (JISC email listserv, Slack). Writing and soliciting survey responses for additional input via non-Sprint participants. Anonymizing and sharing responses to survey via project website. Administrative support from BL as well as finance support for arranging book sprint stipends, replacing logistical support for travel.
Collaborator Contribution Contribution of Co-I effort to this project by Ferriter and Blickhan. Organising Book Sprint, including soliciting and reviewing applications and making offers of participation. Re-organising Book Sprint for virtual participation as a result of COVID-19 travel restrictions and safety measures. Setting up and using tools for project management (Trello, Google Docs). Setting up and using tools for communication with full Sprint team (JISC email listserv, Slack). Writing and soliciting survey responses for additional input via non-Sprint participants. Anonymizing and sharing responses to survey via project website.
Impact - Organisation of project, including Book Sprint and public outputs listed below - Re-organisation of project due to COVID-19 safety restrictions - Announcement of project via public blog posts on BL, Library of Congress, and Zooniverse blogs - Creation of Project Website, including project overview information and Values Statement - Public Call for Participants in Book Sprint - Public surveys (2) for soliciting input from non-sprint participants, including practitioners and volunteers on cultural heritage crowdsourcing projects
Start Year 2020
 
Description Blog post: Call for participants: April 2020 book sprint on the state of the art in crowdsourcing in cultural heritage 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The post supported our Call for Participants in our book sprint. It outlined the-then structure, dates and location of the book sprint, what we sought in ideal participants, how to apply, and next steps. It resulted in a strong range of applications, from which we selected final participants.

The post was later updated to reflect changes in response to the COVID-19 pandemic.
Year(s) Of Engagement Activity 2020
URL https://blogs.bl.uk/digital-scholarship/2020/02/call-for-participants-april-2020-book-sprint-on-the-...
 
Description Blog post: New Collaboration between LC Labs, British Library, and the Zooniverse 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Blog post announcing research collaboration, with a call to action for potential participants, resulted in applications for participation and requests for more details submitted by readers.
Year(s) Of Engagement Activity 2020
URL https://blogs.loc.gov/thesignal/2020/02/new-collaboration-between-lc-labs-british-library-and-the-zo...
 
Description Blog post: New project! 'From crowdsourcing to digitally-enabled participation: the state of the art in collaboration, access, and inclusion for cultural heritage institutions' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Blog post announcing the project, outlining our goals and methods, our timetable for events and activities, and explaining why we've taken that approach.
Year(s) Of Engagement Activity 2020
URL https://blogs.bl.uk/digital-scholarship/2020/02/new-project-from-crowdsourcing-to-digitally-enabled-...
 
Description Blog post: The crowdsourcing stakeholder view on 'What do you wish you'd known at the start?' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post presenting initial results for one of two key questions from our stakeholder survey to support participants in our book sprint, and provide insights into crowdsourcing and digital volunteering for the GLAM, academic and voluntary sectors.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/wish-youd-known-at-the-start/
 
Description Blog post: The crowdsourcing stakeholder view on 'Would you do anything differently with more time or resources?' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post presenting initial results for one of two key questions from our stakeholder survey to support participants in our book sprint, and provide insights into crowdsourcing and digital volunteering for the GLAM, academic and voluntary sectors.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/would-you-do-anything-differently/
 
Description Blog post: The volunteer perspective on 'What are your hopes for the future of crowdsourcing / online volunteering?' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Supporters
Results and Impact A blog post presenting initial results for one of two key questions from our volunteer survey to support participants in our book sprint, and provide insights into crowdsourcing and digital volunteering for the GLAM, academic and voluntary sectors.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/hopes-for-the-future-of-crowdsourcing/
 
Description Blog post: The volunteer perspective on 'What makes a crowdsourcing / citizen research / online volunteering project great?' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Supporters
Results and Impact A blog post presenting initial results for one of two key questions from our volunteer survey to support participants in our book sprint, and provide insights into crowdsourcing and digital volunteering for the GLAM, academic and voluntary sectors.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/what-makes-a-crowdsourcing-project-great/
 
Description Blog post: We want to hear from you! 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The blog post provided context for two short surveys run in advance of our book sprints. They helped potential participants understand why we were asking for input, how responses would be used, and what kinds of questions we were asking. The surveys have had nearly 100 responses.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/we-want-to-hear-from-you/
 
Description British Library project web page 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Created a project web page on the British Library website to provide official visible information about the project in support of our other engagement activities.
Year(s) Of Engagement Activity 2021
URL https://www.bl.uk/projects/collective-wisdom
 
Description Forum discussion: Getting started with crowdsourcing in GLAMS and academia: your questions sought 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact In response to the pandemic and our changing plans, we selected the popular Humanities Commons platform as an appropriate place to ask for questions about crowdsourcing and digitally-enabled participation in cultural heritage. Our aim was to help projects avoid common errors, anticipate and manage issues they might face, and ease fears about what can go wrong. We asked forum participants, 'If you're thinking about setting up a crowdsourcing project, how can we help? What questions, hopes, fears do you have?'.

We posted example questions that we've frequently been asked at in-person events, alongside answers.

The forum posts are publicly available, but the Humanities Commons platform requires free membership to post a response.

Tweets related to the posts received c5,400 impressions and over 50 engagements.
Year(s) Of Engagement Activity 2020
URL https://hcommons.org/groups/crowdsourcing/forum/topic/getting-started-with-crowdsourcing-in-glams-an...
 
Description Library of Congress Labs website page: Collective Wisdom 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Library of Congress Labs website page representing the project, which provided a place to document the project for Library of Congress and LC Labs' audiences; resulted in longer than usual engagement following referral traffic from the LC Labs newsletter and Twitter account tweets.
Year(s) Of Engagement Activity 2020
URL https://labs.loc.gov/work/experiments/ahrc-booksprint/
 
Description Presentation of project aims to cultural heritage crowdsourcing practitioners and researchers 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Co-I Blickhan gave a presentation about the Collective Wisdom project aims and efforts so far to the research team behind the Engaging Crowds project (funded by the AHRC as part of the Towards a National Collection research programme). This presentation was intended to 1) signpost opportunities for Engaging Crowds researchers to participate via the Humanities Commons group and the pre-Book Sprint surveys we conducted; 2) discuss the potential for the output of the Collective Wisdom book sprint as a useful resource for the Engaging Crowds team in their ongoing work on volunteer engagement with heritage materials, helping to sure effort isn't duplicated, and that we are learning from one another's work.
Year(s) Of Engagement Activity 2021
 
Description Project website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Created a website to collate project information, including goals, activities and events, progress updates and news, and contact information.
Year(s) Of Engagement Activity 2020
URL https://collectivewisdomproject.org.uk/
 
Description Values Statement 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The Values Statement provided framework for both sprint participants as well as members of the public as to the commitment of the PI and Co-Is to a set of overarching values. These values were consulted as part of the design of the project and execution of activities proposed.
Year(s) Of Engagement Activity 2020
URL https://collectivewisdomproject.org.uk/our-values/