From crowdsourcing to digitally-enabled participation: the state of the art in collaboration, access, and inclusion for cultural heritage institutions

Lead Research Organisation: British Library
Department Name: Digital Scholarship

Abstract

Crowdsourcing has helped to provide a framework for online participation with, and around, cultural heritage collections for several years. As more cultural institutions ask members of the public to undertake tasks such as transcription, classification and description of digitised collections, expectations about the ability of these tasks to engage audiences more deeply with cultural heritage are rising. Co-creation and developing new models of ownership and authorship present significant challenges for institutions used to framing interactions with collections on their own terms. As crowdsourcing moves from an experimental activity to something that requires closer and more systematic integration with physical and other online outreach activities, institutions are grappling with the associated challenges to their organisational practices.

This project brings together practitioners and researchers representing a range of approaches to digitally-enabled participation in cultural heritage to write a book in a week through a 'book sprint'. In the spirit of digitally-enabled participation, the book will not be considered 'final' until it has been through a period of public comment. The book will be freely available as an open access publication. The project will also assemble leaders in the field for a workshop that captures expert views on the most pressing issues to address through future funding calls, with the results published in a white paper written by the PI and Co-Is.

This project builds on the expertise of three innovators in the field, each at a world-leading institution, who are setting the standards for crowdsourcing and digitally-enabled participation around cultural heritage collections in the UK and US. We have invited a range of expert practitioners and researchers from different fields to participate in our events and publications, enhancing our ability to address challenges from working with marginalised and disenfranchised communities to enhancing access across multiple channels and challenging organisational practice.

In collecting, digesting and sharing best practice in crowdsourcing and digitally-enabled participation, this project will help maximise the value of existing platforms and activities. The support network created through participation in the book sprint, workshop and online discussion will help organisations stay audience-focused as they respond to technical and cultural changes in the field.

This project primarily addresses the 'digitally-enabled participation' theme of this call, but it also touches on Machine Learning and AI and the automatic creation of data based on collections. The focus of the cross-cutting theme 'Contemporary challenges addressed by digital tools and methods' on ethical frameworks to address inequalities in digital engagement, sustaining relationships and developing activities with marginalised groups and cooperative working models are particularly important for this project. Questions of building digital capacity and supporting leaders of digital transformation from the 'Leadership and skills development in digital scholarship' theme are also crucial to the long-term viability of this field.

Planned Impact

This project will frame the current state of crowdsourcing in cultural heritage institutions by convening a group of experts to discuss best practices for designing, managing and integrating crowdsourcing activities, and to look ahead to future challenges and unresolved issues that could be addressed by longer-term collaborations.

It will directly impact crowdsourcing practitioners who work in cultural institutions, as well as those who wish to gain experience with crowdsourcing. By helping those designing and running crowdsourcing and digital engagement projects to maximise the value of their efforts and reduce barriers to participation, this project will help both cultural heritage practitioners and the publics they seek to engage.

The impact on participants in crowdsourcing and digital engagement projects will be transformative, as this work seeks to not only ensure that the publics engaged in this type of project are not exploited, but indeed that they are seen as collaborators on high-quality outcomes. The public will be directly engaged with our research outputs, with participants in crowdsourcing projects encouraged to read and comment on the draft book. This will improve the quality of the final publication, and ensure that participant needs are addressed.

Finally, this work will impact stakeholders at institutions participating in crowdsourcing and digital engagement projects, including colleagues in collections management, copyright and licencing, metadata standards, learning and public engagement.

The listed book sprint attendees are extremely well-suited to deliver this impact as each links a considerable networks of experts in digitally-enabled participation as well as extensive volunteer communities on popular projects into the network. Committed participants have helped to create crowdsourcing projects such as By the People (https://crowd.loc.gov/), the Colored Conventions Project (http://coloredconventions.org/), LibCrowds (https://www.libcrowds.com/), MicroPasts (https://crowdsourced.micropasts.org/) and Zooniverse (https://www.zooniverse.org). These projects have cumulatively engaged with more than 2.5 million volunteers from around the world.

We will disseminate findings in a number of ways: 1) Through knowledge exchange and skills development among the PI, Co-Is, and other attendees at the book sprint; wider engagement of sprint attendees with those providing feedback on the first book draft; and with additional attendees of the follow-up workshop, including those joining remotely. 2) Through the outputs generated from the two events: an open access book and a white paper on priority areas for future research grants. 3) Through a newly-catalysed community of practice that will ensure that the impact of this effort will continue in the long term.

Each method above will be impactful in its own right. The book will provide accessible and practical insights into innovative and effective practice. The follow-up workshop will produce a white paper about the futures of crowdsourcing and digitally-enabled participation. The network will create opportunities for collaborative bids for further funding to address the issues identified through this project. We believe that, once all of the above activities and output are completed, the result will be increased efficiency of public services, and enhanced quality of experience for participants.

We will measure impact in a number of ways, including the successful production of a book draft at the end of the week-long book sprint; the number and quality of comments on the first edition of the book, and the number of comments included in the edits for the second edition; book downloads and sales; white paper downloads and views; and the number of conference papers and blog posts from individual participants.

The timelines and methods for delivering this impact are discussed in the 'Pathways to Impact' attachment.

Publications

10 25 50
 
Description The network-building work funded through this award allowed the grant team to design and author a book documenting practical considerations and the state of the field of crowdsourcing in cultural heritage. This work further allowed a core group of crowdsourcing practitioners to define the key practices, challenges, and opportunities for the future of crowdsourcing in GLAMs in the UK and US. Furthermore, crowdsourcing practitioners, community leaders, researchers, and leadership were able to share of-the-moment considerations about the complex and nuanced dimensions of crowdsourcing, including collectively surfacing the research topics to be pursued and practices to be designed and integrated in the next 1-3 years. As a result of this body of work, current and future practitioners and their leadership are poised to implement informed, better-resourced, and sustainable practices to extend the successes and address challenges of crowdsourcing in cultural heritage.

By March 2023, the project website, launched in October 2020, reached nearly 5,000 visitors and over 8,200 page views from 104 countries.

The Collective Wisdom Handbook has over 17,000 pageviews from nearly 6,200 readers from 123 countries. After the Introduction and Colophon, the most viewed chapter was 3 'Why work with crowdsourcing in cultural heritage?' and the most downloaded chapter was 10 'Working with crowdsourced data'.

We made significant progress on our goal of fostering an international community of practice in crowdsourcing, citizen research and online volunteering in cultural heritage. This community includes both book sprint participants and the wider community of practice made up of crowdsourcing practitioners and volunteers. In addition to sparking interest in the field of practice with the Collective Wisdom Handbook resulting from the Book Sprint in March 2021, we broadened the scope of the network with inclusion of computer scientists, designers, researchers, and representatives of funding bodies at our Collective Wisdom workshops in October 2021.

To select book sprint participants for our in-person book sprint (planned for Baltimore in April 2020), we wrote and shared a public call for participation in early 2020. This resulted in our group of 15 sprint participants (including the PI and Co-Is). However, the COVID-19 pandemic significantly impacted our planned activities for this project. Once it became clear that the pandemic would prevent us from doing so safely, we decided to delay the book sprint activities until 2021 so we could re-evaluate our options. In the interim, the book sprints facilitation team developed an online alternative, and we chose to move forward with virtual planning.

In response to the COVID-19 pandemic, we modified our one-week in-person book sprint, coordinating with the book sprint facilitators to create two sequential online book sprints. We worked with the facilitation team to determine a structure for the virtual sprint that was in line with our values statement (https://collectivewisdomproject.org.uk/our-values/), particularly in regard to mitigating Zoom fatigue, recognising the impact on the household of long working meetings, and ensuring support was in place for sprint members who are caregivers. We redirected our travel budget towards a daily stipend for sprint participants, and helped with childcare support in special circumstances.

Once the new book sprint plan was established, we contacted each participant originally selected to see if they were still able to participate given our new format and timelines. When the participant group was re-established, we set up an internal online discussion list for our book sprint participants. This created a space where they could get to know each other and begin to share expertise, while simplifying the dissemination of our planning and logistical messages. We prepared the participants for the book sprint with a brief series of on-ramping messages in this listserv. This community is further supported with a Slack workspace, which created space for discussion during the book sprint process and permitted ongoing communication between the project team and participants through the end of the grant period. We expect this will underpin upcoming work on future research agendas. For example, we used the Slack and listserv to ask for reflections on the anniversary of our book sprint beginning.

Externally, we established a public forum (https://hcommons.org/groups/crowdsourcing/) on the Humanities Commons website to raise the visibility of our work and provide accessible entry points for newcomers to the field. We also created a project website (https://collectivewisdomproject.org.uk) to communicate the project aims and objectives, and to share early results. We used the project website to reinforce the values we established for the project, including sharing outcomes at each step of the project and even mid-project analysis via blog posts. We devised a Twitter hashtag - #CollectiveWisdomAHRC - to raise awareness of our project and funder, and to allow people to follow our progress on social media.

Our second objective of capturing the state of the field was accomplished in April 2021, following our book sprints held from mid-March to early April. In late April, we completed initial editing of the co-authored work The Collective Wisdom Handbook consisting of over 90,000 words and featuring over 60 case studies from galleries, libraries, archives, museums, international communities of practice, and local communities, as well. We published the work via the open access platform PubPub in May 2021 (available at https://britishlibrary.pubpub.org/ and contextualised with communications such as https://collectivewisdomproject.org.uk/early-access-version/) so that it could be available immediately as a resource for practitioners, educators, and leadership. We continue to pursue publishing opportunities at the time of writing.

Our third objective is ongoing, as we discuss common questions on our Humanities Commons forum, social media and internal mailing list. We anticipate generating shared understandings of issues in need of further research through launch of our white paper in 2023.
Exploitation Route Our white paper addresses the following tightly-interrelated themes for future attention by research teams and practitioners: Infrastructure, Evidencing and Evaluation, Skills and Competencies, Communities of Practice, and Incorporating Emergent Technologies and Methods. We believe that these recommendations, and the ways in which they explicitly integrate the concept of values-driven design and implementation, will enable people to lead and contribute in digitally-enabled participation that result in mutually beneficial exchanges that transform lives and cultural heritage practice.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Leisure Activities, including Sports, Recreation and Tourism,Culture, Heritage, Museums and Collections

URL https://collectivewisdomproject.org.uk/
 
Description In 2022/23 we have continued to receive invitations to present on our work on this project, and were recently successful in proposing a presentation on our White Paper at the international Digital Humanities 2023 conference in Graz, Austria. The book has formed the basis of commissioned workshops for academics and cultural heritage staff planning crowdsourcing projects. Our open access handbook has had over 17,000 page views by nearly 6,200 visitors. The majority of page views come from Google searches, suggesting a wide reach beyond our own channels for promotion. We have also been asked to share our experience of writing collaboratively with other funded projects. Previous update: As our Engagement Activities and Influence on Policy etc sections show, once we delivered the open access Collective Wisdom Handbook, we worked to engage people with the contents of our handbook through an 'open peer review process'. Our 'community review period for the Collective Wisdom project ran from May 12 to August 9 2021. In that time reviewers and authors left 153 comments / threads on the book draft. We credited these contributors in a blog post summarising their comments at https://collectivewisdomproject.org.uk/collective-response-feedback-from-the-community-review-period/ Chapters with the most comments included 2: What is crowdsourcing in cultural heritage? (33 comments); 5: Designing cultural heritage crowdsourcing projects (27 comments); and 3: Why work with crowdsourcing in cultural heritage? (26 comments). The number of comments in these three chapters may well be because they attempt to define the field and put it into wider context. Previously, definitions were often based on personal experience and contextualized by project goals and needs, and largely ad hoc design practice. We further shared experiences of designing the Collective Wisdom project and the Book Sprint with audiences at the Association for Computers in the Humanities conference and Collections Trust network. We wrote a case study about the project for the British Library's Research Report for 2019-20. These communities of scholars and practitioners may now carry forward the findings of this grant, as well as the methods. The Collective Wisdom Workshop design was described as an example of structuring participant engagement in collaborative settings in the forthcoming chapter 'Scaffolding Collaboration: Digital Humanities Workshops: Lessons Learned' by Ridge and Eileen Jakeway Manchester. We have also heard from practitioners including crowdsourcing project leads and heads of collection departments that the Collective Wisdom Handbook has been invaluable in capturing best practice in an easily digestible form. We also know that the Collective Wisdom Handbook has been used in university courses and professional contexts in the UK and the US. PI Ridge used the Handbook when devising the structure of a 2 day workshop in collaboration with the Estonian War Museum for five Estonian museums with Europeana funding. Directly as a result of the Collective Wisdom project, the PI and Co-Is were invited to keynote the final workshop of the 2 year National Endowment for the Humanities grant titled, 'Enabling and Reusing Multilingual Citizen Contributions in the Archival Record', which seeks 'to conceive workflows and recommended practices for the preservation, long-term access, and attribution of collaborative transcription work'.
First Year Of Impact 2021
Sector Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Leisure Activities, including Sports, Recreation and Tourism,Culture, Heritage, Museums and Collections
Impact Types Societal,Economic,Policy & public services

 
Description Collective Wisdom workshops: advanced questions for crowdsourcing in cultural heritage and the digital humanities
Geographic Reach Multiple continents/international 
Policy Influence Type Contribution to new or Improved professional practice
Impact With over 30 attendees, the Collective Wisdom workshops reinvigorated existing knowledge exchange networks between professional practitioners. They also established new connections for future research collaboration between those at the forefront of public engagement, research, software design, as well as funding bodies. Attendees of the two workshops reflected that the experience resulted in new ways of thinking about the collective challenges in the field and offered practical next steps for building ethical and impactful engagement practices. The emerging technologies workshop introduced researchers and community managers to design essential considerations for data generation, management, documentation, and reuse, which are key components for the next phase of practice in the field of digitally-enabled participation.
URL https://collectivewisdomproject.org.uk/activities/
 
Description Focused Reading in University of Washington Computing Cultural Heritage course
Geographic Reach North America 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Guest lecture and assigned reading: Crowdsourcing at the British Library for UCL's MSc in Data Science for Cultural Heritage
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Guest lecture: Europeana masterclass for Open Digital Cultural Heritage
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Guest lecture: INOS project, overview of citizen science and crowdsourcing
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
URL https://inos-project.eu/2021/07/28/workshop-report-citizen-science-why-get-involved/
 
Description Hands-on workshop 'Planning Crowdsourcing Projects in Cultural Heritage' for Europeana Research and the Europeana Research Community
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
Impact Participants reported improved abilities to undertake crowdsourcing projects. As the workshop was held a few weeks ago, evidence is still being gathered.
 
Description Invited lecture, Crowdsourcing at the British Library
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Project output used as assigned reading for training workshop
Geographic Reach North America 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Project output used in collaborative workshop with Estonian museums
Geographic Reach Europe 
Policy Influence Type Influenced training of practitioners or researchers
Impact Attendees developed skills at the workshop, enhanced by their access to our Open Access Handbook.
URL https://esm.ee/for-visitors/news/the-war-museum-helps-estonian-museums-to-put-crowdsourcing-into-use
 
Description Special lecture 'Things to Know When Planning Crowdsourcing Projects in Cultural Heritage' for Europeana
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Title New questionnaires for crowdsourcing organisers and volunteers 
Description We developed, piloted and refined two questionnaires for surveys of crowdsourcing project organisers and volunteers. Our 'Lessons learnt' / 'Case Study' survey is designed for practitioners focused on two key questions: what do you wish you'd known at the start? And, would you do anything differently with more time or resources? The 'Volunteer Voice' survey asks: what makes a crowdsourcing / citizen research / online volunteering project great? And what are your hopes for the future of crowdsourcing / online volunteering? 
Type Of Material Improvements to research infrastructure 
Year Produced 2021 
Provided To Others? Yes  
Impact We used the results in our open access Collective Wisdom Handbook and published the results on our website for access by other researchers. 
URL https://collectivewisdomproject.org.uk/we-want-to-hear-from-you/
 
Title Collective Wisdom crowdsourcing organiser and volunteer survey results 
Description Results from two short surveys run for the Collective Wisdom project. Funded by the UK Arts and Humanities Research Council (AHRC), the Collective Wisdom project captures the collective wisdom of researchers and practitioners in crowdsourcing, citizen history, citizen science and public / community participation in research with cultural heritage collections. The surveys were for people who volunteer for or work 'behind the scenes' in crowdsourcing, citizen science, citizen history, digital / online volunteer projects, programmes, tools or platforms with cultural heritage collections. They were designed to supplement the book written by project participants with broader and more international case studies and insights. It represents a sample of convenience, dependent on the team's reach on social media, and the membership of mailing lists and newsletters we posted to. Further information is available at https://collectivewisdomproject.org.uk/we-want-to-hear-from-you/ and https://collectivewisdomproject.org.uk/tag/survey-results/ ##What's in the files?## #Collective_Wisdom_stakeholder_survey_results_cleaned.csv This survey had 32 responses. The first response was on February 10 and the last response was March , 2021. The file contains columns with the following headings, with the headings representing the survey questions: Timestamp Your name What was your role(s) on the project? What was it called? here can we find it online? What was its main purpose? What is it - a project, platform, tool, programme of events, or something else? What did you ask the public to do? Who were your target participants? What do you wish you'd known at the start? Would you do anything differently with more time or resources? Is there anything else you'd like to tell us? #Collective_Wisdom_Volunteer_Voice_results_cleaned.csv This survey had 70 responses. The first response was on February 9 and the last response was March 1, 2021. The file contains columns with the following headings, with the headings representing the survey questions: Timestamp Your name The project(s) you volunteer with What makes a crowdsourcing / citizen research / online volunteering project great? What are your hopes for the future of crowdsourcing / online volunteering? Is there anything else you want to tell us? 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
URL https://bl.iro.bl.uk/concern/datasets/2b21c224-2070-4839-8bff-4658e382fbf5
 
Description Collaboration with the Estonian War Museum on a Europeana-funded project 
Organisation Europeana
Country Netherlands 
Sector Public 
PI Contribution I was invited to be a named researcher on a bid by the Estonian War Museum to run a workshop and pilot mini-crowdsourcing projects, funded by Europeana. I contributed to their survey design, and devised and ran 6 structured sessions within a 2 day workshop, designed to take organisations through the processes involved in planning a successful crowdsourcing project. The workshops included prompts for discussion across many departments and disciplines within an organisation, and concluded with group presentations of the ideas developed through the workshops.
Collaborator Contribution The project "Crowdsourcing for military heritage in Estonia" is funded by 9925 Euros on a period of January to June 2022. The Estonian War Museum leads the project, organising the workshops, running the survey and reporting on the results, and monitoring the five projects as they develop to June 2022.
Impact The final outputs will be five small-scale crowdsourcing projects by Estonian museums, a survey, and publications on the lessons the institutions running them learned from the research project.
Start Year 2021
 
Description Research collaboration with two US institutions for management of project 
Organisation Adler Planetarium
Country United States 
Sector Charity/Non Profit 
PI Contribution Contribution of PI effort to this project by Ridge. Organising Book Sprint, including soliciting and reviewing applications and making offers of participation. Re-organising Book Sprint for virtual participation as a result of COVID-19 travel restrictions and safety measures. Setting up and using tools for project management (Trello, Google Docs). Setting up and using tools for communication with full Sprint team (JISC email listserv, Slack). Writing and soliciting survey responses for additional input via non-Sprint participants. Anonymizing and sharing responses to survey via project website. Administrative support from BL as well as finance support for arranging book sprint stipends, replacing logistical support for travel.
Collaborator Contribution Contribution of Co-I effort to this project by Ferriter and Blickhan. Organising Book Sprint, including soliciting and reviewing applications and making offers of participation. Re-organising Book Sprint for virtual participation as a result of COVID-19 travel restrictions and safety measures. Setting up and using tools for project management (Trello, Google Docs). Setting up and using tools for communication with full Sprint team (JISC email listserv, Slack). Writing and soliciting survey responses for additional input via non-Sprint participants. Anonymizing and sharing responses to survey via project website.
Impact - Organisation of project, including Book Sprint and public outputs listed below - Re-organisation of project due to COVID-19 safety restrictions - Announcement of project via public blog posts on BL, Library of Congress, and Zooniverse blogs - Creation of Project Website, including project overview information and Values Statement - Public Call for Participants in Book Sprint - Public surveys (2) for soliciting input from non-sprint participants, including practitioners and volunteers on cultural heritage crowdsourcing projects
Start Year 2020
 
Description Research collaboration with two US institutions for management of project 
Organisation Library of Congress
Country United States 
Sector Public 
PI Contribution Contribution of PI effort to this project by Ridge. Organising Book Sprint, including soliciting and reviewing applications and making offers of participation. Re-organising Book Sprint for virtual participation as a result of COVID-19 travel restrictions and safety measures. Setting up and using tools for project management (Trello, Google Docs). Setting up and using tools for communication with full Sprint team (JISC email listserv, Slack). Writing and soliciting survey responses for additional input via non-Sprint participants. Anonymizing and sharing responses to survey via project website. Administrative support from BL as well as finance support for arranging book sprint stipends, replacing logistical support for travel.
Collaborator Contribution Contribution of Co-I effort to this project by Ferriter and Blickhan. Organising Book Sprint, including soliciting and reviewing applications and making offers of participation. Re-organising Book Sprint for virtual participation as a result of COVID-19 travel restrictions and safety measures. Setting up and using tools for project management (Trello, Google Docs). Setting up and using tools for communication with full Sprint team (JISC email listserv, Slack). Writing and soliciting survey responses for additional input via non-Sprint participants. Anonymizing and sharing responses to survey via project website.
Impact - Organisation of project, including Book Sprint and public outputs listed below - Re-organisation of project due to COVID-19 safety restrictions - Announcement of project via public blog posts on BL, Library of Congress, and Zooniverse blogs - Creation of Project Website, including project overview information and Values Statement - Public Call for Participants in Book Sprint - Public surveys (2) for soliciting input from non-sprint participants, including practitioners and volunteers on cultural heritage crowdsourcing projects
Start Year 2020
 
Description Association for Computers and the Humanities paper presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation for the annual Association for Computers and the Humanities conference, "Writing Our Way to Common Ground," about the process of creating, adapting, and leading a book sprint. Sparked discussion on the conference forum.
Year(s) Of Engagement Activity 2021
URL https://ach2021.ach.org/
 
Description Blog post - Collective Wisdom: From Book Sprint to Community Review 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This blog post on the Library of Congress Signal Blog provided a summary of grant activities in mid-2021 and a reminder of the open call for community review. The Signal Blog reaches an audience of over 23,000 subscribers. Following the publication of the blog post, Ferriter was invited to share about the Collective Wisdom open call in Library of Congress internal staff newsletters, increasing the awareness of the Collective Wisdom Handbook and its publication as an open access resource.
Year(s) Of Engagement Activity 2021
URL https://blogs.loc.gov/thesignal/2021/08/collective-wisdom/
 
Description Blog post: 'Platforms for People-Powered Research' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post highlighting contributions to a conference and sharing a video from the panel discussion.
Year(s) Of Engagement Activity 2021
URL https://livingwithmachines.ac.uk/platforms-for-people-powered-research/
 
Description Blog post: Bridging humanities: embedding public participation in a collaborative research project 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post highlighting our contribution to a panel at the Association for Computing in the Humanities conference.
Year(s) Of Engagement Activity 2021
URL https://livingwithmachines.ac.uk/bridging-humanities-embedding-public-participation-in-a-collaborati...
 
Description Blog post: Call for participants: April 2020 book sprint on the state of the art in crowdsourcing in cultural heritage 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact The post supported our Call for Participants in our book sprint. It outlined the-then structure, dates and location of the book sprint, what we sought in ideal participants, how to apply, and next steps. It resulted in a strong range of applications, from which we selected final participants.

The post was later updated to reflect changes in response to the COVID-19 pandemic.
Year(s) Of Engagement Activity 2020
URL https://blogs.bl.uk/digital-scholarship/2020/02/call-for-participants-april-2020-book-sprint-on-the-...
 
Description Blog post: Collective response: feedback from the community review period 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post discussing the feedback received from the community review period for the Collective Wisdom Handbook. The post presents common themes in reviewer feedback, what we learned through the process, as well as what we edited within the text in response to the feedback we received. It also includes a list of community reviewers who participated in the process.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/collective-response-feedback-from-the-community-review-period...
 
Description Blog post: New Collaboration between LC Labs, British Library, and the Zooniverse 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Blog post announcing research collaboration, with a call to action for potential participants, resulted in applications for participation and requests for more details submitted by readers.
Year(s) Of Engagement Activity 2020
URL https://blogs.loc.gov/thesignal/2020/02/new-collaboration-between-lc-labs-british-library-and-the-zo...
 
Description Blog post: New project! 'From crowdsourcing to digitally-enabled participation: the state of the art in collaboration, access, and inclusion for cultural heritage institutions' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Blog post announcing the project, outlining our goals and methods, our timetable for events and activities, and explaining why we've taken that approach.
Year(s) Of Engagement Activity 2020
URL https://blogs.bl.uk/digital-scholarship/2020/02/new-project-from-crowdsourcing-to-digitally-enabled-...
 
Description Blog post: Planning for the conversations we want at our online workshops 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post outlining how we planned activities for our final workshops to ensure that we made the most of the move to online events, while also attempting to minimise the loss of rich and serendipitous conversations at an in-person event.
Year(s) Of Engagement Activity 2022
URL https://collectivewisdomproject.org.uk/planning-for-the-conversations-we-want-at-our-online-workshop...
 
Description Blog post: The Collective Wisdom Handbook: One Year On 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post reflecting on the actions taken during the reporting period, demonstrating some of the outcomes of the experience and the impact of the work, including page views for the Collective Wisdom Handbook and download numbers per chapter.
Year(s) Of Engagement Activity 2022
URL https://collectivewisdomproject.org.uk/the-collective-wisdom-handbook-one-year-on/
 
Description Blog post: The crowdsourcing stakeholder view on 'What do you wish you'd known at the start?' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post presenting initial results for one of two key questions from our stakeholder survey to support participants in our book sprint, and provide insights into crowdsourcing and digital volunteering for the GLAM, academic and voluntary sectors.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/wish-youd-known-at-the-start/
 
Description Blog post: The crowdsourcing stakeholder view on 'Would you do anything differently with more time or resources?' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post presenting initial results for one of two key questions from our stakeholder survey to support participants in our book sprint, and provide insights into crowdsourcing and digital volunteering for the GLAM, academic and voluntary sectors.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/would-you-do-anything-differently/
 
Description Blog post: The volunteer perspective on 'What are your hopes for the future of crowdsourcing / online volunteering?' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Supporters
Results and Impact A blog post presenting initial results for one of two key questions from our volunteer survey to support participants in our book sprint, and provide insights into crowdsourcing and digital volunteering for the GLAM, academic and voluntary sectors.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/hopes-for-the-future-of-crowdsourcing/
 
Description Blog post: The volunteer perspective on 'What makes a crowdsourcing / citizen research / online volunteering project great?' 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Supporters
Results and Impact A blog post presenting initial results for one of two key questions from our volunteer survey to support participants in our book sprint, and provide insights into crowdsourcing and digital volunteering for the GLAM, academic and voluntary sectors.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/what-makes-a-crowdsourcing-project-great/
 
Description Blog post: We want to hear from you! 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The blog post provided context for two short surveys run in advance of our book sprints. They helped potential participants understand why we were asking for input, how responses would be used, and what kinds of questions we were asking. The surveys have had nearly 100 responses.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/we-want-to-hear-from-you/
 
Description Blog post: We're on a podcast! 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A blog post amplifying my participation in the podcast, including key quotes and further context about our motivations for creating the Collective Wisdom Handbook.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/were-on-a-podcast/
 
Description British Library project web page 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Created a project web page on the British Library website to provide official visible information about the project in support of our other engagement activities.
Year(s) Of Engagement Activity 2021
URL https://www.bl.uk/projects/collective-wisdom
 
Description Building Capable Communities for Crowdsourced Transcription 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact 'The Collective Wisdom Handbook' was used frequently as part of a 18-month training institute run by Co-I Blickhan. Institute participants produced their own text transcription crowdsourcing projects, and multiple chapters of the Handbook were assigned as preparatory readings and discussed by the group during institute meetings. Attendees found the Handbook extremely useful, and noted that they appreciated the opportunity to discuss its content as a group.
Year(s) Of Engagement Activity 2021,2022,2023
 
Description DH at Oxford Summer School 'Crowdsourced Research in the Humanities' strand 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Co-I Blickhan convened a week-long course, 'Crowdsourced Research in the Humanities' for the DH at Oxford Summer School, held at Keble College from July 11-15. The 'Collective Wisdom Handbook' was the primary text for the course, and participants were assigned multiple chapters as preparatory readings for the workshop. Discussions referred back to the text frequently, and participants noted they appreciated the opportunity to discuss the text in greater detail.
Year(s) Of Engagement Activity 2022
 
Description Forum discussion: Getting started with crowdsourcing in GLAMS and academia: your questions sought 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact In response to the pandemic and our changing plans, we selected the popular Humanities Commons platform as an appropriate place to ask for questions about crowdsourcing and digitally-enabled participation in cultural heritage. Our aim was to help projects avoid common errors, anticipate and manage issues they might face, and ease fears about what can go wrong. We asked forum participants, 'If you're thinking about setting up a crowdsourcing project, how can we help? What questions, hopes, fears do you have?'.

We posted example questions that we've frequently been asked at in-person events, alongside answers.

The forum posts are publicly available, but the Humanities Commons platform requires free membership to post a response.

Tweets related to the posts received c5,400 impressions and over 50 engagements.
Year(s) Of Engagement Activity 2020
URL https://hcommons.org/groups/crowdsourcing/forum/topic/getting-started-with-crowdsourcing-in-glams-an...
 
Description Invited guest speaker on MadeTech podcast 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact The fortnightly Making Tech Better podcast is designed for an audience of technologists, featuring 'positive content from all over the world on how to improve your software delivery'. The blurb for this episode was, 'What is crowdsourcing, and how is it used to improve the British Library's online cultural heritage collections? Clare Sudbery talks to crowdsourcing expert Dr Mia Ridge about the power of volunteer digital engagement.'

The podcast episode had 399 downloads; 216 from the UK; 46 from the US; 19 from Germany; 17 from Australia, 12 from Italy; 11 from Canada; the other top ten countries were the Netherlands, Spain, Denmark and El Salvador.
Year(s) Of Engagement Activity 2021
URL https://www.madetech.com/resources/podcasts/episode-14-mia-ridge-2/
 
Description Invited lecture, Luxembourg Centre for Contemporary and Digital History (C2DH) Hands-on History lecture series 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact My talk, 'Crowdsourcing in Living with Machines: crowdsourcing for engagement meets data science research', sparked a rich discussion afterwards
Year(s) Of Engagement Activity 2022
URL https://www.c2dh.uni.lu/events/crowdsourcing-living-machines-crowdsourcing-engagement-meets-data-sci...
 
Description Invited panelist, European Holocaust Research Infrastructure WP12 Workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Panelist in a 1-day workshop presented by the United States Holocaust Memorial Museum about with new methods of accessing archival collections aimed at smaller institutions with little to no IT support.
Year(s) Of Engagement Activity 2021
URL https://www.ehri-project.eu/
 
Description Invited talk, Citizen Science Hour for Biodiversity Collections 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Invited talk for a webinar series sponsored by iDigBio, 'dedicated to catalyzing excellence in citizen science that engages biodiversity collections' and aimed at professionals within institutions like museums, botanical gardens, universities, field stations, government research centers, and other places. The talk sparked discussion on social media.
Year(s) Of Engagement Activity 2021
URL https://www.idigbio.org/wiki/index.php/Webinar_Series:_Citizen_Science_Hour_for_Biodiversity_Collect...
 
Description Invited talk, Digital Humanities at Oxford Virtual Summer School 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Invited talk for the virtual DH Oxford Summer School, "Preparing for a Crowd: the Invisible Labor of Online Crowdsourcing Projects." c300 attendees on Zoom from all over the world. The talk sparked discussion on social media and led multiple audience members to explore creating their own crowdsourcing project.
Year(s) Of Engagement Activity 2021
URL https://digital.humanities.ox.ac.uk/digital-humanities-oxford-summer-school
 
Description Invited talk, National Information Standards Organization (NISO PLUS) conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Gave a talk entitled "Design as a tool for producing authoritative information in online crowdsourcing" as part of a panel on information authority. In this talk I heavily referenced material from The Collective Wisdom Handbook, which sparked questions and discussion in the Zoom chat as well as on Twitter. The invitation to participate was received within the funding period, and was due in part to my work with the Collective Wisdom project.
Year(s) Of Engagement Activity 2022
URL https://np22.niso.plus/
 
Description Invited talk, Princeton University, 'Crowdsourcing and the Humanities' 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Invited talk and panel discussion for an event with the Center for Research Data and Digital Scholarship at University of Pennsylvania Libraries, The Center for Digital Humanities at Princeton University Library, the Princeton Geniza Lab, and the Zooniverse, attended by c40 people. The panel and event sparked extended discussion on social media.
Year(s) Of Engagement Activity 2021
URL https://genizalab.princeton.edu/crowdsourcing-and-the-humanities
 
Description Invited talk: Crowdsourcing in cultural heritage lecture for Institut für Kunstgeschichte, LMU München 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact An invited talk for a German seminar group.
Year(s) Of Engagement Activity 2021
 
Description Invited talk: Pandemic Pivots: What We Do When We Can't Digitize 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Co-Investigator Meghan Ferriter was invited in March 2020 to speak in this panel for the Smithsonian Digitization Program Office's annual 2022 Smithsonian Digitization Conference. This invited panel includes discussion with Science Museum Group Digital Director John Stack and Pratt Institute Lecturer and Museums+AI Network co-Investigator Elena Villlaespesa, as well as Smithsonian Data Science Lab Researcher Mike Trizna. The discussion will focus on the types of activities undertaken in 2020 and 2021, covering this grant's funding period, and emphasize engagment and sociotechnical methods for enhancing access to digitized collections, including crowdsourcing.
Year(s) Of Engagement Activity 2022
URL https://dpo.si.edu/2022-program
 
Description Invited talk: User Experience (UX) for Citizen Science , iDigBio event 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact I was invited to speak at the event 'Biodiversity Digitization: Celebrating a decade of progress' in the session 'Innovations: Strategy & Coordination'. My talk outlined the importance of user experience design (UX) for increasing diverse participation in citizen science projects.
Year(s) Of Engagement Activity 2021
URL https://www.idigbio.org/wiki/index.php/Biodiversity_Digitization:_Celebrating_a_decade_of_progress
 
Description Invited talk: Working with crowdsourced data 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dealing with complexity: Collections Trust conference 2021 in a session 'Session 4: Lost in the crowd'.

Working with crowdsourced data
Mia Ridge, Principal Investigator, Collective Wisdom (British Library) and Co-Investigators Meghan Ferriter (Library of Congress) and Sam Blickhan (Zooniverse)
The AHRC-funded Collective Wisdom project is capturing exactly that from researchers and practitioners in crowdsourcing, citizen history, citizen science and other participatory research based on cultural heritage collections. This talk will focus on the lifecycle of data in a crowdsourcing project, planning for data, thinking ahead to how it will be used, the quality control measures that might be required, and the ethical and legal questions that might arise.
Year(s) Of Engagement Activity 2021
URL https://collectionstrust.org.uk/uncategorized/2021-conference-programme/
 
Description Library of Congress Labs website page: Collective Wisdom 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Library of Congress Labs website page representing the project, which provided a place to document the project for Library of Congress and LC Labs' audiences; resulted in longer than usual engagement following referral traffic from the LC Labs newsletter and Twitter account tweets.
Year(s) Of Engagement Activity 2020
URL https://labs.loc.gov/work/experiments/ahrc-booksprint/
 
Description Panel discussion: Expanding and Enriching Metadata through Engagement with Communities 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This panel discusses how cultural institutions are engaging various communities to co-create academic research and/or object metadata in order to increase representation and access to collections; highlighting how this is done in different ways to engage specific audiences and goals, i.e. graduate student assistantships, museum interactive experiences, crowdsourcing, and professional action groups.
Year(s) Of Engagement Activity 2021
URL https://mcn2021virtual.sched.com/event/lwrc/expanding-and-enriching-metadata-through-engagement-with...
 
Description Presentation of project aims to cultural heritage crowdsourcing practitioners and researchers 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact Co-I Blickhan gave a presentation about the Collective Wisdom project aims and efforts so far to the research team behind the Engaging Crowds project (funded by the AHRC as part of the Towards a National Collection research programme). This presentation was intended to 1) signpost opportunities for Engaging Crowds researchers to participate via the Humanities Commons group and the pre-Book Sprint surveys we conducted; 2) discuss the potential for the output of the Collective Wisdom book sprint as a useful resource for the Engaging Crowds team in their ongoing work on volunteer engagement with heritage materials, helping to sure effort isn't duplicated, and that we are learning from one another's work.
Year(s) Of Engagement Activity 2021
 
Description Project website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Created a website to collate project information, including goals, activities and events, progress updates and news, and contact information.
Year(s) Of Engagement Activity 2020
URL https://collectivewisdomproject.org.uk/
 
Description Save the Date: Collective Wisdom Workshop, 20-22 October 2021 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact This blog post announced the Collective Wisdom workshops and emphasized that the workshops would focus on the critical themes resulting from the Collective Wisdom Book Sprint and the Collective Wisdom Handbook. The post also reinforced the values and methods of the Collective Wisdom project; specifically, an emphasis on inclusivity and information sharing at the outcome of each step of the grant activities. As a result, the project team received inquiries for particioation in the workshops and requests to share the key outcomes of the workshop for the broader benefit of the field.
Year(s) Of Engagement Activity 2021
URL https://collectivewisdomproject.org.uk/save-the-date-collective-wisdom-workshop-20-22-october-2021/
 
Description Values Statement 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact The Values Statement provided framework for both sprint participants as well as members of the public as to the commitment of the PI and Co-Is to a set of overarching values. These values were consulted as part of the design of the project and execution of activities proposed.
Year(s) Of Engagement Activity 2020
URL https://collectivewisdomproject.org.uk/our-values/