Picturing the Social: transforming our understanding of images in social media and Big Data research

Lead Research Organisation: University of Sheffield
Department Name: Information School

Abstract

Social media have transformed our understanding of the modern world, but a detailed understanding of the role of images in effecting these transformations has been neglected compared to other forms of data. From a social science perspective this explosion in rich online communication exchanges opens up possibilities for better understanding these new forms of social expression through organic data rather than through more traditional forms of social science data. The scale of social media use also makes it important for social science research concerned with Big Data. Through innovative, cross-disciplinary work, including with industry, using state of the art technical infrastructures, we will develop a framework for analyzing the value of social media images to better understand these new social worlds. Specifically we are interested in how these new forms of social expression can be understood through studying their technical infrastructures. Including a focus on users, we investigate how a bottom-up understanding of social media users can better inform how these structures are experienced. We are concerned with devising comprehensive methodological and theoretical strategies for the capture and interpretation of social media image data. This will be done at different scales, using both quantitative and qualitative methods. We are interested in finding ways to enable the study of images to become a more central focus within social media and Big Data research and will investigate how such a shift in focus may have an impact on key stakeholders and end-users.
People currently produce and use a lot of images as part of their everyday lives, primarily using digital media and online social media platforms. Amongst UK Internet users, posting photos has significantly increased in recent years. The taking and sending of photos are now the two most prevalent activities after text messaging. People send, collect, organize, and show images of things they value, including already existing images that they value, often because they are valued in a wider collective cultural context. Such images can be meaningful as documents, forms of play, acts of witnessing, as aesthetic artefacts, historical records, and/or as tokens of collective identity. By focusing on the structures, users, and the content of social media images, the project will produce cross-disciplinary theoretical and methodological innovations for understanding how images function as pictures of the social world and within the networked relationships of social media. These conceptual devices and methods will address the quantitative magnitude and the qualitative intensity of social media image production and circulation. They will include the development of online tools to capture and study both small-scale and large-scale image datasets.
We suggest that for social media and Big Data research to comprehensively and rigorously address the huge shift in communication technologies, it needs to be able to speak to and account for the complexities and fluid nature of the current information ecosystem. In order to understand these new forms of social expression, focusing here on the production, use and circulation of images these elements have to be studied and understood together. The technical and software infrastructure that shapes how users are able to communicate through social media and how this enables other users to make sense of this content needs to be included in this. In the emerging fields of social media and Big Data research emphasis is placed on the volume, veracity and variety of data, rather than its meaning or depth. Consequently, critiques of Big Data have highlighted that a more nuanced approach is required. Approaches sensitive to multiple data characteristics including an attention to 'deep data', will allow for a fuller understanding of society in a complex world. We argue this can be achieved in a particularly rich way by looking at social media images.

Planned Impact

We see the beneficiaries of this research as being the government, policy-makers and think tanks, the private sector and the academic community. We see strong potential for Economic and Societal impact, specifically for the planned research to make a contribution to evidence-based policy making and influencing policy (at local, national and international level), to enhance research capacity, knowledge and skills of businesses and organizations, and contribute to the training of skilled people in non-academic professions. Ensuring potential beneficiaries have the opportunity to engage with our research, we plan to organize a number of different dissemination activities to develop a strong stakeholder engagement prior to impact:

1. Events: we plan to organize three project workshops for our identified stakeholder communities: those from the government and the policy-making community, the private sector and academic stakeholders;
2. Press activity: we aim to actively engage with the press and broadcast media through the writing of targeted media briefings and press releases as well as invited media appearances where appropriate;
3. Communications activity: we will translate our research in an accessible and tailored format for our identified specific user groups.
Most of these activities will be through the Visual Social Media Lab.

In our Pathways to Impact document we outline our impact objectives per group, as well as our existing strong track records in relation these different stakeholder groups, and finally, we give a number of examples of what measurable impact indicators might look like.
In organizing the three project dissemination events (in London, Sheffield and Manchester), aimed at maximum engagement and therefore capping numbers at around 30 participants for each event, we will draw on our networks and lists of participants from previous impact events. We will also actively engage with the media in sharing our work and news of our events. The PI has recently been invited to Radio 4's flagship programme 'Start the Week' to contribute on the subject of social media. Through potential future invitations of this kind, further public engagement will be enabled. Moreover, all project team members are experienced public speakers and have made national and international media appearances.

The Visual Social Media Lab will be the public face of the project, and will facilitate knowledge exchange by enabling a strong identity and online presence for our research. We will produce six easy-to-read rapid response reports that all project team members will contribute to. We will seek advice from our Advisory Board, from stakeholders through our events, and the wider academic community. Our social media engagement will be through dedicated social media accounts managed by specific team members.

We will develop and agree on an impact strategy in the first three months of the project and seek advice from our Advisory Board to confirm the effectiveness of our planned strategy and help shape future activities. We will organize events through online booking systems (e.g. Eventbrite) and collect data on participants prior to the event and well as through an exit survey to determine if the needs of our identified user communities have been met and if our research has been communicated clearly and productively. We will also monitor and collect metrics for our website as well as our social media engagement.

Impact indicators could include:
- Evidence that stakeholders are actively engaging with and using our research, for example by directly citing it or by inviting project team members to present/brief on the research;
- To see a take-up of our developed research methodology by the identified stakeholders;
- To see a take-up of the project research tool for collecting and processing social media image data;
- In general, to see a wider engagement with social media images, which can be traced to our research.

Publications

10 25 50
 
Description Our project has established an important new space for doing innovative interdisciplinary research on social media images. We have developed a clearer understanding of the key challenges and have developed a new theoretical and methodological framework better able to holistically deal with a new online information system, in which images now play a fundamental role. We set up the Visual Social Media Lab (VSML) to facilitate knowledge exchange and to enable a strong online presence for this as well as our ongoing research. We have developed a free online tool for collecting images and data from different social media platforms, available for free via the Statistical Cybermetrics Research Group at the University of Wolverhampton as part of the Webometric Analyst software: http://lexiurl.wlv.ac.uk/. With industry Co-I Pulsar, we further developed their state of the art technical infrastructure to benefit the project and to embed key ways of working beneficial to academic and commercial users. Training in both tools was offered as part of this project. Insight from the project has also been incorporated into the PIs postgraduate teaching on social media and through international guest teaching opportunities. Substantive research has focused on everyday image sharing and exceptional image sharing across two key strands of research. Everyday image sharing was explored via a 12-month digital ethnography, which used three case studies to consider different users, practices and motivations in relation to online photographic sharing. Participants were recruited from a range of age groups, from rural and urban settings, and both professional and amateur photographers. The aim of all three case studies was to explore how photographs on a similar theme - namely, the city of Sheffield - are used to fulfill different personal and commercial objectives. This strand of research has resulted in a six-month secondment for the RA at the Food Standard Agency after the project completed; to explore the potential for different methods for analysing visual social media data within government. Exceptional image work focused on the images of three-year old Syrian refugee Alan Kurdi (originally misidentified as 'Aylan') taken by a Turkish press photographer after his body was found on Bodrum beach in Turkey, on 2 September 2015. These images were distributed to the mainstream media via wire services and subsequently circulated widely on social media, triggering further online and offline responses. These responses appear to have been driven by the accumulated momentum of the images as they were circulated and used across different contexts. This rapid response project, examining nearly 3 million image posts - involving researchers from the VSML and others - sought to explore this phenomenon and uses this case to consider the wider role of social media in the contemporary creation of iconic images. Findings from this project were published as a rapid response report in December 2015, which received wide international coverage. This wider strand of research has also resulted in an international collaboration with First Draft News for a funded PhD project to start in 2017. Finally, we sought to better understand users through focus groups and a large-scale survey, focused on Facebook, Instagram, Twitter, Snapchat and WhatsApp. We asked questions about both everyday image sharing as well as exceptional image sharing. The results of the focus groups and the survey have highlighted the importance of doing this work longitudinally in order to better understand how online image sharing practices shape society over time.
Exploitation Route The research has been taken forward in different ways beyond academia and we will continue to build on this. Results from our rapid response report have been taken up in numerous ways, including by the World Economic Forum, which the PI has been involved in since 2013. The project RA has since completing the project worked at the Food Standards Agency as part of a six-month secondment and the hiring of the project RA. Through the Visual Social Media Lab our research also attracted strong interest from industry, including citing our work in key international reports. These developments connect to these original impact indicators: (1) Evidence that those working in social research in government, policy-makers and think tanks are engaging with and actively using our research, for example by directly citing it in policy documents and briefings, or by inviting project team members directly to present/brief on the research; (2) Inclusion of the role of social media images and the visual web as one of the key priorities for the WEF's Global Agenda Council on Social Media's research and engagement agenda for 2014/2015; (3) Evidence that those working in industry are engaging with and actively using our research, for example by directly citing it or by inviting project team members to industry events and conferences; (4) In general to see a wider engagement with social media images, which can be traced to our research. In terms of this final more general aim, there is clear longer-term impact of the establishment of the Visual Social Media Lab as part of this award. Specifically, since 2018, in the area of mis- and disinformation and Visual Media Literacy. Tackling misinformation is a pressing global challenge for democratic processes. Our research has indicated that, despite their apparent power, images have been largely overlooked by researchers, policy makers and industry as a key part of the problem. This has led to the creation of key Visual Media Literacy resources developed in collaboration with Canadian NGO CIVIX that have been used in the run up to elections in Canada and Colombia in 2019 and have reached 1.2 Million young Canadians as part of the Canadian elections. This work has recently attracted funding from the Canadian Ministry of Heritage to research the impact of the 'Questioning Images' resource. Further resources will be developed in 2021 and Covid-specific resources were also developed in the UK context during the first national lockdown in 2020. This work on visual misinformation has also enhanced the training of verification practices in journalism. This latest VSML research has shaped national and international policy thinking on this issue.
Sectors Agriculture, Food and Drink,Communities and Social Services/Policy,Creative Economy,Digital/Communication/Information Technologies (including Software),Education,Government, Democracy and Justice,Culture, Heritage, Museums and Collections

URL http://visualsocialmedialab.org/
 
Description The rapid response report on the Alan Kurdi images realises the funding's transformative potential by emphasizing the need for interdisciplinary, risky and innovative approaches to big data image analysis of the refugee crisis, bridging the gap between academia and industry. The report brings together 15 contributions, featuring contributions from the project team, Google News Lab, Tow Center for Digital Journalism, Columbia University, and WITNESS. The 74-page report has contributed to shaping the World Economic Forum's (WEF) 2016 Annual Meeting at Davos. This remains the foremost creative force for engaging the world's top leaders in collaborative activities focused on shaping the global, regional and industry agendas. It shaped the global agenda in two key ways: (1) PI's involvement in WEF's September 2015 Industry Strategy Meeting, which brings together 300 Strategy Officers from leading companies, in order to shape the industry agenda for the Forum's Annual Meeting. Recommendations included in the final report confirm that understanding the Alan Kurdi image is a challenge that 'requires an interdisciplinary approach to develop analytics that help understand what the social explosion of [such] visual content actually means and how a single image drives action.' (2) PI's inclusion in a WEF produced film on civic participation, which is part of the 'Shaping the Future Implications of Digital Media for Society' report, launched at Davos. In this film the image is highlighted as a key moment in the refugee crisis in terms of shaping policy and understanding civic participation. This research has also been taken up by industry, through the research of Susan Etlinger a leading industry analyst at Altimeter Group (a Prophet company, based in San Francisco). This company advises global organizations on brand strategy, innovation, data strategy and a host of other topics. Our report was included in Altimeter's Digital Trends Report for 2016, as well as the 2016 report 'Image Intelligence: Making Visual Content Predictive', referring to work from the project and quoting the PI. Both Altimeter publications were sent to 50,000 global companies, highlighting the need for a new discipline for understanding the visual web, including innovative analytical approaches to better understanding the role of social media images. The Visual Social Media Lab is identified as a place where such new approaches are being developed. The rapid response report was launched in mid-December 2015 and generated significant international media coverage (over 40 articles), triggering further international debate on how the research had shown how the discourse around the refugee crisis had changed. Results from the project were also included in a documentary about these images for Dutch broadcaster VPRO and involved an interview with the PI. This programme was broadcast in March 2016. The report has also been discussed extensively in data journalism circles (such as a panel at the International Journalism Festival in Perugia in April 2016, which included Claire Wardle (Tow Center for Digital Journalism), Craig Silverman (Buzzfeed), Carlotta Sami (UNHCR) as well as members from the project team). To date the report has also become the key academic text on the images of Alan Kurdi with 50 citations (based on Google Scholar) as of March 2019. The original proposal had identified 'rapid response' as a key mechanism through which this research could impact discourses around important events. Through a highly innovative approach, this research contributed to shaping the global agenda and transforming public understanding of the refugee crisis through social media images, as these events were still unfolding. Thus, fully exploiting the potential for international impact connected to one of the most urgent contemporary global crises. A year later work from this project was updated in a collaboration with Oxfam to explore how this image had shaped conversation on Twitter about the refugee crisis one year on. The work also formed the basis of the first PhD project of the VSML (on the potential for iconographic methods using large image datasets 2014-2019), which was recently completed and which will be able to contribute to further impact in due course. Finally, the six-month secondment (November 2016 - April 2017) for the RA at the Food Standards Agency (FSA) after the project finished highlights another strand of impact. By hiring Anne Burns to work with social researchers within the FSA and continuing to work closely with the PI, who co-managed this six-month research opportunity, the methods that have been developed as part of the Picturing the Social project have a real chance of being embedded and adopted long-term by a government body. This work in particular could in the long-term contribute to increasing the effectiveness of public services and policy developed within this government agency. After the completion of the secondment Anne Burns was hired by the FSA as a Data Scientist, confirming this potential and opening up further possibilities for long term impact of the methods developed by the VSML. The longer-term impact of the establishment of the Visual Social Media Lab through this initial funding is evident in the ongoing work that the VSML has done since funding finished in 2016. Specifically, since 2018, in the area of Visual Media Literacy. Tackling misinformation is a pressing global challenge for democratic processes. Our research has indicated that, despite their apparent power, images have been largely overlooked by researchers, policy makers and industry as a key part of the problem. This has led to the creation of key Visual Media Literacy resources developed in collaboration with Canadian NGO CIVIX that have been used in the run up to elections in Canada and Colombia in 2019 and have reached 1.2 Million young Canadians as part of the Canadian elections and 76,000 young Colombians as part of the 2019 local elections. This work has recently attracted funding from the Canadian Ministry of Heritage to research the impact of the 'Questioning Images' resource. Further resources will be developed during 2021. In the UK context, during the first lockdown in 2020, work (again in 'rapid response' mode) was done with a local school Trust in Manchester to develop a teachers guide to understanding Covid-specific (visual) mis- and disinformation. This resource was shared with 80 teachers at the Trust and later, though the Director for Education in Manchester, with all schools in Manchester. Research on the impact of this resource has shown a marked impact on both teachers and pupils within the Trust. This work on visual misinformation has also enhanced the training of verification practices in journalism. This latest VSML research has shaped national and international policy thinking on this issue. Overall this continues to show that the most significant impact of this award was the setting up of the Visual Social Media Lab, which has realised the transformative potential of the original award.
First Year Of Impact 2015
Sector Communities and Social Services/Policy,Digital/Communication/Information Technologies (including Software),Education,Government, Democracy and Justice
Impact Types Cultural,Societal,Policy & public services

 
Description ESRC Festival of Social Science funding
Amount £2,000 (GBP)
Organisation University of Sheffield 
Sector Academic/University
Country United Kingdom
Start 11/2014 
End 11/2014
 
Description MARIE SKLODOWSKA-CURIE ACTIONS
Amount € 100,000 (EUR)
Organisation European Commission 
Department Horizon 2020
Sector Public
Country European Union (EU)
Start 09/2015 
End 08/2016
 
Description Manchester School of Art Collaborative PhD funding
Amount £45,000 (GBP)
Organisation Manchester Metropolitan University 
Department Manchester School of Art
Sector Academic/University
Country United Kingdom
Start 04/2015 
End 09/2018
 
Description Visual Media Literacy for Combatting Disinformation: Analysing How Teachers Used the Questioning Images Resource During the 2019 Canadian Federal Elections
Amount $214,837 (CAD)
Organisation Government of Canada 
Sector Public
Country Canada
Start 02/2020 
End 03/2021
 
Description White Rose Doctoral Training Partnership Collaborative Awards 2017/2018
Amount £60,000 (GBP)
Organisation White Rose College of Arts and Humanities 
Sector Academic/University
Country United Kingdom
Start 09/2017 
End 09/2021
 
Description White Rose ESRC Network (for 3 PhD projects)
Amount £135,000 (GBP)
Organisation White Rose University Consortium 
Sector Academic/University
Country United Kingdom
Start 09/2015 
End 09/2018
 
Title Webometric Analyst - image data collection and analysis tools 
Description Webometric Analyst can be used to download images from Twitter or Tumblr. It will also create lists of the most frequently downloaded identical images. The process takes two stages: first download the tweets; and second, extract, download and count the images in the downloaded tweets. 
Type Of Technology Software 
Year Produced 2015 
Impact All code developed for the Picturing the Social project was added to Webometric Analyst and the core features documented for other researchers. This strategy of combining programs from different research projects into one tool is designed to ensure that they are not forgotten but are automatically maintained and upgraded along with the software. 
URL http://lexiurl.wlv.ac.uk/
 
Description Aylan Kurdi report press release (December 2015) 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Media (as a channel to the public)
Results and Impact A large international audience was reached with our press release for the 'The Iconic Image on Social Media: A Rapid Response to the Death of Aylan Kurdi' report. The report received substantive media coverage in the national and international press, in online publications and the technology press.
Year(s) Of Engagement Activity 2015
URL https://www.sheffield.ac.uk/news/nr/aylan-kurdi-social-media-report-1.533951
 
Description Government Social Research Social Media Research Group presentation (14 September 2015) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Invited speaker for Government Social Research unit's new Social Media Research Group, 14 September 2015 (with Francesco D'Orazio), to talk about 'image analysis using social media data'. This talk sparked further interest from this group in the potential of analysing social media images within government research. This talk focused on implications for policy making.
Year(s) Of Engagement Activity 2015
 
Description Government Social Research Social Media Research Group presentation (3 September 2015) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Policymakers/politicians
Results and Impact Invited speaker for Government Social Research unit's new Social Media Research Group, 3 September 2015 (with Francesco D'Orazio), to talk about 'image analysis using social media data'. This talk sparked further interest from this group in the potential of analysing social media images within government research. This talk focused on methods.
Year(s) Of Engagement Activity 2015
 
Description MediaLogica programme 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact Farida Vis was interviewed for the MediaLogica programme (for the VPRO, a national Dutch broadcaster) about 'The Iconic Image on Social Media: A Rapid Response to the Death of Aylan Kurdi' report. This one-hour long special focused on how this image had contributed to the change in public opinion on the refugee crisis.
Year(s) Of Engagement Activity 2016
URL http://www.human.nl/medialogica/2016/een-zee-van-beelden.html
 
Description Social Data Meetup (Farida presentation, August 2015) 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact Farida Vis was an invited speaker at the Social Data Meetup, and gave a presentation on 'The Visual Social Media Lab: one year on' at the Social Data Meetup 'Beyond the Research Lab'. This was held at the Google Campus, London, 3 August 2015. This talked sparked further interest in the Visual Social Media Lab and a number of participants have since followed up with specific queries about the work of the lab.
Year(s) Of Engagement Activity 2015
URL http://www.meetup.com/socialdata/events/223759939/
 
Description Visual Social Media Lab newsletter 
Form Of Engagement Activity A magazine, newsletter or online publication
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact More than 500 people were reached with the publication of the first newsletter of the Visual Social Media Lab. This generated further interest and engagement with the lab and the Picturing the Social project.
Year(s) Of Engagement Activity 2015
URL http://visualsocialmedialab.org/news
 
Description Visual Social Media Lab website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Between August 2015 and March 2016 more than 10,000 unique visits have been recorded for the Visual Social Media Lab website since it was launched.
Year(s) Of Engagement Activity 2015
URL http://visualsocialmedialab.org
 
Description WEF Civic Participation video 
Form Of Engagement Activity A broadcast e.g. TV/radio/film/podcast (other than news/press)
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact 'Civic Participation' is an episode of a seven-part video series for the World Economic Forum's 'Shaping the Future Implications of Digital Media for Society' project. Click on http://reports.weforum.org/human-implications-of-digital-media-2016/ to explore the full 'Digital Media &Society Report'.
Year(s) Of Engagement Activity 2016
URL https://www.youtube.com/watch?v=SfI35O6WtXE