Social Media Analysis for Social Geography

Tweet My Street (TMS) is a project within the SiDE Social Inclusion Through the Digital Economy Hub that has been exploring the extent to which data derived from Twitter can reveal more about spatial and temporal behaviours, and the meanings attached to these locally. This has been done with a longer-term view to supporting the coproduction and delivery of local services, effective complaint mechanisms and horizontal community support networks.
The project has exposed, and is tackling, 2 of the main barriers to realising the full potential for the use of Twitter (and related social media data) in the social sciences:
i) The lack of availability of tools that can allow social scientists to extract meaning from social media data. Capturing, storing, querying and visualising data derived from sources is very challenging given: a) the high rate at which it is being generated, b) the vast scale of historic data, c) the complexity involved in querying the data
ii) The lack of understanding of methodological issues in the analysis of social media data, for example ethics and the validity of any conclusions drawn from the skewed sample of the population that social media users represent
As these issues cannot be addressed by expertise in one field alone, the project is a multi-disciplinary collaboration involving social geographers, computer scientists and the computer industry (Red Hat Inc. who fund a PhD student working in the group, and take an active part in project meetings). The involvement of the latter represents the fact that the outputs of the research have relevance in many other areas, including marketing, e-commerce and policing.
To date the project has generated some promising early results, including:
- a scalable, cloud-based software infrastructure for the real-time and historic analysis of twitter data. We have demonstrated that this can scale to process data at the full rate that tweets are generated globally.
- a web-based tool to provide an easy-to-use way for social scientists to query social media data and visualise the results, for example on a map.
Just as importantly, it has established a successful virtuous circle for collaboration: the social scientists use the software tool to analyse twitter data in order to advance their case study; this leads to new requirements on the tool which the computer scientists attempt to address; the tool is enhanced and passed to the social scientists. The cycle then repeats.
This project will build on the work carried out in the SiDE Digital Economy Hub, leveraging its results and infrastructure to extend it in five key ways:
- to extend the software tool to include connection analysis (e.g. flows of tweets, changes in connection graphs following an event or intervention)
- to add to the collaboration a group of statisticians to explore the use of modern statistical techniques to extract understanding from twitter data, including social graph analysis, and modelling of network dynamics and meme propagation
- a new case study to explore the use of the tool to augment existing social science methods in connection analysis [Mike's work summarised here]
- a new case study to use the tool to augment existing social science methods, to understand the way third sector organisations are using social media and the barriers they have to overcome to do so
- to run an event to disseminate the results, the methodologies and give training on how to use the tooling
All the software generated by this project will be made freely available to the community as open-source software.

The analysis of social media data has a range of potential beneficiaries:

Organisations in the public, private sector and 3rd sector who could improve their services and products though exploiting the information encapsulated in social media data. Early demos of the existing infrastructure at conferences and meetings have attracted interest from a range of organisations including the police, the home office, and a utility company. We are building relationships with these contacts , holding meetings to gain detailed feedback on the research outputs, and what would be needed for them to exploit it. Over time, we will also target companies in other sectors, for example marketing and advertising. We are making the infrastructure available as open-source in order to encourage take-up, and therefore impact.

Public Sector: Through SiDE and a TSB project we have excellent links to Newcastle City Council and have ongoing informal discussions with them on the potential value of using social media analysis to better understand it's citizens, and the impact of the services it delivers.

An aim is to look for opportunities to commercialise the outputs through licencing or a start-up company. Red Hat fund a PhD student working on the project, and their main contact gives regular input into the infrastructure design from a computer industry best-practise perspective.