Measuring the Societal Benefit of Online Civic 'self-help' Sites
Lead Research Organisation:
University of Manchester
Department Name: Social Sciences
Abstract
The research will be conducted in four main phases. The first phase will involve an assessment of the existing work in the academic literature and also more specific reports by MySociety and similar groups to profile their online users and the societal value of their sites. We will identify the key gaps in practitioner and academic knowledge, both methodologically and theoretically, and specify our research questions and the sites we will study. Based on this review in the second phase of the research we will design the analysis to provide the type of rich qualitative and quantitative evidence to address our central research questions regarding measuring the sites' impact. In the third phase of the research we will carry our empirical data collection. This will consist of four two-wave panel surveys fielded on the MySociety sites - They Work For You, Fix My Street, Fix My Transport and Write To Them. The surveys will be used to identify the new users and those that are less involved in civic and political matters and from a non-typical participatory background. All individuals participating in wave 1 of the surveys will be invited to take part in an extension to the analysis by completing an online time diary over a three month period following their interaction with the site. First time users will be targeted in particular with follow up invitations. The time diaries will be filled in ideally on a weekly basis and track users social, political and communal activities. Time diarists and those completing wave 1 of the survey will then be invited to complete wave 2 once the time diaries are completed. This will measure levels of social and political trust, social connectedness, and feelings of confidence in their ability to solve their own and other/community/political problems. In the final phase of the research we will conduct online focus groups with selected time diarists and triangulate with survey and time diary data to address the key questions of what motivated participation on the site, if first-time use was converted into further activism how and why that occurred. Once the results are produced we will prepare a report for MySociety and dissemination across the voluntary sector that will summarize our findings and provide recommendations about what tools and actions are most likely to be undertaken by first time users of campaigning and community sites, and how these act as triggers to create the skills confidence and networks to allow them to continue to undertake self-initiated activism. We will also include recommendations on how this 'conversion' can be monitored and measured by organizations. Finally we will prepare a paper for academic publication that will utilize the data to present the first qualitative over time study of online users of these public service sites. The panel element will allow us to gain a unique insight into how far an individual's visit to the site influenced subsequent activities and particularly the extent to which, by dealing with a personal problem or local concern they felt more able to deal with problems in their own lives and their neighbourhood/society more generally, how far they achieved things they could not otherwise have done, were better equipped to solve subsequent problems when they arose.
Planned Impact
Over the last five years there have been an explosion in the number of websites that promise to help people become more democratically engaged, or more empowered as citizens. These civic 'self-help' websites are designed to help people undertake a range of important tasks that include contacting politicians, sending complaints to public bodies and making government more accountable and transparent by opening up information for public scrutiny. The British charity mySociety has been a global pioneer in this field, developing new kinds of service and running domestic websites with traffic of almost 5 million annual visitors. While we have evidence of who uses the site, how it is used and how often, we lack a detailed understanding of the subsequent and wider societal impact these services have on their visitors. This project will seek to provide this understanding through a systematic quantitative and qualitative study of mySociety users over a three month period. The work will benefit the following groups in several key ways:
* Social policy makers: Politicians and public servants would be given an important new source of evidence regarding the role that digital services and particularly these civic portals can play in improving the quality of public life and the health of communities. This is an area that has been underexplored to date due to lack of good quality data on the users of the sites and systematic analysis of their effects. If shown to have positive consequences for strengthening citizenship and increasing democratic accountability then the project would help policy makers to more efficiently channel government spending toward these types of initiative.
* Site developers: Innovators in the digital realm who run these sites lack clear, reliable evidence to assess what impact they are making and whether some types of facilities are better than others for promoting the outcomes they seek. This study will help them to pinpoint where to invest their resources to ensure maximum return on their investment.
* Funders, specifically philanthropic and public sector funders of innovative social projects need better evidence and comparable metrics to understand if their current investments in socially positive digital technologies are yielding the rewards that would be desired. This is an especially important group to serve, given that funding in these fields is at an all time high.
* Academics and researchers in the digital democracy and civic activity will gain access to a new and unique data source that will allow for a better understanding of the extent to which the internet and mobile technologies are or are not impacting on the quality of community life and democratic practices.
For more information on how we will reach these groups, please see the Pathways to impact document.
* Social policy makers: Politicians and public servants would be given an important new source of evidence regarding the role that digital services and particularly these civic portals can play in improving the quality of public life and the health of communities. This is an area that has been underexplored to date due to lack of good quality data on the users of the sites and systematic analysis of their effects. If shown to have positive consequences for strengthening citizenship and increasing democratic accountability then the project would help policy makers to more efficiently channel government spending toward these types of initiative.
* Site developers: Innovators in the digital realm who run these sites lack clear, reliable evidence to assess what impact they are making and whether some types of facilities are better than others for promoting the outcomes they seek. This study will help them to pinpoint where to invest their resources to ensure maximum return on their investment.
* Funders, specifically philanthropic and public sector funders of innovative social projects need better evidence and comparable metrics to understand if their current investments in socially positive digital technologies are yielding the rewards that would be desired. This is an especially important group to serve, given that funding in these fields is at an all time high.
* Academics and researchers in the digital democracy and civic activity will gain access to a new and unique data source that will allow for a better understanding of the extent to which the internet and mobile technologies are or are not impacting on the quality of community life and democratic practices.
For more information on how we will reach these groups, please see the Pathways to impact document.
Publications
Cantijoch M
(2016)
'It's not about me, it's about my community': A mixed-method study of civic websites and community efficacy
in New Media & Society
Gibson, R.
(2014)
'The Third Sector and Online Citizen Empowerment: the Case of mySociety'
| Description | This research project examined the impact of using new forms of 'civic technology' on individuals' levels of community and political engagement. We focused specifically on the sites of mySociety, an online organization that promotes citizen empowerment by providing digital 'do-it-yourself'' (DIY) tools that people can use to better understand how the political system works and take action to solve local problems. The work had three main objectives: (1) to understand whether and how such sites mobilize people to participate in their communities and also in the wider political system; (2) to adopt a unique mixed-methods approach that would link individual stories with a broader picture of overall use of the sites and provide a more complete account of their impact over time; (3) to offer practical insights to mySociety about the effectiveness of their sites and recommendations for improvement. In terms of mobilization, our key findings were that although use of the sites does not appear to be linked to changes in levels of political engagement over the short-term, regular use does appear to stimulate a 'virtuous circle' of increasing involvement in community affairs and the value of collective action at the local level that we termed 'community efficacy'. Methodologically we advanced the study of online participation by developing an innovative mixed methodology that directly linked time diaries completed by several hundred people over a three month period with focus group discussion and quantitative evidence from surveys of mySociety users. This provided a much richer and more reliable insight into how individuals use mySociety resources and how far it directly or indirectly prompted subsequent changes in their attitudes and behaviour. In particular it provided the basis for our identifying the sense of 'community efficacy' that the sites help to develop among users. On a practical level we identified an opportunity for mySociety to expand its impact through provision of more collective spaces on its sites where users could find others with shared interests and pursue action together. We also revealed significant differences between first-time and regular users of the sites that suggest the sites may not be working as effectively as possible to retain harder to reach groups that are less engaged politically. mySociety were particularly interested in our findings about the links between the use of mySociety tools and community efficacy and noted that our "key findings would be of interest to [other] community groups" who could promote mySociety tools "to individuals interested in their communities but without the initial efficacy to use them." Finally, through this research we have shown how academics and democratic practitioners can work together to conduct theoretically informed and methodologically rigorous research that can be used to identify concrete steps that can increase the impact of their work and result in work published in major international journals. |
| Exploitation Route | Our findings will inform future academic work in this area by providing unique new data on the users of 'civic technology' sites in the UK. Also our methodology is one of the first to integrate time diary evidence with focus group discussion and aggregate survey data to understand the impact of these e-democracy resources on individuals and wider society and provides a template for future researchers to use. Our methodology is also being put to use by practitioners. Steven Clift, Director of the e-Democracy project in Minnesota who attended the launch of our project report requested a copy of our survey methodology to help develop his current work in this field. Our recommendations regarding the need for communal spaces in our project report have been actively considered by mySociety. More generally our findings that the sites can help to increase levels of community efficacy provide mySociety and similar groups with the basis for securing support and funding to promote their work. The Democratic Society contacted the PI following publication of the report to discuss a possible research project. Specifically, discussions were held with staff of The Democratic Society and one of the Board of Governors who expressed interest in working together to develop a project that could help identify and produce practical initiatives on how social media use might help to engage young people in politics. |
| Sectors | Government, Democracy and Justice |
| URL | https://www.mysociety.org/files/2014/12/manchester.pdf |
| Description | The findings of the research are now published as an online report by the project partner mySociety and are available as download from their site. mySociety have promoted the findings via social media and also sent short summaries of the findings to a range of government bodies, thinktanks and other NGOs that they have links with. A link to the report was also emailed to all the participants in the research. The report was also picked up and promoted to an international audience through an independent blog OpeningParliament.org, an online forum intended to help connect the world's civic organizations engaged in monitoring, supporting and opening up their countries' parliaments and legislative institutions. http://blog.openingparliament.org/post/116930271448/openparl-news-brief-april-20-2015 mySociety staff and their users were the primary consumers of the findings. We set out a series of practical and also methodological recommendations for their practice from the research as part of an executive summary. We discussed these recommendations in person with the Director Tom Steinberg and Research Director Rebecca Rumbul and by email. Our findings were seen by the Director as important in highlighting the need for an expansion of the sites to include a more collective space for users to find one another locally in order to discuss and possibly collaborate on activities offline. The formal launch of the report in London brought together staff of mySociety, a number of users of the sites and other professionals seeking to use online tools to promote democracy. The key findings of the report were presented to the audience and then discussed afterward. This discussion led to a series of specific points being raised for mySociety to take forward and follow-up. These were: • The findings revealed that use of the sites is skewed toward those with greater socioeconomic resources, ethnic minorities and women were also under-represented. This indicates that more needs to be done to appeal to a wider demographic. • The need for closer analysis of first time users was evident since they do appear to have a more diverse demographic profile than regular users. Thus a further recommendation that arose from our work was that mySociety could look more closely at how it might widen its reach in this regard? • When measuring the success of mySociety sites it is important to place this in context and particularly to understand geographic patterns in terms of where users are concentrated. Is it the case that particular local authorities are more responsive to requests for information or action by users? If so, does this then increase the likelihood that users will use the sites again, creating something of a virtuous circle? • The collective aspect of the site is under-exploited. Users of the site express a desire to help their local community and based on our over-time analysis this appears to be developed further if they are regular users of the sites. It would be worthwhile, therefore, for mySociety to add some more collective features to the site that would allow people to debate, discuss issues at minimum. Taking this further it might be possible to offer a facility whereby people can then take action with others to resolve an issue (online and offline). More specifically there could be an 'ask others' button whereby individuals can start a community action. • In the FixmyStreet site that mySociety run to help people address transport problems in their local area it would be useful to include a facility to allow councils to respond to complaints raised and why a specific problem has not yet been addressed. The findings have been presented in conference or workshop papers to academic audiences in the UK and internationally (France and the U.S.) Our findings regarding the impact of mySociety sites on communal activities and local participation were received with great interest. In particular our unique methodological approach in which we link three kinds of longitudinal data to present a multi-dimensional view on the mobilizing effect of mySociety sites has attracted considerable attention and resulted in an invitation to submit a paper for a special issue of New Media and Society, one of the leading journals in the Internet and Politics field. The paper has now been accepted. Rebecca Rumbul Director of Research at MySociety said of the Journal output and scientific impact of the work "I am really pleased that the study is going into journal publication, as I think it's an important contribution to the sparse literature on civic tech, and will definitely have further reaching benefits, although of course those will be slightly longer term." Reflecting on the wider impact of the work for mySociety Dr. Rumbul wrote in an email (28.08.15) to the PI that "The report has been really useful internally in helping to inform our own research agenda and was integral in identifying some areas of potential interest both when looking at our sites in the UK, and also when we were examining implementations abroad." She also noted that mySociety had been in a state of transition in the preceding 6 months given the departure of Director, Tom Steinberg who commissioned the work. This has meant fuller consideration and implementation of the report's findings and recommendations needed to wait until the new CEO was appointed (which has now happened). One further aspect of the report that she identified as likely to be particularly useful going forward were the opportunities it highlighted for potential collaboration by mySociety with other groups. As a small NGO, the budgetary resources of mySociety are limited. This report was significant in in highlighting the link between site usage and 'real world' activities provided a basis for thinking about how mySociety could engage with larger partners to extend its reach real-world and community activity. Since the conclusion of the award mySociety have informed us that they have had 238 page views of the research paper we uploaded to their site, with 187 of those being unique users. A follow up email request for feedback among those who participated in the more qualitative component of the research - the time diaries - prompted a range of responses from respondents in which they reflected on their experience and how it had affected their lives. One respondent wrote "I found the project interesting as it made me more aware of what I was contributing to my community and made me feel I wanted to do more." Another commented that "The survey was a positive process for me, in that it tended to discipline my thoughts and the structure of my daily activities the process did have intellectual benefits for me" A third noted that "I think, that on refection, it was an excellent way for the participants to focus on issues raised and to extrapolate the topics relevant to 'mysociety'" During 2016 the project methodology and findings were used to inform a training programme for community and third sector organizations conducted by researchers at the University of Manchester. The project was entitled Knowing-how: Co-producing a social research training programme for the Third Sector" and our materials were used in Workshop 4 to teach participants about the value of mixed methods. During 2017 the methodology was discussed with BBC R&D staff working in the area of audience measurement. The associated publication from this project that appeared in New Media and Society in 2015 was distributed to them at their request. Members of the BBC read the article and were interested to adapt the methods used to assist their research into consumers of their content. Gibson and Cantijoch were invited to develop a proposal with the BBC as an external partner. The proposal was submitted for an ESRC Impact Acceleration Award and was successful leading a collaborative project in 2018 using the same mixed methods used in the MySociety project. The project examined BBC news audiences and the extent to which public broadcasting content affects knowledge levels among the least and most educated citizens. The project produced a report for the BBC "Knowledge Gaps and the Role of the BBC in the era of Fake News" which was presented at Broadcasting House in London in December 2018. The report was well received and is now forming the basis for discussion about a larger grant collaboration between University of Manchester and the BBC Internet Futures Research and Development Team. During 2020 mixed methodology from the article was used to design a new data collection initiative with a project partner - Who Targets Me - a civic tech group that are creating and managing a crowdsourced global database of political adverts placed on social media. The project is funded by the EU as part of an ERC advanced investigator grant and will investigate individuals exposure and responses to online political advertising during election campaigns in several major democracies during the period 2020-2025. The goal will be to understand how these ads affect attitudes and behavior and to draw out guidelines around best practice in transparency and regulation of the use of online political ads. |
| First Year Of Impact | 2014 |
| Sector | Government, Democracy and Justice |
| Impact Types | Societal,Policy & public services |
| Description | World Bank Report on Delivering Benefits of Information Technology to Society |
| Geographic Reach | Multiple continents/international |
| Policy Influence Type | Citation in other policy documents |
| URL | https://openknowledge.worldbank.org/handle/10986/23347 |
| Description | ESRC Impact Acceleration Award |
| Amount | £20,000 (GBP) |
| Organisation | University of Manchester |
| Sector | Academic/University |
| Country | United Kingdom |
| Start | 01/2018 |
| End | 12/2018 |
| Title | Mixed method - Survey, Time Diary and Focus Group |
| Description | The method is an innovative one that combines large scale survey data with time diary data and focus group discussion to create a 'joined up' qualitative and quantitative understanding of citizens participation in online civic action groups. |
| Type Of Material | Improvements to research infrastructure |
| Year Produced | 2015 |
| Provided To Others? | Yes |
| Impact | The methodology we developed has been used as the basis for a project with the BBC to understand knowledge gaps among its audience. The project has since gained funding from the ESRC under the Impact Acceleration Award (IAA) scheme. A pilot project using the method is being undertaken in 2018. |
| URL | http://journals.sagepub.com.manchester.idm.oclc.org/doi/abs/10.1177/1461444815616225 |
| Description | Knowledge Gaps and the Role of the BBC in the era of fake news |
| Organisation | British Broadcasting Corporation (BBC) |
| Country | United Kingdom |
| Sector | Public |
| PI Contribution | We provided the research questions and academic input to frame the project and the methodological approach. |
| Collaborator Contribution | The BBC provided the data and infrastructure to conduct the research and also contributed to the design and implement of the survey instruments/data collection. Members of the BBC team also helped with the analysis, organizing the dissemination through a presentation at the BBC and helping write the final report. |
| Impact | Paper presentation at the 2018 UK Political Studies Association (PSA) Annual conference, 'Media's Role in the Public Sphere' Authors Jared Ahmad (University of Sheffield) Shirumisha Kwayu (Nottingham Trent University) Natalie Martin (Nottingham Trent University) Marta Cantijoch (University of Manchester) Nikki Soo (University of Manchester) Tim Cowlishaw (BBC) Rachel Gibson (University of Manchester) |
| Start Year | 2018 |
| Description | Blog post to promote the findings from our research |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | Unknown in that it may have been discussed by mySociety users informally/privately The blog post was made on mySociety site and for information purposes. They did not open it for comments and so difficult to judge specific impacts. |
| Year(s) Of Engagement Activity | 2014 |
| URL | https://www.mysociety.org/2014/12/08/research-on-the-impact-of-mysocietys-digital-tools |
| Description | Launch Event for the Project Report 'The Third Sector and Online Citizen Empowerment' |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Participants in your research and patient groups |
| Results and Impact | The event resulted in an extensive discussion and constructive feedback was provided by the experts and mySociety users that were present. mySociety's Director and Director of Research both suggested future discussion about further research projects. The report led to specific policy recommendations for mySociety in regard to their sites. Specifically this involved recommendations for more shared spaces and opportunities for local communal action. Steven Clift, Editor of Democracies Online Newswire and a leading proponent of e-democracy in the U.S. (Executive Director - http://E-Democracy.org) was present and asked for copies of the survey and report to inform his own government funded research currently underway in Minneapolis. Clift contacted PI afterward to discuss the findings of our report and specifically for advice on interpeting the findings from the E-Democracy Neighbours Forum survey' that he was conducting at the local level in Minnesota. Harris Fitch of the Democratic Society http://www.demsoc.org/ approached me and invited me to attend and participate as an Expert in their upcoming event to promote youth involvement with politics at the Manchester People's History Museum that they were running as part of Parliament Week activities. |
| Year(s) Of Engagement Activity | 2014 |
| Description | Meeting and discussion with staff at The Democratic Society |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Professional Practitioners |
| Results and Impact | Plans made for future collaboration. In particular we identified a possible grant project that would extend my current research looking at twitter use in the 2015 General election to allow Demsoc to better understand public discussion about political issues. I will share my paper on the Twitter study with Demsoc when it is finished in March 2015 and based on this we may then apply for Research Council impact funding and other sources which fund Charities. |
| Year(s) Of Engagement Activity | 2014 |
| Description | Meeting with Speakers Commission on Digital Democracy |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | Yes |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | An open and wide ranging discussion was had on the topic of a third chamber plus online voting and also reform of the legislative process. The recommendations of the group were noted by the Commission staff and we will be kept informed of progress and the final recommendations. |
| Year(s) Of Engagement Activity | 2014 |
| URL | http://www.parliament.uk/business/commons/the-speaker/speakers-commission-on-digital-democracy/ |
| Description | Participation as a 'Democracy Professional' in a Parliament Week event organized by the Democratic Society to help increase engagement of young people with Parliament |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | Yes |
| Geographic Reach | Regional |
| Primary Audience | Public/other audiences |
| Results and Impact | To be confirmed |
| Year(s) Of Engagement Activity | 2014 |
| URL | https://www.parliamentweek.org/event/democratic-dialogue-how-young-people-would-like-to-communicate-... |
| Description | Training Workshop for Third Sector on co-producing social research. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Third sector organisations |
| Results and Impact | The project design and findings were used as part of a University of Manchester funded 'impact' initiative designed to increase cooperation with the 3rd sector groups. In particular the goal was to work together to co-produce training in the use of quantitative methods and Silvia Galandini, the project RA led Applied workshop 4 'Creating Evidence: Quantitative Data' using our mySociety project as a 'ideal case' of the types of work that could be done. |
| Year(s) Of Engagement Activity | 2016 |
| Description | mySociety tweeted to promote findings from report. |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | Not aware of any discussion in response to the tweet and how far it led to discussion among readers. It was retweeted mySociety has almost 10,000 followers and so we expect that tweet will have promoted interest in the report and led to individuals following the link to the report. |
| Year(s) Of Engagement Activity | 2014 |
| URL | https://twitter.com/mysociety/status/541977937386680320 |