Interrogating the Dashboard: Data, Indicators and Decision-making
Lead Research Organisation:
University of Warwick
Department Name: Centre for Interdisc. Methodologies
Abstract
In recent times, 'dashboards' have become increasingly common features of business, public policy and everyday life. The term refers to a way of representing multiple indicators, data and news, in real-time and side-by-side. Organisations build dashboards so as to display 'key performance indicators', on which senior management can base decisions. UK Prime Minister David Cameron has commissioned a tailor-made dashboard for his iPad, to keep him constantly updated on key indicators such as GDP and house prices. However, with the rise of Big Data and the proliferation of smart devices in everyday life, this type of display is increasingly available for individuals for everyday activities. For example, the 'London Dashboard' is a website offering real-time data on traffic, weather, tube line status, local twitter trends, markets and other indicators, for everyday purposes. Dashboard apps, such as Google Now, offer ways of bringing together various sources of data into a handy, usable format.
Dashboards are a worthy object of social scientific investigation for a number of reasons. Firstly, they problematise how the notion of risk is understood. Risk is a key concept for how contemporary societies are governed, managed and theorised by social scientists, and has previously been analysed via particular techniques (such as audit and accounting) through which the future is represented numerically. Dashboards offer a different numerical representation, of a future that is constantly emerging, without moments of static evaluation as provided by audit. In this, their numbers resemble market prices. Indeed financial markets (and the digital displays they use) might be seen as one of the progenitors of dashboards more generally. Secondly, dashboards offer a basis on which to reflect critically on how data exists in everyday life, how it fits into practices and decision-making, and the methods involved in making data useful. Thirdly, the spread of dashboards also involves the spread of a particular vision of agency and authority, in which individuals are constantly navigating, deciding, strategising, and interpreting on the basis of constant updates. Finally, as the social sciences are themselves facing the challenges and opportunities of Big Data, we believe that dashboards can be productively incorporated into research practices and the dissemination of outputs.
The research project takes three empirical cases, in order to understand how dashboards are integrated into a range of situations and for different purposes. We will study civil servants in Government Digital Services who have built and are using dashboards to visualise and 'open up' the performance of Whitehall departments. We will also study the Digital Action Lab, a civic activist organisation, as they develop and release a 'crowdsourced' dashboard which provides a range of indicators on prosperity, equality and environment in Britain. Finally, we will study the personal use of dashboards on phones and tablets, where dashboards visualise and arrange data flows in ways that make them intelligible and actionable for individuals.
The main project questions are:
1. How are organisations and individuals using dashboards? Specifically, we ask: what kinds of indicators are used and for what purpose? What is the temporality or 'liveness' of these indicators? What is the relationship between indicators and other signifiers on the dashboard?
2. How are dashboards incorporated into and transforming decision-making processes? Are dashboards evidence of a general transformation in the temporality of judgement and decision-making? How do individuals cope with the multiplicity and inconclusiveness of dashboard indicators?
3. How can the social sciences incorporate dashboards as tools and also as research outputs? What can be learnt from existing dashboards? Conversely, how can the knowledge and expertise of the social sciences be fed into dashboard design?
Dashboards are a worthy object of social scientific investigation for a number of reasons. Firstly, they problematise how the notion of risk is understood. Risk is a key concept for how contemporary societies are governed, managed and theorised by social scientists, and has previously been analysed via particular techniques (such as audit and accounting) through which the future is represented numerically. Dashboards offer a different numerical representation, of a future that is constantly emerging, without moments of static evaluation as provided by audit. In this, their numbers resemble market prices. Indeed financial markets (and the digital displays they use) might be seen as one of the progenitors of dashboards more generally. Secondly, dashboards offer a basis on which to reflect critically on how data exists in everyday life, how it fits into practices and decision-making, and the methods involved in making data useful. Thirdly, the spread of dashboards also involves the spread of a particular vision of agency and authority, in which individuals are constantly navigating, deciding, strategising, and interpreting on the basis of constant updates. Finally, as the social sciences are themselves facing the challenges and opportunities of Big Data, we believe that dashboards can be productively incorporated into research practices and the dissemination of outputs.
The research project takes three empirical cases, in order to understand how dashboards are integrated into a range of situations and for different purposes. We will study civil servants in Government Digital Services who have built and are using dashboards to visualise and 'open up' the performance of Whitehall departments. We will also study the Digital Action Lab, a civic activist organisation, as they develop and release a 'crowdsourced' dashboard which provides a range of indicators on prosperity, equality and environment in Britain. Finally, we will study the personal use of dashboards on phones and tablets, where dashboards visualise and arrange data flows in ways that make them intelligible and actionable for individuals.
The main project questions are:
1. How are organisations and individuals using dashboards? Specifically, we ask: what kinds of indicators are used and for what purpose? What is the temporality or 'liveness' of these indicators? What is the relationship between indicators and other signifiers on the dashboard?
2. How are dashboards incorporated into and transforming decision-making processes? Are dashboards evidence of a general transformation in the temporality of judgement and decision-making? How do individuals cope with the multiplicity and inconclusiveness of dashboard indicators?
3. How can the social sciences incorporate dashboards as tools and also as research outputs? What can be learnt from existing dashboards? Conversely, how can the knowledge and expertise of the social sciences be fed into dashboard design?
Planned Impact
The project concerns issues regarding the visualisation of data, and its organisational and public significance, that is of direct concern to government, civil society, business and everyday users of digital devices. The potential impact of the project is therefore very far-reaching. Moreover, given the choice of case studies and research partners, the project will establish a space of knowledge exchange that cuts between academic, government and civic stakeholders, creating networks which will out-last the duration of the project. In this project, we do not approach impact as a goal only after knowledge has been collected and processed, but aim for constant dialogue between social scientific researchers and expert users, on the question of how data is to be managed, visualised, shared and acted upon. Dashboards provide an object of concern, around which researchers and research users congregate.
The project meets the ESRC's three categories of impact in the following ways:
- 'Instrumental': it will influence policy-makers and civic activists, seeking to use dashboards for purposes of service modernisation, open government and campaigning.
- 'Conceptual': it will advance a social scientific discourse and public understanding regarding dashboards, as critical devices for the management and visualisation of change, in organisations, government, civil society and everyday life.
- 'Capacity building': it will assist in the development of new dashboards, at work in government, business, civil society and academia, by injecting social and cultural knowledge regarding the efficacy and 'social life' of data visualisation.
The key stakeholders for the project are identified as follows:
Government, public policy & public services
- Government Digital Services (GDS)
- Cabinet Office
- Department for Business Innovation & Skills
- Communities & Local Government
- City and local government (Local Government Association, Greater London Authority, local authorities)
- Think tanks - Demos, Policy Exchange, ippr, Policy Network
Civil society, activists & artists
- e-democracy activists and charities - Open Data Institute, mySociety.org, change.org, Digital Action Lab
- Hubs for social innovation - NESTA, The Young Foundation
- Social indicators practitioners - New Economics Foundation
- Experimental and arts groups - London Quantified Self group
The possible benefits for these stakeholders are as follows:
- Influencing public service modernisation: Policy makers and civil servants (in central and local government) will benefit from the research, by considering the 'social life' of data and dashboards, as contributors to organisational accountability and culture. Researchers will share findings with GDS, from other investigations into dashboard design and use.
- Advancing the transparency/openness agenda: The project speaks directly to those in government and outside, interested in advancing 'openness' and 'transparency', through making more data publicly available. UK government is interested in whether dashboards can increase public accountability, through revealing more information about performance to citizens. The project will look at how this works in practice, how it overlaps with other concerns to do with public service reform, and potential legitimacy problems that result.
- Informing citizens using data visualisation: Various civic groups and campaigns currently seek to use data that is available to mobilise publics, around specific issues. Dashboards are at the cutting edge of this, but various questions are still unanswered, regarding the civic potential of Big Data. This project will contribute to better public, citizen and activist understanding of strategies associated with visualisation, design, customisation and use.
- Producing new networks between policy-makers, technologists, activists and researchers.
The project meets the ESRC's three categories of impact in the following ways:
- 'Instrumental': it will influence policy-makers and civic activists, seeking to use dashboards for purposes of service modernisation, open government and campaigning.
- 'Conceptual': it will advance a social scientific discourse and public understanding regarding dashboards, as critical devices for the management and visualisation of change, in organisations, government, civil society and everyday life.
- 'Capacity building': it will assist in the development of new dashboards, at work in government, business, civil society and academia, by injecting social and cultural knowledge regarding the efficacy and 'social life' of data visualisation.
The key stakeholders for the project are identified as follows:
Government, public policy & public services
- Government Digital Services (GDS)
- Cabinet Office
- Department for Business Innovation & Skills
- Communities & Local Government
- City and local government (Local Government Association, Greater London Authority, local authorities)
- Think tanks - Demos, Policy Exchange, ippr, Policy Network
Civil society, activists & artists
- e-democracy activists and charities - Open Data Institute, mySociety.org, change.org, Digital Action Lab
- Hubs for social innovation - NESTA, The Young Foundation
- Social indicators practitioners - New Economics Foundation
- Experimental and arts groups - London Quantified Self group
The possible benefits for these stakeholders are as follows:
- Influencing public service modernisation: Policy makers and civil servants (in central and local government) will benefit from the research, by considering the 'social life' of data and dashboards, as contributors to organisational accountability and culture. Researchers will share findings with GDS, from other investigations into dashboard design and use.
- Advancing the transparency/openness agenda: The project speaks directly to those in government and outside, interested in advancing 'openness' and 'transparency', through making more data publicly available. UK government is interested in whether dashboards can increase public accountability, through revealing more information about performance to citizens. The project will look at how this works in practice, how it overlaps with other concerns to do with public service reform, and potential legitimacy problems that result.
- Informing citizens using data visualisation: Various civic groups and campaigns currently seek to use data that is available to mobilise publics, around specific issues. Dashboards are at the cutting edge of this, but various questions are still unanswered, regarding the civic potential of Big Data. This project will contribute to better public, citizen and activist understanding of strategies associated with visualisation, design, customisation and use.
- Producing new networks between policy-makers, technologists, activists and researchers.
Organisations
Publications
Bartlett, J
(2016)
Governance by Dashboard
Cowley R
(2017)
Forum: resilience & design
in Resilience
Davies W
(2015)
The Chronic Social: Relations of Control Within and Without Neoliberalism
in New Formations
Davies W
(2016)
How are we now? Real-time mood-monitoring as valuation
in Journal of Cultural Economy
Tkacz Nathaniel
(2022)
Being with Data: The Dashboarding of Everyday Life
| Title | The Index of Performative Keys |
| Description | The Index of Performative Keys is the 'visual provocation' for future dashboard design component of the project. The IPK reflects the historical research conducted on the history of dashboards and their intersection with that of performance management. The IPK is a 'key performance indicator' generator (or 'engine') that borrows from dashboard design conventions. The IPK is commentary on the shifting role of performance indicator and metrics in everyday life though the increased production of data. |
| Type Of Art | Artefact (including digital) |
| Year Produced | 2016 |
| Impact | The IPK has been displayed at the Streams of Consciousness conference at Warwick in April 2016, at the International Communication Association conference in Fukuoka, Japan in June 2016 and will be featured at the Culture and Politics of Data Visualisation conference at the University of Sheffield in October 2016. |
| URL | http://www.ipk.solutions |
| Description | Our proposition to study the rise and significance of Big Data through the many small interfaces it is commonly displayed on turned out to be very fruitful and we believe that our research has made several research findings that are genuinely novel, and which have the potential to be further explored and developed in future research. 1. Decision-making and distributed cognition. We proposed to study how people used dashboards to make decisions, and suggested that the nature of decision-making (in terms of time, factors, and procedures) might be changing. Our study did confirm this, but it also suggested something more significant. We found that through dashboards, individuals are coupled with intelligent systems that complicate the idea of decision making as attributable to a specific individual. What we found was most significant was not simply how dashboards change how people make decisions, but how the human-dashboard-system configuration requires a rethinking of what and where decision-making is and resides. 2. Design and Data. We found that the role of design is absolutely central to how data is collected, used and presented, but it barely features at all in discussion about data. Here, we do not mean the design of data collection, but rather 'design thinking' and design practice. To put it concisely, we found that design expertise and design authority (product design, UX design, etc.) was very influential in how flows of data and related interfaces were implemented into organisational settings. 3. Dashboard and Performance (Bias). Our historical study of dashboards investigated how they have been used and understood in very different settings, from cars and planes, to control rooms, organisations and phone apps. Contemporary dashboards often 'carry along' these historical legacies in their design (such as the persistence of gauges originated from car dashboards). However, we found that dashboard design carries with it a performance bias or rather 'Performance tendency' in that the format itself tends to encourage forms of measure, attention and monitoring, and indication that lends itself to performance management. In other words, when you put data in a dashboard, there is a tendency for it to be used to manage performance, even though this might not be a natural or obvious use of the data. 4. Social Science Knowledge Production and Dashboards. Our research proposed to consider how social scientific knowledge might be fed into dashboard design through the create of a series of visual provocations. This was the most challenging but also most satisfy aspect of the research. We found that in order to generate our visualisations, we had to draw upon methods not common within the social sciences. Instead, we had to draw upon the methods of designers and similar creative practice, such as prototyping, pitching ideas, drawing, and so on. It suggests that if the social sciences are to engage with these emerging areas, we have to be willing to 'meet half way' and partially rethink our own methods for knowledge production. |
| Exploitation Route | We believe that the changing relationship between data, interfaces and cognition that our study identified needs further attention. This could be taken forward through more research and the development of new research methods for the study of interfaces. Further study is also needed on the role of design in structuring human interaction with data. It is this area of expertise that mediates and socialises data-driven practices but it is often completely overlooked with the social sciences. A research project on the design of data could take this further. Findings from our study could be used to educate users of dashboards in the education, corporate and public sectors, to help foster more informed use of these interfaces. Artists and designers could use our findings (and especially our visual provocation) to help rethink dashboard design. These could be developed through exhibitions or rapid prototyping workshops, for example. |
| Sectors | Digital/Communication/Information Technologies (including Software),Financial Services, and Management Consultancy,Government, Democracy and Justice |
| URL | http://blogs.cim.warwick.ac.uk/readingdashboards |
| Description | This project was designed as an industry-government-academy collaboration from the beginning. It included a substantial collaboration with the think tank Demos, and further collaborations with the Digital Action Lab, the Government Digital Services Team (within the Cabinet Office) and, later, the business intelligence provider, Qlik. Ethnographic studies (interviews, participant observation, work shadowing) took place while the GDS team were developing their public services dashboards. While conducting these studies I regularly shared my interview findings with civil service employees. Midway through the project, I was asked to report my initial findings to Richard Sargeant, Director of Performance and Delivery for the Government Digital Service. Shortly after reporting my initial findings, the GDS Performance team entered a new "discovery phase", which included a significant review of the purpose and design of public service dashboards. Members from GDS, Demos, the Digital Action Lab and Qlik, also took part in a one-day 'collaboratory' where we reflected on the use and design of dashboards for decision-making. While conducting the study, research findings were also shared with the developers of the Dublin Dashboard (a city dashboard) and with the City of Melbourne Smart City Office in Australia, who were both at the early stages of designing their own city dashboards. A letter from the Smart City Office states, "Your presentation generated a lot of discussion within our team and helped us identify opportunities (and also a few challenges) that had not occurred to us in the early stages of our project." Shortly after the conclusion of the project, I was invited to present the findings to CEMADEN, a natural disaster monitoring and alert centre in Brazil. The research was of significant interest to the team, who use data dashboards to monitor disasters, and we have since engaged in a new collaboration (in the form of an EPSRC GCRF networking award - Tkacz as Co-I) to further explore how new types of data can contribute to existing forms of decision-making within the centre's control room. In April 2017, the project's main policy document, "Governance by Dashboard", was published. The publication was covered by many of the policy-orientated press, including Civil Service World, Public Technology, Local Government, Smart Cities World and Public Finance. A short article accompanying the release, "Speeding up Whitehall's Digital Dashboard", was also published in Civil Service World. A social media analysis of the report's circulation will be ready in the coming months. At this point, the project has achieved substantial and well-targeted engagement with a number of non-academic partners. More substantial impact may develop in coming years. |
| First Year Of Impact | 2015 |
| Sector | Digital/Communication/Information Technologies (including Software),Government, Democracy and Justice |
| Impact Types | Societal,Policy & public services |
| Description | Consultation/Seminar with City of Melbourne, Smart City Office |
| Geographic Reach | Australia |
| Policy Influence Type | Implementation circular/rapid advice/letter to e.g. Ministry of Health |
| Impact | My influence here is not on policy but practice. In December 2015 I was invited to present my research on dashboards within public services to the City of Melbourne, Smart City Office. The presentation was followed by an open consultation with a team of public servants tasked with creating a dashboard for the city of Melbourne, Australia. It was clear from the discussion that the team did not possess preexisting expertise in the creation of City Dashboards, nor were they overly aware of the numerous types of dashboards they could conceivably create. It was very clear that my visit impacted the way they were thinking about dashboards. Key areas that they had not considered were: who the users would be (tourists, locals, regular or one-time users, etc.); whether it would be designed for mobile or web; whether it would be generic or personalisable; whether it would focus on environmental and logistical measures or focus more on social ones. My consultation here shaped the thinking of the designers and developers of a public service during its early development stage. |
| Description | ESRC DTC Collaborative Studentship |
| Amount | £57,000 (GBP) |
| Funding ID | A.IMIM.1415.CXG |
| Organisation | Economic and Social Research Council |
| Sector | Public |
| Country | United Kingdom |
| Start | 09/2014 |
| End | 09/2018 |
| Description | Article for Demos Quarterly: "Keeping an eye on the dashboard" |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | This was a short article written for a non-academic, policy, industry and third sector audience. Its purposes was to outline the concerns that are driving the research project in a way that speaks to non-experts. Demos Quarterly has an average circulation of 20000. The nature of this engagement activity was to increase awareness about the issues that are at the heart of the project to a relevant audience, and not to generate direct impact. |
| Year(s) Of Engagement Activity | 2014 |
| URL | http://quarterly.demos.co.uk/article/issue-4/keeping-an-eye-on-the-dashboard/ |
| Description | Article in The Atlantic: "Mark Zuckerberg and the End of Language" |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | This article looked at smart technologies and cognition, drawing on our research on data, indicators and decision making. The article was shared 2,400 times on Facebook (as of March 2016) and was commented on 22 times on the Atlantic website. |
| Year(s) Of Engagement Activity | 2015 |
| URL | http://www.theatlantic.com/technology/archive/2015/09/silicon-valley-telepathy-wearables/404641/ |
| Description | Article in The Chronicle Higher Education: "Silly Robots!" |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Public/other audiences |
| Results and Impact | The article considered the limits of automation, designed for a broader audience. The Chronicle has a large readership and the article has received 18 comments as of march 2016. |
| Year(s) Of Engagement Activity | 2015 |
| URL | http://chronicle.com/article/Silly-Robots-/233965 |
| Description | Blog Post: "How to Give a Damn about Data?" |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Blog post intended to communicate project insights in progress. |
| Year(s) Of Engagement Activity | 2015 |
| URL | http://blogs.cim.warwick.ac.uk/readingdashboards/2015/10/20/how-to-give-a-damn-about-data/ |
| Description | Collaboratory (London) (two in total) |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Third sector organisations |
| Results and Impact | Over the two collaboratories we demonstrated, discussed and reflected upon the different uses of dashboards, with a focus on decision making. Participant shared best practice, but more importantly, discussed current challenges as dashboards are incorporated into new areas for the first time. |
| Year(s) Of Engagement Activity | 2015,2016 |
| Description | Demos Event: How Big Data is Transforming Decision Making |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Industry/Business |
| Results and Impact | This event was hosted by the think tank Demos in order to bring together a number of stakeholders and likely interested parties in order to discuss the contours of the research at an early stage. Dr Tkacz and Dr Davies presented their early thoughts on the project and this was followed by a general discussion. Registered attendees included representatives from the Home Office, the Cabinet Office, the BBC, Deloitte, the Digital Action Lab, Google, Twitter, Nesta, Demos and Qlik Technologies, as well as academic participation from the LSE, Oxford, Goldsmiths and Warwick. The event generated a lot of discussion, and resulted in a new collaboration with Qlik Technologies. Qlik has since given Dr Tkacz access to software and training facilities, staff and client interviews, and members from Qlik have been present for the project's 'collaboratories'. |
| Year(s) Of Engagement Activity | 2014 |
| Description | LSE Media Policy Blog contribution: "The Bounded Rationality of Algorithmic Accountability" |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Based on my current dashboard research, I was invited to attend a closed workshop at the LSE to discuss algorithmic accountability. After the work I contributed a blog post, which was hosted in a special theme section on the LSE Media Policy blog. |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://blogs.lse.ac.uk/mediapolicyproject/category/algorithmic-accountability/ |
| Description | News article |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | News article for the Guardian, 'The Long Read' section. As of March, 2017, this article has been shared over 24,000 times and has received over 1,200 comments. |
| Year(s) Of Engagement Activity | 2017 |
| URL | https://www.theguardian.com/politics/2017/jan/19/crisis-of-statistics-big-data-democracy |
| Description | News article citation, La Nacion (Argentina): "Me gusta tu politica: como las redes sociales influyen en los gobiernos" |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Public/other audiences |
| Results and Impact | Newspaper articles that covers real time data and prediction, with contributions from three project members. Article was published in major Argentinian newspaper and has been shared 243 times on Facebook as of March 2016 |
| Year(s) Of Engagement Activity | 2016 |
| URL | http://www.lanacion.com.ar/1874706-me-gusta-tu-politica-como-las-redes-sociales-influyen-en-los-gobi... |
| Description | Short Article for Civil Service World |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Short article for Civil Sevice World, a main news source for public servants, reporting on our study of dashboards within government. |
| Year(s) Of Engagement Activity | 2017 |
| URL | http://www.civilserviceworld.com/articles/opinion/opinion-speeding-whitehall%E2%80%99s-digital-dashb... |