Digital Realism: Visualising the social through digital art practice

Lead Research Organisation: University of Westminster
Department Name: Faculty of Media Arts and Design

Abstract

How can digital artists use the vast amounts of digital data we produce on a daily basis ('big data') to tell stories about ourselves and our lives?

In an era when public issues of concern are increasingly framed, mirrored and played out as exchanges and circulations of data (e.g. via Twitter and Facebook), we are told that big data technologies promise fundamental changes in our ability to represent and understand human behaviour. These approaches offer distinctively new ways of capturing what we do through analysis of how we interact with the Internet's digital services and through the data traces we leave behind us in this process. Given enough computing power, argue advocates, the patterns revealed in these datasets have the potential to reveal solutions to some of our most pressing social, economic, political, and environmental issues.

This research suggests that just as documentary forms arising from photography and film have historically sought to use technology to tell stories about our human lives, big data technologies may offer distinctively new ways to represent us through the ways they capture the data we produce through our everyday activities.

We will produce prototype artworks in the form of visualisations, animations, large-scale static images and digital 3D models derived from social data sets. Current approaches in big data, focus on how it can be employed in a commercial setting by using data derived from social media for example, to target customers more effectively. Similar approaches in science research seek correlations in scientific data to better understand interactions in natural systems. However this proposed project will represent the patterns of human activity discovered in big data to tell stories about what we believe in, what we do and who we are. Visualisations are an effective way of doing this as they take abstract and difficult to understand information and turn it into image forms that make data comprehensible.

As the world around us, and our activity within it, is increasingly marked by the data traces we leave behind, there exists a pressing need for the kinds of exploratory work this project proposes. Specifically research looks to find ways of bringing to attention aspects of the social overlooked in normative big data approaches. From a creative perspective there is scope to foreground the interpretive nature of big data, reflect upon what is missing in its coverage and find ways of depicting these absences.

Research unites digital arts researchers from the University of Westminster with expertise in visualisation and digital arts practice with social geographers from the Oxford Internet Institute, Oxford University, whose research is concerned with how certain groups and communities are excluded from digital data sets.

A project website will document research and act as a platform to publish artworks produced during the project which will be made freely available to the public. Similarly all computer code and wherever possible data sets will be released to the public and wider research community. A free conference open to the public that includes academics, digital artists and designers and other specialists will be held at project end to debate the consequences and benefits of research and discuss ways of taking work forward.

Planned Impact

Given the interest in the topic of big data and the interdisciplinary nature of the project we posit a number of different kinds of non-academic beneficiaries across a range of areas:

-Digital art and design practitioner groups interested in gaining access to specific knowledge and understanding of data driven approaches to art and design practice. Research delivers significant additional value for practitioner groups by producing models of practice, production insights and technological innovation in artwork design and production. Access to research occurs via attendance at the final public conference and through access to artworks, annotated code, data sets and design notes published on the project website. We will widely publicize the project to this group through a social media presence, via relevant mailing lists, blogs and newsgroups (e.g. Furtherfield, Rhizome) and through the PIs extensive networks of contacts in the field of digital art and design.

-Cultural and museum organizations looking for clear guidance and insights into the emerging potentials of big data as a cultural phenomena. There is considerable interest in the topic of big data from organizations such as Lighthouse Digital Culture Agency (Brighton) and the Arts Catalyst (London) amongst others, with the former already working in collaboration with the Open Data Institute to explore the ramifications of open data for cultural practice. We have already had initial contact with the Arts Catalyst and Museum of London who are interested in the project, and the Arts Catalyst and the Open Data Institute have agreed to constitute an impact steering committee alongside members of the research team. The project potentially delivers significant insights for these organizations into how big data can function as a tool to representation of social realities. Research will provide better understanding of what our data is telling us about the world, which is directly relevant to the mission of these organisations to engender public understanding of technology through cultural production. Dissemination of research to these groups occurs through the final conference and via the website. Initial meetings with the Museum of London have scoped the possibility for an exhibition of visual outputs in the Museum's Sackler Gallery post-funded project period.

-Policy groups in culture and technology such as the Open Data Institute have existing projects such as Data as Culture which seek to raise awareness of how data has cultural and political ramifications and which seek to promote good practice in its use. Additionally organisations such as NESTA and the Arts Council are actively engaged with exploring the wider ramifications of big data to the cultural and creative industries. These groups derive benefits from research in that it builds out from their existing initiatives concerned with commerce and audience services, through a critical approach to big data with particular focus on issues of representation of social groups. Access to research will occur via invitation to public events, the Open Data Institute are acting as an impact partner and will participate in the conference at end of project.

-Policy groups concerned with the digital divide. The Oxford Internet Institute (OII) has a number of links with policy groups involved in discussions around social inclusion (e.g. Department for International Development, SciDev). We would expect research into imaging under-represented groups to be of interest to these organisations as it allows comprehension of gaps in digital provision relevant to their research focus. Policy groups will be informed of pertinent outputs via the OII, the website and social media presence, and through invitation to the final public-facing conference.

Publications

10 25 50
 
Title CODEX Asia 
Description CODEX consists of a number of animations displayed in installation format, and large prints derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities (9). Article titles derived from this data are used to produce glyphs and phonemes that hint at the underlying content and structure of the data, displaying visual fields of partial signs and phrastic utterances. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space. 
Type Of Art Artwork 
Year Produced 2015 
Impact Exhibited at: The 11th Athens Digital Arts Festival, Athens, Greece, 1st May 2015 "Geo-Codes: Mapping a Practice in the Post-Print Age" exhibition at the China Academy of Art in Hangzou, September 22th to October 3rd, 2015 CODEX, large-scale animations of social data describing alternative geographies. '25' Alumni  exhibition at London Gallery West Thursday 15 October until 1 November, 2015 CODEX large scale data prints of 'social geologies' 
URL http://digital-realism.net/2015/09/25/codex-asia/
 
Title CODEX Europe 
Description CODEX consists of a number of animations displayed in installation format, and large prints derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities (9). Article titles derived from this data are used to produce glyphs and phonemes that hint at the underlying content and structure of the data, displaying visual fields of partial signs and phrastic utterances. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space. 
Type Of Art Artwork 
Year Produced 2015 
Impact Exhibited at: The 11th Athens Digital Arts Festival, Athens, Greece, 1st May 2015 "Geo-Codes: Mapping a Practice in the Post-Print Age" exhibition at the China Academy of Art in Hangzou, September 22th to October 3rd, 2015 CODEX, large-scale animations of social data describing alternative geographies. '25' Alumni  exhibition at London Gallery West Thursday 15 October until 1 November, 2015 CODEX large scale data prints of 'social geologies' 
URL http://digital-realism.net/2015/09/25/codex-europe/
 
Title CODEX Global 
Description CODEX consists of a number of animations displayed in installation format, and large prints derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities (9). Article titles derived from this data are used to produce glyphs and phonemes that hint at the underlying content and structure of the data, displaying visual fields of partial signs and phrastic utterances. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space. 
Type Of Art Artwork 
Year Produced 2015 
Impact Exhibited at: The 11th Athens Digital Arts Festival, Athens, Greece, 1st May 2015 "Geo-Codes: Mapping a Practice in the Post-Print Age" exhibition at the China Academy of Art in Hangzou, September 22th to October 3rd, 2015 CODEX, large-scale animations of social data describing alternative geographies. '25' Alumni  exhibition at London Gallery West Thursday 15 October until 1 November, 2015 CODEX large scale data prints of 'social geologies' 
URL http://digital-realism.net/2015/09/25/codex-global/
 
Title CODEX Global 
Description CODEX consists of a number of animations displayed in installation format, and large prints derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities (9). Article titles derived from this data are used to produce glyphs and phonemes that hint at the underlying content and structure of the data, displaying visual fields of partial signs and phrastic utterances. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space. 
Type Of Art Artwork 
Year Produced 2015 
Impact Exhibited at: The 11th Athens Digital Arts Festival, Athens, Greece, 1st May 2015 "Geo-Codes: Mapping a Practice in the Post-Print Age" exhibition at the China Academy of Art in Hangzou, September 22th to October 3rd, 2015 CODEX, large-scale animations of social data describing alternative geographies. '25' Alumni  exhibition at London Gallery West Thursday 15 October until 1 November, 2015 CODEX large scale data prints of 'social geologies' 
URL http://digital-realism.net/2015/09/25/codex-global/
 
Title CODEX Mercator 
Description CODEX consists of a number of animations displayed in installation format, and large prints derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities (9). Article titles derived from this data are used to produce glyphs and phonemes that hint at the underlying content and structure of the data, displaying visual fields of partial signs and phrastic utterances. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space. 
Type Of Art Artwork 
Year Produced 2014 
Impact Exhibited at: The 11th Athens Digital Arts Festival, Athens, Greece, 1st May 2015 "Geo-Codes: Mapping a Practice in the Post-Print Age" exhibition at the China Academy of Art in Hangzou, September 22th to October 3rd, 2015 CODEX, large-scale animations of social data describing alternative geographies. '25' Alumni  exhibition at London Gallery West Thursday 15 October until 1 November, 2015 CODEX large scale data prints of 'social geologies' 
URL http://digital-realism.net/2015/09/25/codex-mercator/
 
Title CODEX Mercator 
Description CODEX consists of a number of animations displayed in installation format, and large prints derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities. Article titles derived from this data are used to produce glyphs and phonemes that hint at the underlying content and structure of the data, displaying visual fields of partial signs and phrastic utterances. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space. 
Type Of Art Artwork 
Year Produced 2014 
Impact Exhibited at: The 11th Athens Digital Arts Festival, Athens, Greece, 1st May 2015 Geo-Codes: Mapping a Practice in the Post-Print Agea exhibition at the China Academy of Art in Hangzou, September 22th to October 3rd, 2015 CODEX, large-scale animations of social data describing alternative geographies. '25' Alumni  exhibition at London Gallery West Thursday 15 October until 1 November, 2015 CODEX large scale data prints of 'social geologies' 
URL http://digital-realism.net/2015/09/25/codex-mercator/
 
Title CODEX Polar 
Description CODEX consists of a number of animations displayed in installation format, and large prints derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities (9). Article titles derived from this data are used to produce glyphs and phonemes that hint at the underlying content and structure of the data, displaying visual fields of partial signs and phrastic utterances. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space. 
Type Of Art Artwork 
Year Produced 2015 
Impact Exhibited at: The 11th Athens Digital Arts Festival, Athens, Greece, 1st May 2015 "Geo-Codes: Mapping a Practice in the Post-Print Age" exhibition at the China Academy of Art in Hangzou, September 22th to October 3rd, 2015 CODEX, large-scale animations of social data describing alternative geographies. '25' Alumni  exhibition at London Gallery West Thursday 15 October until 1 November, 2015 CODEX large scale data prints of 'social geologies' 
URL http://digital-realism.net/2015/09/25/codex-polar/
 
Title CODEX Titles 
Description CODEX consists of a number of animations displayed in installation format, and large prints derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities. Article titles derived from this data are used to produce glyphs and phonemes that hint at the underlying content and structure of the data, displaying visual fields of partial signs and phrastic utterances. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space. 
Type Of Art Artwork 
Year Produced 2015 
Impact Exhibited at: The 11th Athens Digital Arts Festival, Athens, Greece, 1st May 2015 "Geo-Codes: Mapping a Practice in the Post-Print Age" exhibition at the China Academy of Art in Hangzou, September 22th to October 3rd, 2015 CODEX, large-scale animations of social data describing alternative geographies. '25' Alumni  exhibition at London Gallery West Thursday 15 October until 1 November, 2015 CODEX large scale data prints of 'social geologies' 
URL http://digital-realism.net/2015/09/25/codex-titles/
 
Title Exhibition: International Symposium of Electronic Arts (ISEA), 30 Oct - 8 Nov, 2014, Dubai, United Arab Emirates 
Description Terra Incognita installation exhibited 
Type Of Art Artistic/Creative Exhibition 
Year Produced 2014 
Impact to follow 
 
Description Several artworks with a collective name of CODEX were developed from this project and internationally exhibited including large-scale data animations and prints on paper.

CODEX follows in a history of artistic and philosophical interventions into cartography, such as Agnes Denes' map projections, which relocate scientific and technological knowledge toward an aesthetic reimagining of humanity's relationship to the earth, or Buckminster Fuller's Dymaxion maps which develop a similar ecological argument through speculative geologies. Working from this critical and aesthetic tradition, but with a contemporary focus on data and algorithmic sorting, CODEX explores how a given socially constructed dataset (in this case Wikipedia) demonstrates how the dispersal of online knowledge reflects imbalances in political and geographic power in the real world.

CODEX developed images and animations derived from a geographic mapping of Wikipedia's language edition articles that refer to locations, countries and cities . Article titles derived from this data are used to produce images that hint at the underlying content and structure of the data, displaying visual fields of partial signs and texts. En masse, the texts form accretions of Wikipedia contributions, remnants of collective activity and shared bodies of knowledge. The metadata associated with each article shows which locations are most edited and revised, which are the longest, and which are translated into other languages; often an indication that these coded spaces are contested and fought over much as real space.

In the prints and animations, the structures of the encyclopedia are made tangible by mapping the visual properties of the text to the metadata of the articles (e.g. the text fragments might be scaled in relation to the number of times an article has been edited). The effect of this is a differential texturing across the map, where major articles appear as large-scale labels and less developed but more abundant articles form topographies of varying density and in some cases phlegm like viscosity. In the prints this shows as layered, scabrous and inked surfaces that have a three-dimensional quality which could be considered as geological forms.

The temporal development of article distribution on Wikipedia can be gleaned by looking at the creation dates of the articles. Animations of this editing activity shows how each language community has developed and grown at different times, and how it describes geographic space.

If Buckminster Fuller or Agnes Denes' work hints at the 'utopian' or redemptive power of reason and a human capacity to progress to a saner more ecologically and socially equitable future, the mappings revealed in CODEX present a less certain vision. The work's deformations draw attention to how excessive flows of capital, technology, and material are reshaping geography, space and time, thereby inviting us to contemplate our role in a world in upheaval. Additionally, we argue, by critiquing how apparently objective conventions of space, knowledge and data production are inflected by political and social power structures, cultural bias, and author subjectivities, the work asks (and partly answers): who is represented, and who is left out of the picture?
Exploitation Route In conjunction with the artworks, we developed a software toolkit under a GNU General Public License that enables the visual output of large scale social and geographic datasets for other artists and creative practitioners. We have used the toolkit to render the human activity evident in archives of geo-coded social media, Wikipedia articles, and Open Street Map exports and it is designed to enable the output large scale map images and geographic animations. The toolkit can also be employed to explore and interact with the datasets, and to configure how the maps are rendered.

The toolkit enables other artists and creative practitioners to develop their own data driven artworks and are released as open source software.

We also argue that the art works developed during the project provide a model of creative practice for artists and designers working with geographic data that enables alternative geographical renderings of the way that technological and social infrastructures are developed, maintained and distributed electronically.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections

URL http://digital-realism.net/
 
Description The artworks developed in this research project have been internationally exhibited. They provide aesthetic models of art practice that demonstrate the creative potentials of big data as an artistic medium and material. In particular, we develop critical aesthetic approaches to the use of big data overlooked in standard data-driven analysis, (and by extension normative data visualization practices) which are primarily concerned with cognition. Exhibited works have been used to show how big data can be employed to reflect upon social realities through the formulation of critical, aesthetic and speculative geographies and how these approaches can broker discussion of the wider social, environmental and political impact of big data in the broad.
First Year Of Impact 2001
Sector Creative Economy,Digital/Communication/Information Technologies (including Software),Culture, Heritage, Museums and Collections
Impact Types Cultural,Societal

 
Description Materializing Data, Embodying Climate Change
Amount £705,588 (GBP)
Funding ID AH/S00369X/1 
Organisation Arts & Humanities Research Council (AHRC) 
Sector Public
Country United Kingdom
Start 02/2019 
End 02/2022
 
Title Visualisation toolkit 
Description In the process of creating the project artworks a software toolkit has been developed that visualises large scale social and geographic datasets. The toolkit has been used to render the human activity evident in archives of geo-coded social media, Wikipedia articles, and Open Street Map exports. The toolkit is available on Github: https://github.com/tracemedia/geo_qt This initial version of the software is designed to output large scale map images and geographic animations. It can be used to explore and interact with the datasets, and to configure how the maps are rendered. The source code is intended to be used as a toolkit rather than as a comprehensive library, and consists of a number of base classes which can be extended to create specific visualisations. The process of creating a visualisation would typically involve developing custom data-processing, rendering, and user interface tools, and creating a configuration file which specifies the map layers, data sources, and styling options. To utilise an extensive resource of visualisation and GIS libraries, the software has been developed in Python (2.7), and uses the PySide Qt framework for user interface components. Each of the package files includes an executable example which shows how the visualisation and user interface are configured. The main base classes are as follows: MapQt: Main Qt map window that contains the map layers. A map is initialised with a canvas size, a projection in Proj4 format, and a geographic extent. Layer: Abstract Map Layer class. Renders a map layer as an image, and can handle many millions of map features. MapDialog: User interface for manipulating the visualisation. The map parameters, and the layer styles and properties can be edited via the dialog. The layer styles are in JSON format which allows for complex data-structures. The visualisations are generated from a composite of map layers, which means that layers can be edited and re-rendered individually. Examples of map layers types include: GeoJsonLayer: Simple GeoJSON parser and renderer, can be extended for custom styling. This has been used for rendering country boundaries and city level OSM data. PointDataLayer: Simple point data renderer, can be extended for custom styling. ThematicPointLayer: Point data renderer that maps data properties to text and point styles. The styles can be specified either globally or per feature. ModestMapsLayer: ModestMaps layer renders tiles from providers including Open Street Map and Microsoft Bing. This is useful for plotting the data against city level basemaps. BasemapLayer: Matplotlib Basemap layer, used for rendering country boundaries on a wide variety of map projections that not handled by the GeoJsonLayer. The Matplotlib paths are extracted and rendered to a PySide image layer. Timeline Each of the main base classes are overridden to create animated maps. TimelineMap and TimelineDialog include a date range for the complete dataset, and the date range of the current view. The dialog has controls for setting the current date range, playing the animation, and outputting each frame as an image. TimelineDataLayer displays the map data and can overridden to specify how the features are rendered as a function of the current date range. Software tools Enthought Canopy was used to install and manage the relevant Python packages, since it provides a robust and well tested platform. 
Type Of Technology Software 
Year Produced 2015 
Open Source License? Yes  
Impact Still awaiting feedback from users 
URL http://digital-realism.net/2015/06/10/visualisation-toolkit/