Using deep learning and street-level imagery to predict social and political attributes

Lead Research Organisation: University of Bristol
Department Name: Geographical Sciences

Abstract

In the last five years, one of the most interesting applications of machine learning has been in the area of affective computing. This domain of applied machine learning has used computer vision techniques to predict how humans feel about images they see. When applied to street- level and satellite imagery, affective computing gives us a method to try to understand how the built environment makes us feel or act. As Owen notes in his statement, these visual cues have been used extensively to build "feel-of-a-street" models that predict social and economic outcomes for a given place based on how the place looks from the street. Owen discusses a few of these in his literature review, such as Gebru et al. (2017)'s predictions of the 2008 presidential election based on the types of street-parked cars. In geography, the classification of Multidimensional Open Data of Urban Morphology (MODUM) produced by Alexiou et al. (2016) aimed for a general-purpose classification based on data about urban structure and morphology. Owen's dissertation aims to do this through the look and feel of a place, constructed from street and satellite-level imagery.

This is not just a parlor trick: the ability to predict political information using street-level imagery allows us to really get a feel for how physical context matters for social processes, as well as opening the door into "now-casting" important social and political traits. In this sense, street-level computer vision models have truly expanded the reach of social science. However, further work is needed to determine whether these predictions are reliable. This is the work Owen proposes.

Publications

10 25 50

Studentship Projects

Project Reference Relationship Related To Start End Student Name
ES/P000630/1 01/10/2017 30/09/2027
2530380 Studentship ES/P000630/1 01/10/2021 30/09/2025 Owen Beric Winter