Learning an urban grammar from satellite data through AI

Lead Research Organisation: University of Liverpool
Department Name: Geography and Planning

Abstract

This project will propose an urban grammar to describe urban form and will develop artificial intelligence (AI) techniques to learn such a grammar from satellite imagery. Urban form has critical implications for economic productivity, social (in)equality, and the sustainability of both local finances and the environment. Yet, current approaches to measuring the morphology of cities are fragmented and coarse, impeding their appropriate use in decision making and planning.

This project will aim to: 1) conceptualise an urban grammar to describe urban form as a combination of "spatial signatures", computable classes describing a unique spatial pattern of urban development (e.g. "fragmented low density", "compact organic", "regular dense"); 2) develop a data-driven typology of spatial signatures as building blocks; 3) create AI techniques that can learn signatures from satellite imagery; and 4) build a computable urban grammar of the UK from high-resolution trajectories of spatial signatures that helps us understand its future evolution.

This project proposes to make the conceptual urban grammar computable by leveraging satellite data sources and state-of-the-art machine learning and AI techniques. Satellite technology is undergoing a revolution that is making more and better data available to study societal challenges. However, the potential of satellite data can only be unlocked through the application of refined machine learning and AI algorithms. In this context, we will combine geodemographics, deep learning, transfer learning, sequence analysis, and recurrent neural networks. These approaches expand and complement traditional techniques used in the social sciences by allowing to extract insight from highly unstructured data such as images. In doing so, the methodological aspect of the project will develop methods that will set the foundations of other applications in the social sciences.

The framework of the project unfolds in four main stages, or work packages (WPs):

1) Data acquisition - two large sets of data will be brought together and spatially aligned in a consistent database: attributes of urban form, and satellite imagery.
2) Development of a typology of spatial signatures - Using the urban form attributes, geodemographics will be used to build a typology of spatial signatures for the UK at high spatial resolution.
3) Satellite imagery + AI - The typology will be used to train deep learning and transfer learning algorithms to identify spatial signatures automatically and in a scalable way from medium resolution satellite imagery, which will allow us to back cast this approach to imagery from the last three decades.
4) Trajectory analysis - Using sequences of spatial signatures generated in the previous package, we will use machine learning to identify an urban grammar by studying the evolution of urban form in the UK over the last three decades.

Academic outputs include journal articles, open source software, and open data products in an effort to reach as wide of an academic audience as possible, and to diversify the delivery channel so that outputs provide value in a range of contexts. The impact strategy is structured around two main areas: establishing constant communication with stakeholders through bi-directional dissemination; and data insights broadcast, which will ensure the data and evidence generated reach their intended users.

Planned Impact

This project takes impact very seriously and, accordingly, has planned a careful strategy to maximise the range of actors and the extent to which they will benefit from its outputs. Ultimately, the project will help to study, plan and manage cities in the UK. Its output portfolio includes academic deliverables, such as active participation in world conferences and publication of articles in internationally renowned peer-reviewed journals, as well as a wide range of items, such as engagement workshops, open source software and data products, specifically targeted at non-academic actors.

There are four key (non-academic) stakeholder groups who will benefit directly and indirectly from this project: first, local governments, which are in charge of shaping policies that affect the way activities are spatially distributed within their boundaries; second, central government, which needs consistent measures across the country to assess both the global evolution of the urban system, and to which extent different cities are changing in different ways following systematic patterns (e.g. north-south divide); third, national data organisations such as the Ordnance Survey and the Office of National Statistics, whose primary mission is to develop evidence and products that collectively inform and measure different aspects of society and environment in the UK; and, fourth, the general public, for the majority of whom cities are their home, and are interested in better understanding how the building blocks that make them up are distributed over space and change over time. Please see the "Academic beneficiaries" section for a detailed list of beneficiaries within the broader scientific community.

The research will be relevant to these beneficiaries through the following three channels:

1) By providing much needed, timely and detailed evidence about the nature of urban form in the UK and its evolution over time.
2) By driving better decisions about how cities are planned and managed, made possible thanks to the combination of data and insights delivered through appropriate channels to the relevant stakeholders.
3) By enabling a better understanding of the structure and form of cities, as well as how they evolve over time, preparing us to design better future cities.

To ensure relevant stakeholders have the opportunity to benefit from the project, the PI has designed a careful impact plan structured along two main dimensions: data insights broadcasting, and bi-directional dissemination. Through a diverse range of delivery channels (open data products, open source software packages, local government atlases, policy brief), the project will broadcast outputs of the project of direct relevance to non-academic stakeholders, including both direct evidence (e.g. data) as well as insights from analysis carried out that might be of interest to decision making (e.g. local atlases, policy brief). In parallel, a series of events designed to foster interaction and generate collaboration will ensure that findings and research in the project is co-designed with stakeholders, and that there is a constant channel of bi-directional communication that keeps beneficiaries informed and the project relevant.

Publications

10 25 50
publication icon
Arribas-Bel D (2021) Open data products-A framework for creating valuable analysis ready data. in Journal of geographical systems

publication icon
Boeing G (2021) GIS and Computational Notebooks in Geographic Information Science & Technology Body of Knowledge

publication icon
Franklin R (2020) Who Counts? Gender, Gatekeeping, and Quantitative Human Geography in The Professional Geographer

publication icon
Lumnitz S (2020) splot - visual analytics for spatial statistics in Journal of Open Source Software

publication icon
Rey S (2021) The PySAL Ecosystem: Philosophy and Implementation in Geographical Analysis

 
Description The project has so far, achieved the following key milestones since it start:

Conceptual developments
---------------------------------

- We have developed the concept of "enclosed tessellations" as the atomic units to organically grow spatial signatures, the fundamental building block of an Urban Grammar
- We have refined and clearly proposed the notion of Spatial Signatures in a paper that is currently under review, where we also apply it to five world cities that represent very different histories, geographies and morphologies.
- Using the concept of Spatial Signatures, we have deployed it to the entirety of Great Britain, building a dataset of more than 300 characters for over 14 small areas, and used it to develop a classification of signatures. An Open Data Product with the British signatures is currently available from the ESRC funded Consumer Data Research Centre data portal, and a companion data descriptor is under review, and an empirical paper analysing the main patterns of the signatures we find is under preparation
- We have begun work to develop AI models that are able to identify our spatial signatures from satellite imagery. Results are preliminary but promising.


Data developments
-------------------------

- We have developed a dataset of enclosed tessellation cells for Great Britain (+14m polygons) and enriched it with over 300 characters relating to urban form and function. The database is available as part of our spatial signatures Open Data Product
- We have locally mirrored a cloudless mosaic of satellite imagery at 10m resolution (Sentinel 2) for the entire extent of Great Britain, the largest component of all the data acquisition requirements of the project, as well as the vast majority of vector datasets required to develop a classification of spatial signatures in Great Britain
- We have ingested over 2TB of raw Sentinel 2 imagery from Sentinel hub and have started using it as additional data to train our models
Exploitation Route The project is making fantastic progress and is scheduled to have deliverables to broader audiences later in the year
Sectors Communities and Social Services/Policy,Construction,Digital/Communication/Information Technologies (including Software),Energy,Environment,Leisure Activities, including Sports, Recreation and Tourism,Government, Democracy and Justice,Retail,Transport

URL https://urbangrammarai.xyz
 
Description The Spatial Signatures open data product this project released in 2022 is currently being used in one of the pilot project for the National Land Data Programme, run by the Geospatial Commission. This project is feeding into the development of a framework to shape the next generation of land use data in the UK.
First Year Of Impact 2023
Sector Communities and Social Services/Policy,Environment,Government, Democracy and Justice
Impact Types Policy & public services

 
Description National Land Data Programme interview
Geographic Reach National 
Policy Influence Type Contribution to a national consultation/review
 
Description Assuring safe port navigation by applying machine learning (ML) to wave data for automated monitoring of changes in nearshore bathymetry
Amount £169,232 (GBP)
Organisation Department of Transport 
Sector Public
Country United Kingdom
Start 10/2020 
End 09/2021
 
Description Building and shaping cities for a healthy and prosperous future
Amount € 100,000 (EUR)
Funding ID SR21-00068 
Organisation La Caixa Banking Foundation 
Sector Private
Country Spain
Start 01/2022 
End 12/2023
 
Description ITINERANT: InequaliTies IN Experiencing uRbAn fuNcTion (Cuebiq Data subscription)
Amount £14,000 (GBP)
Organisation Engineering and Physical Sciences Research Council (EPSRC) 
Sector Public
Country United Kingdom
Start 12/2021 
End 06/2022
 
Description Inequalities in experiencing urban function (ITINERANT)
Amount £96,624 (GBP)
Organisation Alan Turing Institute 
Sector Academic/University
Country United Kingdom
Start 08/2021 
End 03/2022
 
Title Classifying urban form at national scale: The British morphosignatures 
Description Data produced for the Classifying urban form at national scale: The British morphosignatures research paper. More details about the project can be found at the project website https://urbangrammarai.xyz. The paper is available from https://doi.org/10.17868/strath.00080527. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact Public access to the data generated as part of the research and increased transparency/reproducibility of the core research results presented in the paper 
URL https://figshare.com/articles/dataset/Classifying_urban_form_at_national_scale_The_British_morphosig...
 
Title Classifying urban form at national scale: The British morphosignatures 
Description Data produced for the Classifying urban form at national scale: The British morphosignatures research paper. More details about the project can be found at the project website https://urbangrammarai.xyz. The paper is available from https://doi.org/10.17868/strath.00080527. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact Public access to the data generated as part of the research and increased transparency/reproducibility of the core research results presented in the paper 
URL https://figshare.com/articles/dataset/Classifying_urban_form_at_national_scale_The_British_morphosig...
 
Title Data for Spatial Signatures: Understanding (urban) spaces through form and function 
Description The data resulting from the research article published as Arribas-Bel, D. and Fleischmann, M. 'Spatial Signatures: Understanding (urban) spaces through form and function' Habitat International, 128, (102641). doi:10.1016/j.habitatint.2022.102641. The complete data for Dar es Salaam and Houston are available upon request due to the file size over the figshare limit (6.8GB and 22GB respectively). The signature delineation is available for all case studies. The code is available at https://github.com/urbangrammarai/spatial_signatures_concept 
Type Of Material Database/Collection of data 
Year Produced 2023 
Provided To Others? Yes  
Impact Public access to the data generated as part of the research and increased transparency/reproducibility of the core research results presented in the paper 
URL https://figshare.com/articles/dataset/Data_for_Spatial_Signatures_Understanding_urban_spaces_through...
 
Title Data for Spatial Signatures: Understanding (urban) spaces through form and function 
Description The data resulting from the research article published as Arribas-Bel, D. and Fleischmann, M. 'Spatial Signatures: Understanding (urban) spaces through form and function' Habitat International, 128, (102641). doi:10.1016/j.habitatint.2022.102641. The complete data for Dar es Salaam and Houston are available upon request due to the file size over the figshare limit (6.8GB and 22GB respectively). The signature delineation is available for all case studies. The code is available at https://github.com/urbangrammarai/spatial_signatures_concept 
Type Of Material Database/Collection of data 
Year Produced 2023 
Provided To Others? Yes  
Impact Public access to the data generated as part of the research and increased transparency/reproducibility of the core research results presented in the paper 
URL https://figshare.com/articles/dataset/Data_for_Spatial_Signatures_Understanding_urban_spaces_through...
 
Title Geographical Characterisation of British Urban Form and Function using the Spatial Signatures Framework 
Description Spatial signatures characterise space based on form and function in a way designed to understand urban environments. This data product, part of the Urban Grammar project, contains a typology of spatial signatures in Great Britain. Each type has a distinct character capturing what the place looks like (form) and how it is used (function). The data product contains bespoke Signature geometry with signature type, summary of input variables per each geometry and per each type, interpolation of signature types to OA and LSOA geometry and short pen portraits for the typology, shorthand descriptions of the characteristics of each signature type. The interactive map showing the typology is available at https://urbangrammarai.xyz/great-britain/. More details about the project can be found at the project website https://urbangrammarai.xyz. 
Type Of Material Database/Collection of data 
Year Produced 2022 
Provided To Others? Yes  
Impact Public access to the data generated as part of the research and increased transparency/reproducibility of the core research results presented in the paper 
URL https://figshare.com/articles/dataset/Geographical_Characterisation_of_British_Urban_Form_and_Functi...
 
Title Geographical Characterisation of British Urban Form and Function using the Spatial Signatures Framework 
Description Spatial signatures characterise space based on form and function in a way designed to understand urban environments. This data product, part of the Urban Grammar project, contains a typology of spatial signatures in Great Britain. Each type has a distinct character capturing what the place looks like (form) and how it is used (function). The data product contains bespoke Signature geometry with signature type, summary of input variables per each geometry and per each type, interpolation of signature types to OA and LSOA geometry and short pen portraits for the typology, shorthand descriptions of the characteristics of each signature type. The interactive map showing the typology is available at https://urbangrammarai.xyz/great-britain/. More details about the project can be found at the project website https://urbangrammarai.xyz. 
Type Of Material Database/Collection of data 
Year Produced 2023 
Provided To Others? Yes  
Impact Public access to the data generated as part of the research and increased transparency/reproducibility of the core research results presented in the paper 
URL https://figshare.com/articles/dataset/Geographical_Characterisation_of_British_Urban_Form_and_Functi...
 
Title Geographical Characterisation of British Urban Form and Function using the Spatial Signatures Framework 
Description Spatial signatures characterise space based on form and function in a way designed to understand urban environments. This data product, part of the Urban Grammar project, contains a typology of spatial signatures in Great Britain. Each type has a distinct character capturing what the place looks like (form) and how it is used (function). The data product contains bespoke Signature geometry with signature type, summary of input variables per each geometry and per each type, interpolation of signature types to OA and LSOA geometry and short pen portraits for the typology, shorthand descriptions of the characteristics of each signature type. The interactive map showing the typology is available at https://urbangrammarai.xyz/great-britain/. More details about the project can be found at the project website https://urbangrammarai.xyz. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact This dataset, part of an ongoing effort to build an Open Data Product on (urban) form and function in Great Britain, provides an analysis-ready layer that uses hundreds of characters for millions of small areas to delineate regions (signatures) with a similar bundle of form and function. It was released in November'21 and is expected it will be used by other researchers, practitioners, and policy makers. 
URL https://figshare.com/articles/dataset/Geographical_Characterisation_of_British_Urban_Form_and_Functi...
 
Title Geographical Characterisation of British Urban Form and Function using the Spatial Signatures Framework 
Description Spatial signatures characterise space based on form and function in a way designed to understand urban environments. This data product, part of the Urban Grammar project, contains a typology of spatial signatures in Great Britain. Each type has a distinct character capturing what the place looks like (form) and how it is used (function). The data product contains bespoke Signature geometry with signature type, summary of input variables per each geometry and per each type, interpolation of signature types to OA and LSOA geometry and short pen portraits for the typology, shorthand descriptions of the characteristics of each signature type. The interactive map showing the typology is available at https://urbangrammarai.xyz/great-britain/. More details about the project can be found at the project website https://urbangrammarai.xyz. 
Type Of Material Database/Collection of data 
Year Produced 2021 
Provided To Others? Yes  
Impact This dataset, part of an ongoing effort to build an Open Data Product on (urban) form and function in Great Britain, provides an analysis-ready layer that uses hundreds of characters for millions of small areas to delineate regions (signatures) with a similar bundle of form and function. It was released in November'21 and is expected it will be used by other researchers, practitioners, and policy makers. 
URL https://figshare.com/articles/dataset/Geographical_Characterisation_of_British_Urban_Form_and_Functi...
 
Description Cluster on Urban Form 
Organisation National Center for Scientific Research (Centre National de la Recherche Scientifique CNRS)
Country France 
Sector Academic/University 
PI Contribution The team contributes expertise on geographic data science and quantitative urban analysis
Collaborator Contribution We benefit from a wider community of specialists on urban morphometrics and from the intellectual infrastructure provided by the consortium, which we expect to operationalise into future grant applications
Impact We are in the final stages of signing a memorandum of understanding between the partner universities and have started discussion for future collaborative projects
Start Year 2020
 
Description Cluster on Urban Form 
Organisation University of Strathclyde
Country United Kingdom 
Sector Academic/University 
PI Contribution The team contributes expertise on geographic data science and quantitative urban analysis
Collaborator Contribution We benefit from a wider community of specialists on urban morphometrics and from the intellectual infrastructure provided by the consortium, which we expect to operationalise into future grant applications
Impact We are in the final stages of signing a memorandum of understanding between the partner universities and have started discussion for future collaborative projects
Start Year 2020
 
Description Cluster on Urban Form 
Organisation University of Twente
Country Netherlands 
Sector Academic/University 
PI Contribution The team contributes expertise on geographic data science and quantitative urban analysis
Collaborator Contribution We benefit from a wider community of specialists on urban morphometrics and from the intellectual infrastructure provided by the consortium, which we expect to operationalise into future grant applications
Impact We are in the final stages of signing a memorandum of understanding between the partner universities and have started discussion for future collaborative projects
Start Year 2020
 
Description Isaac Newton Institute - Infectious Dynamics of Pandemics 
Organisation Isaac Newton Institute for Mathematical Sciences
Country United Kingdom 
Sector Academic/University 
PI Contribution I participated on an early workshop of the meeting "Infectious Dynamics of Pandemics: Mathematical and statistical challenges in understanding the dynamics of infectious disease pandemics" programe ("Models for an exit strategy"), which resulted in a collective publication at the Proceedings of the Royal Society B (Thompson et al., 2020; listed on publications).
Collaborator Contribution I learned a great deal at the workshop and further writing process from colleagues in a wide variety of disciplines. In particular, their insights on transmission of viruses are important to the ongoing work on the project to conceptualise urban spaces in a way that is useful for other disciplines.
Impact Thompson, R. N.; Hollingsworth, T. D; Isham, V.; Arribas-Bel, D.; Ashby, B.; Britton, T.; Challenor, P.; Chappell, L. H. K.; Clapham, H.; Cunniffe, N. J.; Dawid, A. P.; Donnelly, C. A.; Eggo, R. M.; Funk, S.; Gilbert, N.; Glendinning, P.; Gog, J. R.; Hart, W. S.; Heesterbeek, H.; House, T.; Keeling, M.; Kiss, I. Z.; Kretzschmar, M. E.; Lloyd, A. L.; McBryde, E. S.; McCaw, J. M.; McKinley, T. J.; Miller, J. C.; Morris, M.; O'Neill, P. D.; Parag, K. V.; Pearson, C. A. B.; Pellis, L.; Pulliam, J. R. C.; Ross, J. V.; Tomba, G. S.; Silverman, B. W.; Struchiner, C. J.; Tildesley, M. J.; Trapman, P.; Webb, C. R.; Mollison, D.; Restif, O. (2020) "Key questions for modelling COVID-19 exit strategies". Proceedings of the Royal Society B: Biological Sciences. 287(1932), 20201405. 10.1098/rspb.2020.1405 This is an extremely interdisciplinary collaboration among scholars (epidemiology, virology, biostatistics, mathematical sciences, geography, urban studies, network science) from several international institutions.
Start Year 2020
 
Title Areal interpolation speed-up's 
Description In our work in the WP2, we need to link many sources of data, each of them attached to different geographies. We have to transfer data from various sources, linked to output areas, urban blocks or other spatial units to our own bespoke set of geographies. Therefore, we often need to do areal interpolation to correctly map data from one layer to another. Luckily, the open-source Python ecosystem can help. Tobler, a part of PySAL family, is a library for areal interpolation and dasymetric mapping which already offered what we needed. However, our data tends to be large, up to 15 million rows on which we need to interpolate several hundreds of thousands of rows of input data. That can take a while, so each performance improvement can help a lot. Speeding up existing functions We have looked into the existing code of tobler and contributed to the refactoring if its area_interpolate function. The original implementation was using custom code for spatial indexing, which was replaced by a performant vectorised implementation based on the more recent pygeos project. With sample data, we've been able to speed up the interpolation from 2.4 seconds to less than 400 milliseconds, having the same result 6x faster. Such an improvement is great, but it still uses only a single core (as most of the geospatial code to be honest), leaving the rest in a modern computer (four or more cores is not uncommon these days) lazily laying around. We have tried to change this and contributed a (still experimental) parallel implementation of the same algorithm (based on joblib). The resulting improvements depend a lot on the input data structure, and we are still working of further tweaking, but certain combinations can provide significant speedups, which can later save us some time. New addition Interpolation is not always the best way of transferring data from one layer to the other, especially for categorical variables. We have found ourselves in need of a spatial join based on the largest intersection between geometries - the category from the largest intersection gets joined to the target geometry. Such a function was not available (at least we have not found one) so we have developed it. Once it was done and battle-tested on our application, we have contributed it as a new addition to tobler. Everyone can now use it as area_join. ## Full list of contributions The complete list of pull requests implementing code resulting from the Urban Grammar AI project is below: use STRtree.query_bulk in _area_tables_binning #110 STRTree parallel implementation #112 add area_join for join based on the largest intersection #119 fix parallel binning #123 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2020 
Open Source License? Yes  
Impact This contribution has made possible completion of a major milestone of Work Package I in the project, and has led to further engagement with the tobler development team, which is funding the package partly through projects with the National Science Foundation in the US 
URL https://urbangrammarai.github.io/blog/post6_tobler.html
 
Title Implementing morphological functions 
Description Processing of data within WP1 and morphometric assessment within WP2 entail the development of new bespoke algorithms and implementation of some which are currently available in the Python ecosystem. However, even those already existing were often not performant enough for the scale of this project. Speedups of existing code As part of the data processing stage of the project, we have refactored some of them to gain the performance enhancements we needed. Since we strongly believe in replicability of research, all software developed within Urban Grammar AI project should be available for other researchers, optimally packaged in a friendly shape of a Python library. At the same time, we want to support open-source software which we use for the research. We think the natural approach is to include enhancements made within the area of urban morphometrics to momepy an existing toolkit for urban morphology. WP2 heavily builds on momepy's code and every relevant piece of code we made is now merged back into momepy. That covers both performance-focused changes to implementation (#219, #209, #207, #205), mostly based on pygeos and vectorization, and new additions. New functionality Two key features of Spatial Signatures, the concepts of enclosures and enclosed tessellation are now available in momepy.elements module and you can create both using only a few lines of code. Tools we used to preprocess street network and railway tracks before creating enclosures for a core of a new module momepy.preprocessing, where you can find them as momepy.remove_false_nodes, momepy.close_gaps and momepy.extend_lines. To measure percentiles used as a convolution layer for delineation of Spatial Signatures, you can use momepy.Percentiles and to link enclosed tessellation to street network, there is a new function momepy.get_network_ratio. Full list of contributions The complete list of pull requests implementing code resulting from Urban Grammar AI is below: #219 REF: vectorize StreetProfile #218 ENH: ratio-based network links #217 ENH: extend_lines #215 ENH: Close gaps #214 REF: preprocessing module #213 API: unified Tessellation #212 ENH: Enclosed tessellation #211 ENH: add enclosures function #209 ENH: speedups, Percentiles #207 REF: pygeos-based Tessellation #205 ENH: faster circular compactness #204 ENH: vectorised remove_false_nodes 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2020 
Open Source License? Yes  
Impact The software contributed has made certain key operations in urban morphometrics possible and/or more efficient to compute. We have early signs of wider adoption by the scientific community (eg. contact with researchers at the University of Chicago who are using our software in their project). 
URL https://urbangrammarai.github.io/blog/post4.html
 
Title PySAL/Tobler 
Description Last year, we needed areal interpolation to transfer data from various administrative and statistical spatial units like Output areas to enclosed tessellation cells. To do that, we used the Tobler Python package from PySAL family. And since we needed to scale it to a national level, we spent some time refactoring the tools to make it much more performant. All that is discussed in this blog post. After a year, we needed areal interpolation again. This time, we took a slightly different approach. We did not want to partition the data a priori. We also knew that the first part of the function would work well as we had already reimplemented it before. However, we hit another bottleneck. This time in the second part of the code, we did not touch before. What has happened? It is a bit technical. We need to store a relation between source and target geometries. Say that we have 100 000 source geometries and 1 000 000 target ones. We essentially want an array of 100 000 rows and 1 000 000. That does not fit in memory. But since a lot of the cells of such an array would say 0, we can omit them and use a sparse array (or sparse matrix) format, which behaves like a normal dense array but is way more memory efficient. However, there are multiple ways of storing the data in a sparse array. Tobler was using DOK - Dictionary of Keys. That is really fast if you need to access individual records quickly. But that is not the application we have in tobler. We need quick row indexing (apart from a few other things). Fortunately, the fix was simple. We replaced DOK with CSR - Compressed Sparse Row matrix, and the results we miraculous. While the small test benchmark was about 45x faster, our actual interpolation was faster by several orders of magnitude. The original code did not finish after hours. The new once was done in under a second. Small things, like storage formats, matter. If you want to see the effect of different sparse array formats, see this notebook. And the change in tobler is here and will be part of the next release. 
Type Of Technology Software 
Year Produced 2022 
Open Source License? Yes  
Impact Substantial improvement in computation runtime for some of the critical tasks required to perform areal interpolation 
URL https://urbangrammarai.xyz/blog/post21_tobler_once_again.html
 
Title XYZservices 
Description Source of XYZ tiles providers Within the project, we often need to map the results within different contexts ranging from static to interactive maps. We felt that it could be a smoother experience and built xyzservices. A Python ecosystem offers numerous tools for the visualisation of data on a map. A lot of them depend on XYZ tiles, providing a base map layer, either from OpenStreetMap, satellite or other sources. The issue is that each package that offers XYZ support manages its own list of supported providers. We have built xyzservices package to support any Python library making use of XYZ tiles. I'll try to explain the rationale why we did that, without going into the details of the package. If you want those details, check its documentation (https://xyzservices.readthedocs.io/en/latest/). 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact This package unifies the provision of webmaps to the Python ecosystem. This means that any Python package which requires providers of web tiles can rely on our new standardised approach. Perhaps because of this, the package has quickly become instrumental infrastructure in the Python for geospatial ecosystem. Evidence on this can be seen on this Github issue, which tracks all the external projects which already rely on xyzservices: https://github.com/geopandas/xyzservices/issues/49 
URL https://xyzservices.readthedocs.io/en/stable/
 
Title dask-geopandas mentoring at Google Summer of Code 
Description Members of the Urban Grammar project are getting involved in developing the next generation set of tools for distributing processing of geospatial vector data. In its first part, the Urban Grammar project heavily depends on the processing of vector geospatial data using GeoPandas Python library. However, to scale GeoPandas algorithms to the extent of Great Britain, we need to do more than the library can do by default. GeoPandas operations are currently all single-threaded, severely limiting the scalability of its usage and leaving most of the CPU cores just laying around, doing nothing. Dask is a library that brings parallel and distributed computing to the ecosystem. For example, it provides a Dask DataFrame that consists of partitioned pandas DataFrames. Each partition can be processed by a different process enabling the computation to be done in parallel or even out-of-core. We are using Dask within our workflows in bespoke scripts. However, Dask could provide ways to scale geospatial operations in GeoPandas in a similar way it does with pandas. There has been some effort to build a bridge between Dask and GeoPandas, currently taking the shape of the dask-geopandas library. While that already supports basic parallelisation, which we used in our code, some critical components are not ready yet. That should change during this summer within the Google Summer of Code project Martin is (co-)mentoring. We hope that this effort will allow us to significantly simplify and even speed up the custom machinery we built to create spatial signatures in WP2. 
Type Of Technology New/Improved Technique/Technology 
Year Produced 2021 
Impact We expect the outputs from this project to have wide adoption in academia, industry, and government. Even at its early stage of development, the project has already been adopted by industry leaders such as Microsoft, for its Planetary Computer project. 
URL https://github.com/tastatham/gsoc_dask_geopandas_2021
 
Title martinfleis/clustergram: Version v0.6.0 
Description Clustergram - Visualization and diagnostics for cluster analysis in Python 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact The package provides functionality to easily create clustergrams from a variety of (CPU and GPU) backends within a Python environment. Given clustering is a popular approach in data science, and Python is the main programming language for data science, we expect the package to support the analysis of a large amount of data scientists in academia, industry, and government. 
URL https://zenodo.org/record/4750483
 
Title urbangrammar_graphics 
Description Visual styles for the Urban Grammar AI research project The package provides a unified visual style across all public outputs. That entails a colour map, plotting styles and map styles. All components are outlined and illustrated below in a demo Jupyter notebook. The style uses custom colours, seaborn predefined styles, custom contextily basemaps, and additonal features collected in urbangrammar_graphics Python package. 
Type Of Technology Software 
Year Produced 2021 
Open Source License? Yes  
Impact The package has given the project a unified look, which makes our public presence (presentations, publications, etc.) more consistent. Furthermore, since it is published under an open source license, the style is available to everyone who finds it useful. 
URL https://github.com/urbangrammarai/graphics/releases/tag/v1.2.3
 
Description British Spatial Signatures 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact On Tuesday, November 30th, Martin gave a lighting talk on the Spatial Signatures of Great Britain at the Alan Turing Insititute event Towards urban analytics 2.0 that was in person in Leeds, UK. The talk introduced the classification of Great Britain that is now available in the form of an interactive map and for download either from the Consumer Data Research Centre's open data portal or the archived version from figshare.
Year(s) Of Engagement Activity 2021
URL https://youtu.be/ooOb6LyY5f0
 
Description Classifying urban form at a national scale 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact On June 30th, Martin presented a classification of Great Britain based on the form component of Spatial Signatures at the International Seminar on Urban Form 2021, which was held virtually in Glasgow, Scotland. This was the first time we have presented results covering the whole GB, albeit using a single component only (function has been temporarily excluded). The work was received positively and spurred a bit of discussion. Within a few months, the results should be published in the conference proceedings.
Year(s) Of Engagement Activity 2021
URL https://www.youtube.com/watch?v=8d4e1uVMg3g
 
Description GISRUK presentation 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact On April 7th, we had a chance to attend the first large in-person academic conference since the project started more than two years ago. Martin joined GISRUK, coincidentally organised by the Geographic Data Science Lab, and presented ongoing work on developing an AI model to detect spatial signatures based on satellite imagery. The talk focused on the progress we have made so far and specifically on geographical questions arising when we try to link granular geometry of signatures to satellite images and rectangular chips sampled from them.

We have received good feedback and suggestions for the next steps we are currently trying to implement in our plans. For anyone interested, the slides presented during the conference are available here or as a PDF (33MB) here. The extended abstract of the talk is also available in the conference proceedings.
Year(s) Of Engagement Activity 2022
URL https://zenodo.org/record/6411621#.ZBJRIB_MLe8
 
Description GeoPython'22 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Industry/Business
Results and Impact Earlier this month, Martin represented the Urban Grammar at this year's edition of GeoPython. During his time there, he kept himself rather busy. Here is a quick summary with references of all the bits he participated in:

Delivered the talk "Open by Default - Developing reproducible, computational research" (slides available here)

Co-delivered the talk "State of GeoPandas", together with Joris Van den Bossche (slides available here)

Co-delivered with Joris Van den Bossche the workshop "Scaling up vector analysis with Dask-Geopandas (materials available in this repo)
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post35_geopython.html
 
Description Heseltine Institute Smart Cities workshop 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Against the fitting backdrop of Liverpool's Sensor City, the Heseltine Institute's Building Smart Cities symposium aimed to inform and advance scholarly debates, public conversation and policy development around the 'smart city' agenda.

Featuring leading academics, policymakers and tech professionals locally, nationally and internationally, the event centred upon the performance, perils, and untapped potentialities of embracing further a 'smart city' agenda. In the age of the 'smart city', municipal leaders everywhere are confronted with the same imperative of harnessing a new generation of smart technology if they are to tackle effectively the most pressing economic, social and environmental problems that face their cities. Yet, whilst the idea of the 'smart city' is one full of acknowledged possibility, the ownership, stewardship and deployment of smart technology have equally provided cause for concern and caution.

Speakers, panellists and attendees addressed a number of pressing questions relating to the 'smart city' concept over the course of the day - including what a citizen-centred 'Smart Liverpool City Region' might look like and how, more practically, it might be realised. In support of this work, the Heseltine Institute used the event to launch its own contribution to the debate, a Good Practices Reference Guide by experts BABLE, together with an accompanying Position Statement.
Year(s) Of Engagement Activity 2020
URL https://www.liverpool.ac.uk/heseltine-institute/events/buildingsmartcitieswithcitizensandforthepubli...
 
Description ISUF Italy 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact In early June, Martin attended the 6th ISUF Italy conference "Morphology and Urban Design. New strategies for a changing society", organised by the University of Bologna in a hybrid mode (Martin was present online). We took the opportunity to talk about Detecting urban typology from multispectral satellite imagery using neural networks, where we combined the work we currently focus on in the Urban Grammar project and the one published in a recent CEUS paper. It allowed us to illustrate the application of remote sensing and neural networks on urban form in both supervised (detection of signatures from Sentinel 2) and unsupervised (the CEUS paper) use cases. With the audience composed of the ISUF community revolving heavilty around architecture, the talk started an engaging discussion on the current limits of openly available satellite imagery in the context of urban morphology.

The talk itself was not recorded but the slides are available from the usual place on HTML (this time we couldn't build a PDF for download satisfactorily).
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post26_isufitaly.html
 
Description OECD Geospatial Lab Kickoff 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact In December 2020, the OECD held an invite-only workshop to launch their Geospatial Lab. I was invited to be one of the speakers on the launch and to consult with them on future activity for the Lab.
Year(s) Of Engagement Activity 2020
URL https://www.oecd.org/regional/regional-statistics/networks-and-communities.htm
 
Description Project website 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact This is the main project website, which acts as a central hub of all the materials generated and released by the project. It is a living document that is constantly evolving.
Year(s) Of Engagement Activity 2021,2022
URL https://urbangrammarai.xyz
 
Description SEA Keynote 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact Earlier this month, Dani delivered this year's Spatial Economic Analysis keynote at the RSAI-BIS Annual Meeting in Sirling (Scotland). The talk provided an overview of the motivation and progress in the Urban Grammar project to date. The programme of the conference, including Dani's keynote is available here. The slides Dani used are available in the usual repository (HTML and PDF).
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post39_sea_keynote.html
 
Description Second advisory board 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact On April 15th. 2021, we held the second meeting of the Advisory Board for the project. We are delighted that all board members joined us on Zoom for a few hours of exciting discussions on the progress and the future of the project.

Dani started with an overview of our progress since the last meeting, which you can check in his UBDC talk. We followed by the focused discussion on the concepts of Spatial Signature and Enclosed Tessellation and our initial paper illustrating both on the sample of cities worldwide. We discussed the clarity of our ideas and the needs for new spatial units and classification methods, and their potential drawbacks and enhancements. In the last part, we tried to zoom out to see a bigger picture and fit the research within existing projects within academia and the public sector.

After three hours of a very fruitful discussion, we finished with a lot of food for thought and ideas to be explored in the future. Let's just hope that the Advisory Board meetings will soon happen physically in Liverpool to have an even more productive and friendly environment!
Year(s) Of Engagement Activity 2021
URL https://urbangrammarai.xyz/blog/post11_second_advisory_board.html
 
Description Spatial Signatures - (Urban) Form & Function in Great Britain 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact On Monday November 29th, Dani gave a talk on the Spatial Signatures project at the CASA seminars. The talk covered known bits from previous talks, mostly about framing of the problem and the foundational blocks of the Spatial Signatures but, more importantly, presented for the first time our results from the British signatures. The audience was engaging and had many super interesting questions. Thanks!
Year(s) Of Engagement Activity 2021
URL https://www.youtube.com/watch?v=drqIXgK8ptI
 
Description Spatial Signatures - (Urban) Form & Function in Great Britain 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact On June 30th, Dani provided an overview of ongoing work to build Spatial Signatures for all of Great Britain at the Turing's Urban Analytics monthly meetup. This was the first public presentation covering our work at a national scale and we are excited at how positively it was received.
Year(s) Of Engagement Activity 2021
URL https://www.youtube.com/watch?v=fHccCnUF9yc
 
Description Spatial Signatures - Understanding (urban) spaces through form and function 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact On March 16th, Dani presented ongoing work on the development of Spatial Signatures at the University of Glasgow's Urban Big Data Centre webinar series. This was the first time we took the project on tour and it was very well received. With an audience that peaked at about 75 folks and many great questions that extended the session well over one hour, we are super happy with how the foundational ideas of the project were received.
Year(s) Of Engagement Activity 2021
URL https://www.youtube.com/watch?v=5fq1bJX9NcE
 
Description Spatial Signatures: Dynamic classification of the built environment 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact On March 30th, Martin presented the current progress in the development of Spatial Signatures at the Spatial Analytics + Data Seminar Series organised by the University of Newcastle, the University of Bristol and the Alan Turing Institute. Martin presented the basics of urban morphometrics, showing examples of relevant research based on the momepy Python package to provide a background for the second part of the talk focusing on Spatial Signatures. The research was well received and initiated a great discussion, which we hope will continue on some other platforms soon.
Year(s) Of Engagement Activity 2021
URL https://www.youtube.com/watch?v=pTiy6uvhw-4
 
Description Stakeholder engagement workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Dani and Martin participated earlier this month in the stakeholder engagement workshop organised by the ITINERANT project. This was an event where the project presented final results to a series of stakeholders including the ONS Data Science Campus, the Liverpool City Region, or the UK 2070 Commission. The event was held in Liverpool but streamed online and the talks are available on YouTube now (Spatial Signatures starting at minute 23).
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post32_itinerant.html
 
Description Talk at the ONS Data Science Seminar Series 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Earlier this month, Dani presented the spatial signatures at the ONS Data Science Seminar Series, run by their Data Science campus. The talk covered most of the work undertaken to date on the project and it is a good summary of many of its achievements. You can access the series page here and watch the talk below:
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post37_ons_talk.html
 
Description Talk at the RSS Merseyside 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Last month, Dani participated in the "Using open data sources" event put together by the lovely folks at the Merseyside chapter of the RSS and HiPy. There, he delivered the talk "Open by default - Developing reproducible, computational research". You can find the slides on the usual place (or here and in PDF directly). The video is also available on YouTube.
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post29_obd.html
 
Description The Urban Grammar on MapScaping podcast 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Dani had a chance to speak with Daniel from the Mapscaping podcast about the Urban Grammar project.
Year(s) Of Engagement Activity 2021
URL https://mapscaping.com/podcast/urban-grammar-form-function-culture-and-satellite-imagery/
 
Description Third Advisory Board 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact On December 9th. 2021, we held the meeting of the Advisory Board for the project. Still being limited by the global pandemic, we had a frutiful Zoom call filled with an exciting discussion on the opportunities Spatial Signatures offer.

The session started with an overview of the current progress focusing on the current progress and explanation of the whole process of generating spatial signatures from data acquistion to empirical exploration. We then spent some time discussed the open infrastruture built around the project resulting in an open data product and maximum transparency of the process behind it. We followed by the focused discussion on dissemination and impact of the classification.

Three hours later, we finished with a lot of ideas and potential research avenues and collaborations to be explored. We hope that the situation permits another Advisory board meeting soon and hopefully, in person.
Year(s) Of Engagement Activity 2021
URL https://urbangrammarai.xyz/blog/post20_third_advisory_board.html
 
Description Urban Grammar 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Other audiences
Results and Impact On November 23rd, Dani gave a quick overview of the Urban Grammar project at the monthly catch up of the Alan Turing Institute. This is an internal call open to all fellows and staff. Dani gave an overview of the project, and covered in a bit more detail the aspects that have been completed already, including the development of a Spatial Signature classification for Great Britain.
Year(s) Of Engagement Activity 2021
URL https://urbangrammarai.xyz/talks/202111_ati/
 
Description Visit to Chicago 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact The first week of May, Dani visited the University of Chicago. As part of the trip, he (re-)connected with folks at the Center for Spatial Data Science and met a bunch of new friends at the Mansueto Institute for Urban Innovation. All in all, it was a fantastic week where there was even a bit of time to discuss all things urban form and function.

On May 4th, he delivered the lecture "'Open by Default' - Developing reproducible, computational research" on open workflows for modern computational research at one of the GIS courses offered at Chicago and taught by Dr. Marynia Kolak. The slides of the talk are available at the Urban Grammar's talks repository in our two common formats:

HTML for the browser

PDF for download

This was a particularly tricky talk to conceive and prepare. Conceptually, it was a bit out of our "comfort zone" in that it is not really about the research we are doing at the Urban Grammar, but about the process we follow to realize it. It was also hard to structure it in a way that made sense because it pulled from many different aspects of the project, from the approach we take to writing slide decks to the computational infrastructure on which all of our computations rely. We are nevertheless happy with the outcome. It's not ideal, and we will probably refine it in successive iterations (we're planning on giving similar talks in the near future, stay tuned!), but this is a great start.
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post27_chicago.html
 
Description Visit to Ordnance Survey 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Industry/Business
Results and Impact On May 17th and 18th, Martin and Dani spent a week visiting the Ordnance Survey in Southampton. The trip was planned as an institutional visit to exchange ideas, discuss the project and, more generally, interact with our advisory board member, Dr. Isabel Sargent and her team at OS Research.

We had a fantastic time. We presented our current work on using computer vision, a bit of geographic magic, and satellite imagery to automate the recognition of our upcoming spatial signatures for Great Britain. You can check out the slides we used for that presentation in our usual talks repository in our two common formats:

HTML for the browser

PDF for download

Beyond the purely scheduled, we had the oportunity to see OS from the inside and to chat to a lot of really clever people about our project, its value, and potential solutions to the challenges we're currently facing.
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post24_os_visit.html
 
Description Workshop Turing/Met Office 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact In December 2021, The Alan Turing Institute and the Met Office organised a joint workshop to explore collaborations and further engagement, on the back of Memorandum of Understanding that has been signed. As part of the workshop, Dani presented on the Urban Grammar, its vision, current progress, and future plans.
Year(s) Of Engagement Activity 2021
 
Description World Urban Forum 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Last June, Martin and Dani attended the World Urban Forum. It was a fascinating experience in many ways. Dani wrote down his thoughts on a blog post you can read here.
Year(s) Of Engagement Activity 2022
URL https://urbangrammarai.xyz/blog/post31_wuf.html