Geolinking UKcensus 1841-1921: new data, methods, and perspectives
Lead Research Organisation:
UNIVERSITY OF CAMBRIDGE
Abstract
This project will develop a new methodology to create the first geocoded record linking of all censuses for England and Wales (EW) between 1841 and 1921 and use this unprecedented longitudinal dataset to analyse internal migration, urbanisation, and women's work and demographic outcomes.
It combines advances in Machine Learning (ML), Handwritten Text Recognition (HTR), document understanding, geocoding, and both probabilistic and ML record linking. By bringing together the expertise of leading researchers in these fields, The National Archives, and the two largest providers of census data, FindMyPast and Ancestry, we will significantly improve what is currently possible for census linking and offer the first set of high-precision, low-bias links for EW. Data for Scotland is held separately and still undergoing work by Digitising Scotland. If funded, we hope to have more leverage to convince them to share their data so we can include Scotland in our analyses.
Our methodological development will create a new class of linking approach altogether – based on token-level probability matching – which has the potential to significantly increase current matching rates and quality. We will also improve the linkage of women by pioneering the inclusion of marriage registers at scale to identify married women across censuses, and we will augment the census by geocoding all individuals to the streets and buildings in which they resided, allowing for even more granular spatial analysis.
Our new data will transform the way we study internal migrations. Studies of nineteenth and twentieth-century migratory patterns have primarily relied on the comparison of places of birth and enumeration at each census, which prevents the analysis of individual and time-dynamic components of migrations. Using longitudinal data, we will finally be able to uncover these very fine-grained migratory paths for every decade, allowing us to provide a much fuller account of the economic, lifecycle, and geographic determinants of internal migrations in a period of rapid social change and sustained urbanisation.
Longitudinal geocoded data will also transform our understanding of urban growth. We will be able to track residential movements within highly specific urban spaces (down to the building level) on every census date, allowing us to provide a much fuller picture of residential segregation patterns, changing urban systems, urban morphology, and their interactions with transport infrastructure and institutional settings.
Finally, we will be able for the first time to reconstruct individual marriage and fertility timelines over long periods on a national scale and at a very high level of spatial disaggregation. Currently, the interaction between women’s work, marriage, and fertility cannot be satisfactorily studied because we cannot observe the occupational and household status of women before marriage and during their childbearing years. We will now be able to identify the precise timing of marriage and dramatically improve our estimates of parity progression to study this phenomenon with unprecedented precision.
We will make all the links available to researchers, unlocking an infinite number of future projects and research questions and creating a fundamental new resource for British historians, economic historians, demographers, economists, sociologists, and many more. Our partners, genealogists, and the broader public will benefit from a much-enhanced ability to find data about their ancestors using our linked data. We will also develop with secondary-school teachers ready-for-classroom pedagogical resources and lesson plans using our new data to improve the teaching of history and geography.
It combines advances in Machine Learning (ML), Handwritten Text Recognition (HTR), document understanding, geocoding, and both probabilistic and ML record linking. By bringing together the expertise of leading researchers in these fields, The National Archives, and the two largest providers of census data, FindMyPast and Ancestry, we will significantly improve what is currently possible for census linking and offer the first set of high-precision, low-bias links for EW. Data for Scotland is held separately and still undergoing work by Digitising Scotland. If funded, we hope to have more leverage to convince them to share their data so we can include Scotland in our analyses.
Our methodological development will create a new class of linking approach altogether – based on token-level probability matching – which has the potential to significantly increase current matching rates and quality. We will also improve the linkage of women by pioneering the inclusion of marriage registers at scale to identify married women across censuses, and we will augment the census by geocoding all individuals to the streets and buildings in which they resided, allowing for even more granular spatial analysis.
Our new data will transform the way we study internal migrations. Studies of nineteenth and twentieth-century migratory patterns have primarily relied on the comparison of places of birth and enumeration at each census, which prevents the analysis of individual and time-dynamic components of migrations. Using longitudinal data, we will finally be able to uncover these very fine-grained migratory paths for every decade, allowing us to provide a much fuller account of the economic, lifecycle, and geographic determinants of internal migrations in a period of rapid social change and sustained urbanisation.
Longitudinal geocoded data will also transform our understanding of urban growth. We will be able to track residential movements within highly specific urban spaces (down to the building level) on every census date, allowing us to provide a much fuller picture of residential segregation patterns, changing urban systems, urban morphology, and their interactions with transport infrastructure and institutional settings.
Finally, we will be able for the first time to reconstruct individual marriage and fertility timelines over long periods on a national scale and at a very high level of spatial disaggregation. Currently, the interaction between women’s work, marriage, and fertility cannot be satisfactorily studied because we cannot observe the occupational and household status of women before marriage and during their childbearing years. We will now be able to identify the precise timing of marriage and dramatically improve our estimates of parity progression to study this phenomenon with unprecedented precision.
We will make all the links available to researchers, unlocking an infinite number of future projects and research questions and creating a fundamental new resource for British historians, economic historians, demographers, economists, sociologists, and many more. Our partners, genealogists, and the broader public will benefit from a much-enhanced ability to find data about their ancestors using our linked data. We will also develop with secondary-school teachers ready-for-classroom pedagogical resources and lesson plans using our new data to improve the teaching of history and geography.