Coverage ESRC Data Infrastructure
Lead Research Organisation:
University of Southampton
Department Name: Sch of Economic, Social & Political Sci
Abstract
This research project investigates how to make a consistent assessment of the coverage of different Longitudinal Population Studies (LPS). LPS are very important data investments in the UK, with great potential to help answer a wide range of biomedical, social and economic research questions. It is of crucial importannce to understand when population subgroups are under-represented or missing in the LPS datasets. This will provide evidence whether the conclusions from research are widely generalisable, and also indicate when there is sufficient coverage of particular subgroups for analyses to be of reliable quality. Based on a range of quantitative and alternative measures for coverage, we will develop a toolkit for the consistent measurement and reporting of coverage in LPS, and illustrate its use with two case studies, the 1970 British Cohort Study and UK BioBank. We will also use the toolkit to indicate ways in which coverage can be improved. An intended outcome of the research project is to remove some of the barriers to the use of LPS to answer topical research questions in different social, economic and biomedical disciplines and to maximise the use of the existing LPS.
Organisations
| Description | The different ways in which coverage and representativeness have been described were summarised from the published literature, in particular taking relevant components from a typology of uses of "representative sampling". The features important for understanding coverage and representativeness were summarised - in particular what are features of the data, and what are features of an analysis based on the data. These researches provided the basis for a survey of experts (a Delphi study) to understand their views on what different terms related to these features and concepts mean. This fed into a description of different types of representativeness, which should help to clarify how this term is used in different situations. Different measures and indicators of features of coverage and representativeness were gathered, and their methods reviewed to demonstrate how they could be applied effectively to longitudinal population studies (LPS). All of these indicators must be assessed relative to some variables or measures from a benchmark source, and conclusions about the usefulness of LPS to answer research questions may therefore vary depending on the question being asked and the benchmark information available to make an assessment. There is a strong link to analysis, as some deficiencies in representativeness (particularly) can be remedied by appropriate analysis tools. The indicators were then used to make example assessments from two case studies, the British Cohort Study 1970 and UK BioBank. A new application of principal components analysis was used to assess the ways in which waves of the LPS differed from each other, and to relate this to representativeness. The biplot, a way to visualise principal components, provides a helpful display of differences between the waves of a LPS. |
| Exploitation Route | A webpage with a toolkit for providers and users of longitudinal population study resources will be produced by the research team, but it is not yet available. The research team is discussing further activities with Population Research UK (PRUK), potentially including a wider event involving others working on this topic with other longitudinal population studies; additions to the toolkit; a training course in coverage/representativeness assessment (complement existing training on missing data). The research team plans academic outputs on language and meaning of representativeness, and example applications. The findings should be developed into a checklist for reporting coverage and representativeness, and there will then need to be an effort to get it taken up and embedded in research practice. The wider research community should take up, pilot and develop approaches to measurement and assessment of coverage and representativeness. There will be an eventual need for meta-analysis to assess the link between quality of analyses and indicators of coverage/representativeness. |
| Sectors | Communities and Social Services/Policy Healthcare Government Democracy and Justice |
| Description | End of project international on-line workshop |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | More than 80 people attended a workshop where the findings of the study were presented, and feedback was obtained from participants. In particular feedback on the form of the toolkit was sought, but participants had limited views. |
| Year(s) Of Engagement Activity | 2025 |
| Description | Provide evidence to ESRC for cohort studies review |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Give evidence to ESRC review of cohort studies |
| Year(s) Of Engagement Activity | 2025 |
| Description | Royal Statistical Society's National Statistics Advisory Group |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | NSAG is a body which influences the policies of the UK Statistical System, mainly focused on making statistics serve public uses. |
| Year(s) Of Engagement Activity | 2023,2024,2025 |
| URL | https://rss.org.uk/policy-campaigns/policy-groups/national-statistics-advisory-group/ |