UK-Australia Network Promoting Longitudinal Survey Methods and Analysis
Lead Research Organisation:
University of Essex
Department Name: Inst for Social and Economic Research
Abstract
This proposal would create an international partnership network between the Institute for Social and Economic Research (ISER) at the University of Essex and the Universities of Melbourne and Queensland in Australia. The objective of this network is to build and exchange knowledge focused on the link between panel survey methods and substantive analysis in key policy areas. The funding in this proposal would cover travel, subsistence and costs associated with the organisation of a meeting, two workshops and international researcher exchange visits. The funding would not be used for research or for staff costs.
Those who work trying to improve the quality of survey data generally use a framework of "Total Survey Error", which separates the different types of errors that surveys can have. These errors affect all surveys but are particularly significant on longitudinal surveys where the same people are being interviewed over time. Research into these errors generally focus on summary measures such as the means and percentages generated by the survey. However, most substantive research into important policy areas relies on types of multivariate analysis which use information about the distributions of responses within the data (regression, logit or probit models). The impact of survey error on estimates derived from such multivariate methods cannot be understood by focusing only on marginal summary measures. The results of substantive researchers using multivariate analyses may be seriously distorted by errors that arise due to the design of the survey, for example, if the way that the survey was organised influences the answers that people give in different ways. Little is known about how the design of panel surveys affects the stability of substantive answers across time, particularly in the key policy areas of employment, health, well-being, poverty and family dynamics. To our knowledge research on the causes of longitudinal inconsistencies and, most importantly, their substantive effects is lacking.
This network will provide opportunities to forge long-term relationships with Australian colleagues, to exchange ideas and to promote research on these topics. The network will, first, target methods for improving data quality, both during data collection and after collection. The network will then aim to develop sustainable research collaborations focused on the effect of improved data quality in substantive research, such as measures of employment change, changes in health or well-being, or family dynamics over time. The funding from this scheme would establish the infrastructure of an international partnership which would enable researchers to travel between the UK and Australia to collaborate on research projects. These research projects will be funded separately.
The funding from this scheme would establish the infrastructure of an international partnership which would enable researchers to travel between the UK and Australia to collaborate on research projects. These research projects will be funded separately. The level of commitment to the project shown by the international partners is demonstrated by the level of proposed co-funding. The international partners are pledging direct financial contributions of up to £21,000 over the three year project.
Those who work trying to improve the quality of survey data generally use a framework of "Total Survey Error", which separates the different types of errors that surveys can have. These errors affect all surveys but are particularly significant on longitudinal surveys where the same people are being interviewed over time. Research into these errors generally focus on summary measures such as the means and percentages generated by the survey. However, most substantive research into important policy areas relies on types of multivariate analysis which use information about the distributions of responses within the data (regression, logit or probit models). The impact of survey error on estimates derived from such multivariate methods cannot be understood by focusing only on marginal summary measures. The results of substantive researchers using multivariate analyses may be seriously distorted by errors that arise due to the design of the survey, for example, if the way that the survey was organised influences the answers that people give in different ways. Little is known about how the design of panel surveys affects the stability of substantive answers across time, particularly in the key policy areas of employment, health, well-being, poverty and family dynamics. To our knowledge research on the causes of longitudinal inconsistencies and, most importantly, their substantive effects is lacking.
This network will provide opportunities to forge long-term relationships with Australian colleagues, to exchange ideas and to promote research on these topics. The network will, first, target methods for improving data quality, both during data collection and after collection. The network will then aim to develop sustainable research collaborations focused on the effect of improved data quality in substantive research, such as measures of employment change, changes in health or well-being, or family dynamics over time. The funding from this scheme would establish the infrastructure of an international partnership which would enable researchers to travel between the UK and Australia to collaborate on research projects. These research projects will be funded separately.
The funding from this scheme would establish the infrastructure of an international partnership which would enable researchers to travel between the UK and Australia to collaborate on research projects. These research projects will be funded separately. The level of commitment to the project shown by the international partners is demonstrated by the level of proposed co-funding. The international partners are pledging direct financial contributions of up to £21,000 over the three year project.
Planned Impact
ISER is a world-leader in the production and analysis of longitudinal surveys. It has been responsible for the design and management of the BHPS and more recently Understanding Society: the UK Household Longitudinal Study. ISER is home to the ULSC and is a specialist centre for survey methodology research and postgraduate study.
ISER is an outward-looking Institute, with an international staff and student body, with a vibrant and much in-demand international visitors' programme and with collaboration and partnership at the heart of all of its survey and research activities. It already has a number of programmes of comparative research and is looking to increase this aspect of its methodological and its substantive research programmes.
ISER has a range of well-established partnerships and networks with similar research institutes around the world which are detailed elsewhere in this proposal, and this project aims to build on that portfolio of relationships to the benefit of existing and future longitudinal surveys and related research.
In the short term we will improve the quality of existing longitudinal studies in the UK and Australia, in the medium term we expect to influence and improve the quality of similar surveys around the world and in the longer term to influence for the better the quality of the substantive research undertaken using the data, and the levels of confidence that policy makers, third sector and public sector organisations, the media and the general public have in that data to explore the major social and economic issues we face today and in the future. A particular focus will be on issues at the forefront of the policy agenda not just in the UK but in other countries including wellbeing, health and employment.
Those who will benefit directly from this collaboration are researchers who are based in the UK or Australia. The funding is for travel and subsistence to enable researchers to work together on research projects which will improve the quality of data in longitudinal studies and showcase the effect improved data have on substantive results. These research projects will be funded separately.
Indirectly, those who will benefit from the collaboration are those who rely on analysis from longitudinal data
- policy-makers
- private organisations
- public sector bodies
- third sector bodies
Other potential beneficiaries include the media who rely on accurate, high quality and rigorous data and research to report on, to explain and to discuss the social and economic issues of the day and the general public whose lives are affected by the policy decisions taken by Governments around the world.
Those who commission and manage longitudinal surveys will also benefit from this collaboration. This includes:
In the UK
- Economic and Social Research Council (ESRC)
- Office for National Statistics
- Government Departments (DWP, DfE, DH, DEFRA, DT, DCLG, DCMS)
- Scottish Government
- Welsh Assembly Government
- Centre for Longitudinal Studies
In Australia
- Australian Government Department of Families, Housing, Community Services and Indigenous Affairs.
- Department of Education, Employment and Workplace Relations
- Department of Health and Ageing
- Australian Institute of Family Studies (which runs Longitudinal Study of Australian Children)
- National Centre for Vocational Education Research (which runs the Longitudinal Studies of Australian Youth)
Organisations like these plough large sums of public money and resource into longitudinal studies and associated research. It is essential that that spending can be fully accounted for in respect of the quality and the rigour of the data produced, of the subsequent research carried out making use of that data and of the important and valuable contributions that research makes to society as a whole. The better the quality of the study, the better the quality of the data, the research and the ensuing impact.
ISER is an outward-looking Institute, with an international staff and student body, with a vibrant and much in-demand international visitors' programme and with collaboration and partnership at the heart of all of its survey and research activities. It already has a number of programmes of comparative research and is looking to increase this aspect of its methodological and its substantive research programmes.
ISER has a range of well-established partnerships and networks with similar research institutes around the world which are detailed elsewhere in this proposal, and this project aims to build on that portfolio of relationships to the benefit of existing and future longitudinal surveys and related research.
In the short term we will improve the quality of existing longitudinal studies in the UK and Australia, in the medium term we expect to influence and improve the quality of similar surveys around the world and in the longer term to influence for the better the quality of the substantive research undertaken using the data, and the levels of confidence that policy makers, third sector and public sector organisations, the media and the general public have in that data to explore the major social and economic issues we face today and in the future. A particular focus will be on issues at the forefront of the policy agenda not just in the UK but in other countries including wellbeing, health and employment.
Those who will benefit directly from this collaboration are researchers who are based in the UK or Australia. The funding is for travel and subsistence to enable researchers to work together on research projects which will improve the quality of data in longitudinal studies and showcase the effect improved data have on substantive results. These research projects will be funded separately.
Indirectly, those who will benefit from the collaboration are those who rely on analysis from longitudinal data
- policy-makers
- private organisations
- public sector bodies
- third sector bodies
Other potential beneficiaries include the media who rely on accurate, high quality and rigorous data and research to report on, to explain and to discuss the social and economic issues of the day and the general public whose lives are affected by the policy decisions taken by Governments around the world.
Those who commission and manage longitudinal surveys will also benefit from this collaboration. This includes:
In the UK
- Economic and Social Research Council (ESRC)
- Office for National Statistics
- Government Departments (DWP, DfE, DH, DEFRA, DT, DCLG, DCMS)
- Scottish Government
- Welsh Assembly Government
- Centre for Longitudinal Studies
In Australia
- Australian Government Department of Families, Housing, Community Services and Indigenous Affairs.
- Department of Education, Employment and Workplace Relations
- Department of Health and Ageing
- Australian Institute of Family Studies (which runs Longitudinal Study of Australian Children)
- National Centre for Vocational Education Research (which runs the Longitudinal Studies of Australian Youth)
Organisations like these plough large sums of public money and resource into longitudinal studies and associated research. It is essential that that spending can be fully accounted for in respect of the quality and the rigour of the data produced, of the subsequent research carried out making use of that data and of the important and valuable contributions that research makes to society as a whole. The better the quality of the study, the better the quality of the data, the research and the ensuing impact.
Publications
| Description | This grant was for travel/subsistence to facilitate researchers from UK and Australia to work together on joint funding bids, it did not cover funding for research. However, the network did support collaboration on application to the Australian Research Council (ARC) for the Centre of Excellence for Children and Families over the Life Course, which was successful (AUD $28 million over 7 years). The funding also allowed 6 UK-based and 3 Australia-based researchers to travel, allowing for meetings, discussions and collaborating on research proposals. |
| Exploitation Route | The network has been set up and will continue to work together on potential collaborations. The Centre of Excellence for Children and Families over the Life Course continues to thrive, with network members from the UK and Australia involved as Co-Investigators, as well as researchers from outside the network who will continue to benefit. |
| Sectors | Communities and Social Services/Policy,Education |
| URL | https://www.iser.essex.ac.uk/research/projects/uk-australia-network-project |
| Description | Grant was for travel/subsistence to facilitate researchers from UK and Australia to work together on joint funding bids, and contributed to a number of visits of UK-based researchers to Australia, and Australia-based researchers to the UK. Note: There are no findings directly from this grant because this was for travel/subsistence and was specifically not to be used to fund research. However, collaborations which were facilitated by this funding resulted in published research, submitted research proposals and collaboration on the successful application to the Australian Research Council for the Centre of Excellence for Children and Families over the Life Course. |
| First Year Of Impact | 2013 |
| Description | Centre of Excellence for Children and Families over the Life Course |
| Organisation | University of Queensland |
| Country | Australia |
| Sector | Academic/University |
| PI Contribution | This funding enabled Professor Heather Laurie to visit the University of Melbourne to collaborate on the research application to the Australian Research Council which led to the establishment of the Centre of Excellence for Children and Families over the Life Course. |
| Collaborator Contribution | Professor Heather Laurie was named as an Interntional Partner Investigator of the Centre.. |
| Impact | Successful application to the ARC for the Centre of Excellence for Children and Families over the Life Course |
| Start Year | 2015 |
| Description | Measuring employment in panel surveys: a comparison of reliability estimates in HILDA and BHPS |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | An important use of large household panel surveys is an examination of inequality dynamics in society. An example might be how research on discrimination in employment focuses on sex or race differences in wages, job quality, mobility chances or status outcomes. A well known problem, however, is how random measurement error can lead to attenuation bias in observed substantive coefficients. Moreover, there is mixed evidence on how panel conditioning might affect measures as panels age. One approach to assessing changing data quality in a panel context is to estimate the reliability of variables using quasi-simplex Markov models initially formulated by Heise (1969) and Wiley and Wiley (1970). This approach relies on panel data with at least three time-points to estimate reliabilities from a measurement model incorporating latent true values. Comparing data from the Household, Income and Labour Dynamics in Australia Survey and the British Household Panel Survey, our research addresses the questions of whether and under what conditions the reliability of core employment measures change over time. We further examine whether change in reliability is related to a number of covariates including sex, age and education. We conclude with a discussion of how reliability assessments may affect substantive research using panel data, including cross-country comparisons, and whether calculating and publishing reliabilities may be a desirable feature of a panel data quality profiling exercise. None |
| Year(s) Of Engagement Activity | 2014 |