How should we measure school performance and hold schools accountable? A study of competing statistical methods and how they compare to Progress 8
Lead Research Organisation:
University of Bristol
Department Name: Education
Abstract
In 2016, the Department for Education radically overhauled their secondary school accountability system and introduced 'Progress 8', arguing it to be the simplest and fairest school performance measure to date. Progress 8 aims to quantify and communicate the average academic value each school adds to their pupils' learning. Specifically, Progress 8 measures how much higher each school's pupils score in their age 16 GCSE examinations than expected given their age 11 KS2 test scores when they started secondary schooling. Progress 8 scores are used to hold schools to account, with the lowest scoring schools judged 'underperforming' and 'coasting'; classifications that trigger intense scrutiny and intervention from the school inspectorate, Ofsted. Given the high-stakes involved, research is urgently needed to first evaluate the strengths and weaknesses of the statistical method underlying Progress 8, and second, to explore the potential benefits of alternative methods for measuring school performance. Our proposed research will address these needs and in doing so will advance scientific understanding about school performance measurement.
The first aim of the proposed research is to study how sensitive Progress 8 and other value-added measures' scores, rankings and classifications are to the statistical modelling decisions and assumptions implicit in their design. These include fundamental decisions over whether value-added measures should additionally consider pupil non-academic outcomes and whether they should account for the substantial differences in pupils' demographic and socioeconomic characteristics between schools. We will also explore an assumption implicit in Progress 8 and many other value-added measures which is that there is no relationship between pupils' family backgrounds and the quality of the schools they attend (i.e., confounding or selection into school). We will study the extent to which alternative value-added approaches can address this (using fixed- and random-effects models). We will also explore another assumption of Progress 8 and many other value-added measures which is that the GCSE outcomes of low scoring pupils at KS2 are no more variable than those of high scoring pupils at KS2 (i.e., homoskedasticity). We will study different approaches proposed in the literature to address this concern.
The second aim of the proposed research is to adapt and evaluate a range of cutting edge statistical methods not yet applied to measuring school performance. We will study their advantages and disadvantages over standard value-added approaches including Progress 8. These include methods that allow users to compare not just the average progress made by pupils in each school, but to also study the variability in pupils' progress (i.e., mixed-effects location scale models and multilevel quantile regression). These also include methods that are less reliant on extrapolating the relationships in the data and may therefore lead to fairer and more meaningful school comparisons (i.e., propensity score and other matching methods for comparing multiple treatments).
Whereas the above aims will advance the academic research in the school effectiveness and methods literatures, our third aim is to make fundamental contributions to the wider non-academic understanding of alternative methods for measuring school performance in England. We will therefore disseminate our findings to the users and producers of school and public-sector institution performance measures. These groups include policy makers and their advisers, statisticians and researchers in relevant Government departments and charities, as well as schools, the media and parents. Planned activities include: knowledge exchange meetings, research briefings, a data visualization website, an end-of-grant symposium, online training materials, and a face-to-face short training course.
The first aim of the proposed research is to study how sensitive Progress 8 and other value-added measures' scores, rankings and classifications are to the statistical modelling decisions and assumptions implicit in their design. These include fundamental decisions over whether value-added measures should additionally consider pupil non-academic outcomes and whether they should account for the substantial differences in pupils' demographic and socioeconomic characteristics between schools. We will also explore an assumption implicit in Progress 8 and many other value-added measures which is that there is no relationship between pupils' family backgrounds and the quality of the schools they attend (i.e., confounding or selection into school). We will study the extent to which alternative value-added approaches can address this (using fixed- and random-effects models). We will also explore another assumption of Progress 8 and many other value-added measures which is that the GCSE outcomes of low scoring pupils at KS2 are no more variable than those of high scoring pupils at KS2 (i.e., homoskedasticity). We will study different approaches proposed in the literature to address this concern.
The second aim of the proposed research is to adapt and evaluate a range of cutting edge statistical methods not yet applied to measuring school performance. We will study their advantages and disadvantages over standard value-added approaches including Progress 8. These include methods that allow users to compare not just the average progress made by pupils in each school, but to also study the variability in pupils' progress (i.e., mixed-effects location scale models and multilevel quantile regression). These also include methods that are less reliant on extrapolating the relationships in the data and may therefore lead to fairer and more meaningful school comparisons (i.e., propensity score and other matching methods for comparing multiple treatments).
Whereas the above aims will advance the academic research in the school effectiveness and methods literatures, our third aim is to make fundamental contributions to the wider non-academic understanding of alternative methods for measuring school performance in England. We will therefore disseminate our findings to the users and producers of school and public-sector institution performance measures. These groups include policy makers and their advisers, statisticians and researchers in relevant Government departments and charities, as well as schools, the media and parents. Planned activities include: knowledge exchange meetings, research briefings, a data visualization website, an end-of-grant symposium, online training materials, and a face-to-face short training course.
Planned Impact
WHO WILL BENEFIT FROM THIS RESEARCH?
GROUP A are the USERS OF PROGRESS 8 who need to be able to understand the strengths and weaknesses of the published statistics. This group includes: Department for Education (DfE) policymakers and their advisors who use Progress 8 to classify schools as 'underperforming', 'coasting' or 'high progress'; Ofsted who use these data to target and inform their school inspections; third sector organisations, such as the Education Endowment Foundation, who use Progress 8 to select schools for their randomised control trials and to assess the effectiveness of their interventions; local authorities, academy chains and schools, who use Progress 8 for performance monitoring and resource allocation; the media, who present and explain Progress 8 scores to the public; and parents who are encouraged to use Progress 8 to choose which schools to send their children to.
GROUP B are the PRODUCERS OF SCHOOL AND OTHER PUBLIC-SECTOR INSTITUTION PERFORMANCE MEASURES who need to understand the different statistical methods, choices and issues involved when designing indicators. This group includes: DfE statisticians responsible for Progress 8 and the Fischer Family Trust and other commercial and charitable organisations who produce alternative school performance measures to support target setting and self-evaluation. Internationally, this group includes an increasing number of school systems using pupil test score data to hold schools to account. In terms of other public-sector areas, this group includes: the Higher Education Funding Council for England who are developing university performance measures of 'learning gain', potentially to be included as part of the Teaching Excellence Framework; the Department of Health who publish case-mix adjusted hospital mortality rates; and the Home Office who predict context adjusted police force crime rates.
HOW WILL THEY BENEFIT FROM THIS RESEARCH?
GROUP A will benefit from our substantive research which will carefully explain what Progress 8 attempts to do, how and why it ignores pupil demographic and socioeconomic characteristics and the implications this has for making fair comparisons across schools; and the influence this and other decisions surrounding the design and presentation of Progress 8 have on schools' scores, ranking and classifications. This group will also benefit from our investigations into the risk of over interpreting small differences in schools' performances, and how this will likely prove particularly important when comparing within school socioeconomic and other Progress 8 performance gaps. The novel statistical methods we shall explore will generate new substantive insights into the effectiveness of schools in England and this will further benefit this group and feed directly into national debates and policy surrounding school improvement and equity as well as school accountability.
GROUP B will benefit from our methodological research which explores the potential statistical strengths and weakness of not only different value-added approaches, but also of a range of cutting edge methods not yet applied to measuring school performance. These include mixed-effects location scale models, multilevel quantile regression and matching methods. This will enable this group to make more informed choices when developing their own measures. This group will also benefit from the training materials we shall develop as well as any versions of these adapted by others to their disciplines. The materials will provide step-by-step instructions on how to implement the different statistical methods in standard software. They will also act as exemplars as to how to richly interpret the estimated institution effects generated by the different methods as well as good practices in communicating institution performances and their uncertainty to lay audiences.
WHAT WILL BE DONE TO ENSURE THAT THEY BENEFIT FROM THE RESEARCH?
See PATHWAYS TO IMPACT.
GROUP A are the USERS OF PROGRESS 8 who need to be able to understand the strengths and weaknesses of the published statistics. This group includes: Department for Education (DfE) policymakers and their advisors who use Progress 8 to classify schools as 'underperforming', 'coasting' or 'high progress'; Ofsted who use these data to target and inform their school inspections; third sector organisations, such as the Education Endowment Foundation, who use Progress 8 to select schools for their randomised control trials and to assess the effectiveness of their interventions; local authorities, academy chains and schools, who use Progress 8 for performance monitoring and resource allocation; the media, who present and explain Progress 8 scores to the public; and parents who are encouraged to use Progress 8 to choose which schools to send their children to.
GROUP B are the PRODUCERS OF SCHOOL AND OTHER PUBLIC-SECTOR INSTITUTION PERFORMANCE MEASURES who need to understand the different statistical methods, choices and issues involved when designing indicators. This group includes: DfE statisticians responsible for Progress 8 and the Fischer Family Trust and other commercial and charitable organisations who produce alternative school performance measures to support target setting and self-evaluation. Internationally, this group includes an increasing number of school systems using pupil test score data to hold schools to account. In terms of other public-sector areas, this group includes: the Higher Education Funding Council for England who are developing university performance measures of 'learning gain', potentially to be included as part of the Teaching Excellence Framework; the Department of Health who publish case-mix adjusted hospital mortality rates; and the Home Office who predict context adjusted police force crime rates.
HOW WILL THEY BENEFIT FROM THIS RESEARCH?
GROUP A will benefit from our substantive research which will carefully explain what Progress 8 attempts to do, how and why it ignores pupil demographic and socioeconomic characteristics and the implications this has for making fair comparisons across schools; and the influence this and other decisions surrounding the design and presentation of Progress 8 have on schools' scores, ranking and classifications. This group will also benefit from our investigations into the risk of over interpreting small differences in schools' performances, and how this will likely prove particularly important when comparing within school socioeconomic and other Progress 8 performance gaps. The novel statistical methods we shall explore will generate new substantive insights into the effectiveness of schools in England and this will further benefit this group and feed directly into national debates and policy surrounding school improvement and equity as well as school accountability.
GROUP B will benefit from our methodological research which explores the potential statistical strengths and weakness of not only different value-added approaches, but also of a range of cutting edge methods not yet applied to measuring school performance. These include mixed-effects location scale models, multilevel quantile regression and matching methods. This will enable this group to make more informed choices when developing their own measures. This group will also benefit from the training materials we shall develop as well as any versions of these adapted by others to their disciplines. The materials will provide step-by-step instructions on how to implement the different statistical methods in standard software. They will also act as exemplars as to how to richly interpret the estimated institution effects generated by the different methods as well as good practices in communicating institution performances and their uncertainty to lay audiences.
WHAT WILL BE DONE TO ENSURE THAT THEY BENEFIT FROM THE RESEARCH?
See PATHWAYS TO IMPACT.
Organisations
Publications
Austin P
(2020)
Bootstrapped inference for variance parameters, measures of heterogeneity and random effects in multilevel logistic regression models
in Journal of Statistical Computation and Simulation
French R
(2022)
Estimating the Importance of Families in Modeling Educational Achievement Using Linked Swedish Administrative Data
in Journal of Research on Educational Effectiveness
French, R.
(2022)
Estimating the importance of families in modeling educational achievement using linked Swedish administrative data
in Journal of Research on Educational Effectiveness
Gallo Cordoba B
(2022)
Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups: a mediation approach
in Sociological Methodology
Gallo Cordoba B
(2022)
Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups
in Sociological Methodology
Goldstein H
(2020)
Estimating reliability statistics and measurement error variances using instrumental variables with longitudinal data
in Longitudinal and Life Course Studies
Goldstein Harvey
(2020)
Providing educational accountability for Local Authorities based upon sampling pupils within schools: moving away from simplistic school league tables
in arXiv e-prints
Heck R
(2022)
Incorporating student mobility in studying academic growth in math: comparing several alternative multilevel formulations
in School Effectiveness and School Improvement
| Description | This grant funded three years of intensive research on England's Government school performance tables. The grant focussed on statistically critiquing and comparing various school performance measures published in these tables, especially value-added and progress measures. We particularly focussed on the influence of choice of statistical modelling and estimation approach on the resulting performance measures. Below we list the five most important research outputs we have produced with accessible summaries. (1) Leckie, G. and Goldstein, H. (2019). The importance of adjusting for pupil background in school value-added models: A study of Progress 8 and school accountability in England. British Educational Research Journal. DOI: 10.1002/berj.3511. Abstract: In the UK, USA and elsewhere, school accountability systems increasingly compare schools using value-added measures of school performance derived from pupil scores in high-stakes standardised tests. Rather than naïvely comparing school average scores, which largely reflect school intake differences in prior attainment, these measures attempt to compare the average progress or improvement pupils make during a year or phase of schooling. Schools, however, also differ in terms of their pupil demographic and socioeconomic characteristics and these factors also predict why some schools subsequently score higher than others. Many therefore argue that value-added measures unadjusted for pupil background are biased in favour of schools with more 'educationally advantaged' intakes. But others worry that adjusting for pupil background entrenches socioeconomic inequities and excuses low-performing schools. In this article we explore these theoretical arguments and their practical importance in the context of the 'Progress 8' secondary school accountability system in England, which has chosen to ignore pupil background. We reveal how the reported low or high performance of many schools changes dramatically once adjustments are made for pupil background, and these changes also affect the reported differential performances of regions and of different school types. We conclude that accountability systems which choose to ignore pupil background are likely to reward and punish the wrong schools and this will likely have detrimental effects on pupil learning. These findings, especially when coupled with more general concerns surrounding high-stakes testing and school value-added models, raise serious doubts about their use in school accountability systems. (2) Leckie, G., Browne, W., Goldstein, H., Merlo, J., and Austin, P. (2020). Variance partitioning in multilevel models for count data. Psychological Methods. DOI: 10.1037/met0000265. Abstract: A first step when fitting multilevel models to continuous responses is to explore the degree of clustering in the data. Researchers fit variance-component models and then report the proportion of variation in the response that is due to systematic differences between clusters. Equally they report the response correlation between units within a cluster. These statistics are popularly referred to as variance partition coefficients (VPCs) and intraclass correlation coefficients (ICCs). When fitting multilevel models to categorical (binary, ordinal, or nominal) and count responses, these statistics prove more challenging to calculate. For categorical response models, researchers appeal to their latent response formulations and report VPCs/ICCs in terms of latent continuous responses envisaged to underly the observed categorical responses. For standard count response models, however, there are no corresponding latent response formulations. More generally, there is a paucity of guidance on how to partition the variation. As a result, applied researchers are likely to avoid or inadequately report and discuss the substantive importance of clustering and cluster effects in their studies. A recent article drew attention to a little-known exact algebraic expression for the VPC/ICC for the special case of the two-level random-intercept Poisson model. In this article, we make a substantial new contribution. First, we derive exact VPC/ICC expressions for more flexible negative binomial models that allows for overdispersion, a phenomenon which often occurs in practice. Then we derive exact VPC/ICC expressions for three-level and random-coefficient extensions to these models. We illustrate our work with an application to student absenteeism. (3) Prior, L., Goldstein, H., and Leckie, G. (2021). School value-added models for multivariate academic and non-academic outcomes: exploring implications for performance monitoring and accountability. School Effectiveness and School Improvement. DOI: 10.1080/09243453.2021.1919719. Abstract: Education systems around the world increasingly rely on school value-added models to monitor school performance and hold schools to account. These models typically focus on a limited number of academic outcomes. We explore how the traditional multilevel modelling approach to school value-added models can be extended to simultaneously analyse multiple academic and non-academic outcomes and the implications of doing so for systems using student data to monitor schools and inform school accountability. We jointly model student attainment, absence and exclusion data for schools in England. We find very different results across the three outcomes, in terms of the size and consistency of school effects, and the importance of adjusting for student and school characteristics. The results suggest the three outcomes are capturing fundamentally distinct aspects of school performance, all of which are therefore important for education systems to monitor and explore including in systems of school accountability. (4) Prior, L., Jerrim, J., Thomson, D., and Leckie, G. (2021). A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses and challenges for 'Progress 8' raised by COVID-19. Review of Education. DOI: 10.1002/rev3.3299. Abstract: School performance measures are published annually in England to hold schools to account and to support parental school choice. This article reviews and evaluates the 'Progress 8' secondary school accountability system for state-funded schools. We assess the statistical strengths and weaknesses of Progress 8 relating to: choice of pupil outcome attainment measure; potential adjustments for pupil input attainment and background characteristics; decisions around which schools and pupils are excluded from the measure; presentation of Progress 8 to users, choice of statistical model, and calculation of statistical uncertainty; and issues related to the volatility of school performance over time, including scope for reporting multi-year averages. We then discuss challenges for Progress 8 raised by the COVID-19 pandemic. Six simple recommendations follow to improve Progress 8 and school accountability in England. (5) Leckie, G. and Prior, L. (2022). A comparison of value-added models for school accountability. School Effectiveness and School Improvement. DOI: 10.1080/09243453.2022.2032763. Abstract: School accountability systems increasingly hold schools to account for their performances using value-added models purporting to measure the effects of schools on student learning. The most common approach is to fit a linear regression of student current achievement on student prior achievement, where the school effects are the school means of the predicted residuals. In the literature, further adjustments are usually made for student sociodemographics and sometimes school composition and "non-malleable" characteristics. However, accountability systems typically make fewer adjustments: for transparency to end users, because data are unavailable or of insufficient quality, or for ideological reasons. There is therefore considerable interest in understanding the extent to which simpler models give similar school effects to more theoretically justified but complex models. We explore these issues via a case study and empirical analysis of England's "Progress 8" secondary school accountability system. |
| Exploitation Route | Our research outcomes are relevant both to academics doing future research on school performance tables and accountability measures and governments and other non-academic bodies who actually implement these tables and measures. |
| Sectors | Education |
| URL | http://www.bristol.ac.uk/cmm/research/school-performance/ |
| Description | Policy context 'Progress 8' is the Department for Education's (DfE) headline school performance and accountability measure for all state secondary schools in England and is published annually in their school league tables (see https://www.gov.uk/school-performance-tables). Progress 8 aims to measure the average academic 'progress' pupils make over secondary schooling by comparing their end of secondary schooling GCSE exam results to their end of primary schooling KS2 test results. The DfE and Ofsted both rely heavily on Progress 8 to hold schools to account and so schools' futures are very much dictated by this measure. The disruption of the COVID-19 pandemic has led Progress 8 to not be published for 2019/20 and 2020/21, however Progress 8 is due to be published again for 2021/22. More generally, our work analysing this measure remains relevant for future school performance measures that will likely be developed. Our research Much of our research has focussed on Progress 8 and has highlighted the statistical strengths and weaknesses of this measure. Our review paper (Prior et al., 2021) argued that strengths included accounting for school differences in prior attainment, encouraging focus on students across the ability spectrum (thus moving away from previous borderline effects at the C/D grade boundary) and presenting statistical uncertainty. However, we also noted several statistical weaknesses of Progress 8, leading to the following six recommendations: (1) reduce the focus on the academic EBacc subjects (English, maths, sciences, humanities, languages) to present a more holistic picture of school performance relevant to more schools and pupils; (2) present a pupil background adjusted Progress 8 measure to give a picture of performance informed by school context; (3) recognise pupil mobility by including all pupils who attended each school in Progress 8, weighting by the time pupils spent in each school; (4) enhance communication of the relative importance of school Progress 8 scores to explaining overall variation in pupil progress and providing clarification on the magnitude of each school's individual Progress 8 score; (5) increase warnings regarding the uncertainty in using Progress 8 to predict the future performance of schools to improve its use for school choice purposes; (6) report multi-year averages alongside single-year summaries to combat the instability in performance over time. More broadly, however, we argue far less emphasis be placed on all school performance measures as a failure to adjust for pupil background is just one of many statistical and more general concerns that we and others have raised with current high-stakes testing and league tables in England (Leckie and Goldstein, 2017). In terms of point (2) above, our additional research in this area (Leckie and Goldstein, 2019) shows that Progress 8 is an unfair measure of school performance as while it accounts for school intake differences in pupils' KS2 results, it still ignores school intake differences in all other pupil background characteristics, yet these also predict why some schools score higher at GCSE than others. Our statistical analyses of the DfE's own data shows that adjusting for pupil background would see the national Progress 8 school league table rankings of one-fifth of schools change by over 500 places. Furthermore, 40% of schools judged 'underperforming' would move up out of this banding. The main driver of these results is that Progress 8 penalises schools teaching above average proportions of free school meal and white British pupils, two pupil groups which struggle nationally. We conclude that Progress 8 is biased against schools with socially disadvantaged pupil groups and therefore punishes and rewards the wrong schools. We recommend the DfE adjust Progress 8 for pupil backgrounds to provide fairer and more meaningful summaries of the impact that schools have on pupil learning. Our impact Our research has had impact from the start, via two-way knowledge exchange meetings with multiple stakeholders. First, we presented to local schools in Bristol. Then we shared our initial findings with Ofsted at their Bristol offices who prompted us to conduct useful further analyses. We then shared our draft research with the Fischer Family Trust, an organisation which offers school performance monitoring tools to schools, and used their feedback to further improve our work. A second meeting with Ofsted helped further refine our analyses. Next, we discussed the relevance of our findings for OfS and their related work on measuring University 'learning gains'. We have had a rich and varied discussion with DfE policy advisors in their Westminster offices around the likely future direction of school performance measures in England. The University of Bristol press release of our work led to national radio (Talk Sport and Heart FM), newspaper and online coverage (The Guardian, The Independent, The Times, and Times Educational Supplement) leading to increased public awareness and understanding or our work. Following this media attention, we gave an invited talk to teachers at the National Education Union and have presented our work at an invited private meeting with the Education Select Committee. We have since had a follow-on face-to-face meeting with Emma Hardy, Labour MP (now Shadow FE and HE Minister). This meeting included discussing alternatives to Progress 8 and data-driven school accountability given Labour's May 2019 announcement to scrap KS2 tests. Our work has also prompted the Co-Op Academies Trust to issue their own press release citing our work and calling for Progress 8 to be adjusted and encouraging other schools to contact us. This lead to more media attention this time in TES and Schools Week. Following this, we have corresponded with many headteachers who have emailed to find out their pupil background adjusted Progress 8 scores and to ask to be involved in future research. Frank Norris, Director of Co-Op Academies Trust, has also discussed our work in a meeting with Labour MP Angela Rayner (Shadow Secretary of State for Education) and 25 Labour MPs. We presented to all the Co-Op headteachers in Manchester and Frank Norris visited us in Bristol. We have also collaborated extensively with The Northern Powerhouse Partnership (NPP) who have championed our work at a second Education Select Committee on 'Education in the North' and in a meeting with shadow cabinet members on education and skills. Partnering with the NPP, we have then published Adjusted Progress 8 scores for all schools in England and on a dedicated interactive website (see http://www.northernpowerhousepartnership.co.uk/publications/latest-research-shows-league-tables-punish-and-reward-wrong-schools/) as a way of encouraging further debate around concerns with Progress 8 and data driven accountability and to place more pressure on the DfE to engage on these issues. Lucy Powell, Labour MP, Education Select Committee member and former Shadow Secretary of State for Education said in her foreword to this work "This is a ground-breaking piece of work by the Norther Powerhouse Partnership with Bristol University". The press release of this joint work led to further media coverage by the BBC, Independent, FT, and Times Education Supplement. We have also been active in responding to national consultations on education, this included submitting a statement to the Ofqual consultation on 'GCSE and A level grading proposals for 2020', and the Education Policy Institute's consultation on 'The features of effective school groups: measuring pupil inclusion at school-group level'. Additionally, through continued knowledge exchange meetings, we have an open and constructive dialogue with the Fischer Family Trust Education Datalab to discuss our work and their related research work. Following publication and dissemination of our work reviewing Progress 8, we were invited to a roundtable discussion focused on the question of what policymakers need to know in order to support educational equity. This was hosted by Loic Menzies and the Cambridge Assessment Network and brought together leading figures for policymaking, Multi-Academy Trusts, and other education and social commissions. Attendees included: Hardip Begol OBE, Dame Rachel De Souza (Children's Commissioner for England), Dr Rebecca Conway, Tim Oates CBE, David Simister-Thomas OBE, Professor John Jerrim, and Sammy Wright (Social Mobility Commission lead for schools and HE). References Leckie, G., & Goldstein, H. (2017). The evolution of school league tables in England 1992-2016: 'contextual value-added', 'expected progress' and 'progress 8'. British Educational Research Journal, 43(2), 193-212. URL: https://onlinelibrary.wiley.com/doi/full/10.1002/berj.3264 Leckie, G. and Goldstein, H. (2019). The importance of adjusting for pupil background in school value-added models: A study of Progress 8 and school accountability in England. British Educational Research Journal, 45(3), 518-537. URL: https://onlinelibrary.wiley.com/doi/10.1002/berj.3511 Prior, L., Jerrim, J., Thomson, D., and Leckie, G. (2021). A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses and challenges for 'Progress 8' raised by COVID-19. Review of Education, 9(3), e3299. URL: https://bera-journals.onlinelibrary.wiley.com/doi/10.1002/rev3.3299 |
| First Year Of Impact | 2018 |
| Sector | Education |
| Impact Types | Policy & public services |
| Description | Discussions with Bulgarian minister of education and heads of directorates as expert member of World Bank team |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | G. Leckie discussion with Research Director EsadeEcPol Center for Economic Policy in Spain about school value-added analyses. January 2023. |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Israel Academy of Sciences: Discussion about school value-added models, 2019 |
| Geographic Reach | Asia |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Knowledge exchange meeting - Fischer Family Trust. Discussed Progress 8 review paper. December 2020. |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Knowledge exchange meeting - Fischer Family Trust. Discussed their related research work. January 2021. |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Knowledge exchange meeting. August 2019. Fischer Family Trust. Discussed their related work. |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Knowledge exchange meeting. December 2018. Fischer Family Trust. Discussed adjusted Progress 8 work. |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Knowledge exchange meeting. July 2020. Discussion with Director and Chief Economist of Northern Powerhouse Partnership |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Knowledge exchange meeting. July 2021. Fischer Family Trust: Education Datalab. Discussed their related research work. |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Knowledge exchange meeting. May 2019. Discussion with Director and Chief Economist of Northern Powerhouse Partnership |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Meeting with Office for Statistics Regulation (OSR) DfE to discuss "Assessment of Statistics on Key Stage 4 Performance produced by the Department for Education", February 2022 |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Office for Standards in Education, Children's Services and Skills (Ofsted) knowledge exchange meeting about potential Ofsted projects and Progress 8 research |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Office for Standards in Education, Children's Services and Skills (Ofsted) knowledge exchange meeting about potential Ofsted projects and Progress 8 research |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Office for Standards in Education, Children's Services and Skills (Ofsted): knowledge exchange meeting for Ofsted about general research collaboration with University of Bristol and potential PhD studentship |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Office for Student (OfS) Benchmarking review advisory group |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Office for Students (OfS), Benchmarking review advisory group |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Presentation of Progress 8 research to Emma Hardy MP (Education Select Committee Member) at Portcullis House |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Presentation of our Progress 8 research at UK Parliament Education Select Committee |
| Geographic Reach | National |
| Policy Influence Type | Contribution to a national consultation/review |
| Description | Round table discussion on educational equity hosted by Loic Menzies and Cambridge Assessment Network. January 2022. Cambridge. |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Submitted statement to Education Policy Institute (EPI) consultation on 'The Features of Effective School Groups: Measuring pupil inclusion at school-group level'. February 2022. |
| Geographic Reach | National |
| Policy Influence Type | Contribution to a national consultation/review |
| Description | Submitted statement to Ofqual consultation "GCSE and A level grading proposals for 2020" |
| Geographic Reach | National |
| Policy Influence Type | Contribution to a national consultation/review |
| Description | Submitted written evidence to Education Select Committee on "Left behind white pupils from disadvantaged backgrounds" |
| Geographic Reach | National |
| Policy Influence Type | Contribution to a national consultation/review |
| Description | Sweden Department for Education: Advising on school value-added models |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | Trinidad and Tobago Department for Education: Advising on school value-added models |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | World Bank Expert advisor on school value-added modelling project |
| Geographic Reach | National |
| Policy Influence Type | Participation in a guidance/advisory committee |
| Description | This R script provides a function to perform a multiple-level multi-group decomposition for the analysis of the ethnic achievement gap, as described in the article 'Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups' published in the journal Sociological Methodology. Gallo Cordoba B, Leckie G, Browne W.J. Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups. Sociological Methodology. pp. 1-31. Copyright [June 2022] (American Sociological Association). doi: 10.1177/00811750221099503 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| URL | https://bridges.monash.edu/articles/software/R_Function_for_multiple-level_multi-group_decomposition... |
| Description | This R script provides a function to perform a multiple-level multi-group decomposition for the analysis of the ethnic achievement gap, as described in the article 'Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups' published in the journal Sociological Methodology. Gallo Cordoba B, Leckie G, Browne W.J. Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups. Sociological Methodology. pp. 1-31. Copyright [June 2022] (American Sociological Association). doi: 10.1177/00811750221099503 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| URL | https://bridges.monash.edu/articles/software/R_Function_for_multiple-level_multi-group_decomposition... |
| Description | This Stata do file performs a multiple-level two-group decomposition using a mediation analysis approach, as described in the article 'Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups' published in the journal Sociological Methodology. Gallo Cordoba B, Leckie G, Browne W.J. Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups. Sociological Methodology. pp. 1-31. Copyright [June 2022] (American Sociological Association). doi: 10.1177/00811750221099503 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| URL | https://bridges.monash.edu/articles/software/Stata_do_file_for_multiple-level_two-group_decompositio... |
| Description | This Stata do file performs a multiple-level two-group decomposition using a mediation analysis approach, as described in the article 'Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups' published in the journal Sociological Methodology. Gallo Cordoba B, Leckie G, Browne W.J. Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups. Sociological Methodology. pp. 1-31. Copyright [June 2022] (American Sociological Association). doi: 10.1177/00811750221099503 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| URL | https://bridges.monash.edu/articles/software/Stata_do_file_for_multiple-level_two-group_decompositio... |
| Description | This Stata do file performs a two-level multiple-group decomposition using a mediation analysis approach, as described in the article 'Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups' published in the journal Sociological Methodology. Gallo Cordoba B, Leckie G, Browne W.J. Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups. Sociological Methodology. pp. 1-31. Copyright [June 2022] (American Sociological Association). doi: 10.1177/00811750221099503 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| URL | https://bridges.monash.edu/articles/software/Stata_do_file_for_two-level_multiple-group_decompositio... |
| Description | This Stata do file performs a two-level multiple-group decomposition using a mediation analysis approach, as described in the article 'Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups' published in the journal Sociological Methodology. Gallo Cordoba B, Leckie G, Browne W.J. Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups. Sociological Methodology. pp. 1-31. Copyright [June 2022] (American Sociological Association). doi: 10.1177/00811750221099503 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| URL | https://bridges.monash.edu/articles/software/Stata_do_file_for_two-level_multiple-group_decompositio... |
| Description | This Stata do file performs the conventional two-level two-group decomposition using a mediation analysis approach, as described in the article 'Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups' published in the journal Sociological Methodology. Gallo Cordoba B, Leckie G, Browne W.J. Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups. Sociological Methodology. pp. 1-31. Copyright [June 2022] (American Sociological Association). doi: 10.1177/00811750221099503 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| URL | https://bridges.monash.edu/articles/software/Stata_do_file_for_two-level_two-group_decomposition/196... |
| Description | This Stata do file performs the conventional two-level two-group decomposition using a mediation analysis approach, as described in the article 'Decomposing ethnic achievement gaps across multiple levels of analysis and for multiple ethnic groups' published in the journal Sociological Methodology. Gallo Cordoba B, Leckie G, Browne W.J. Decomposing Ethnic Achievement Gaps across Multiple Levels of Analysis and for Multiple Ethnic Groups. Sociological Methodology. pp. 1-31. Copyright [June 2022] (American Sociological Association). doi: 10.1177/00811750221099503 |
| Type Of Technology | Software |
| Year Produced | 2022 |
| Open Source License? | Yes |
| URL | https://bridges.monash.edu/articles/software/Stata_do_file_for_two-level_two-group_decomposition/196... |
| Description | 'Incorporating student mobility in studies of science achievement: Testing a cross-classified multiple membership model'. Reid, T., Leckie, G., & Heck, R. 2022 AERA Annual Meeting, San Diego |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | April 2022. Conference presentation |
| Year(s) Of Engagement Activity | 2022 |
| Description | 'Is it still worth calculating Progress 8?' Prior, L., Jerrim, J., Thomson, D., and Leckie, G. 2021. BERA blog. |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Blog post for BERA blog. |
| Year(s) Of Engagement Activity | 2021 |
| URL | https://www.bera.ac.uk/blog/is-it-still-worth-calculating-progress-8 |
| Description | 'Progress 8' National Education Union annual conference, Liverpool |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Schools |
| Results and Impact | April 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | 'Pupil mobility: the extent, impacts and predictors of moving school'. Society for Longitudinal and Lifecourse Studies (SLLS). Online. September 2021 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | September 2021. Conference presentation |
| Year(s) Of Engagement Activity | 2021 |
| Description | 'School accountability in crisis - holding the judges to accounts', British Educational Research Association (BERA) 2019 Annual Conference Joint coordinator with Terry Wrigley and Tom Perry |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | September 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | 'School value-added models for multivariate academic and non-academic outcomes: Exploring implications for performance monitoring and accountability'. British Educational Research Association (BERA). Online. September 2021 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | September 2021. Conference presentation. |
| Year(s) Of Engagement Activity | 2021 |
| Description | 'School value-added models of academic and non-academic student outcomes: A more rounded approach to using student data to inform school accountability with application to attainment, absences and exclusions in England. NPD User Group Meeting, London. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | January 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | 'Secondary school performance measures and league tables in England: An overview of the official measures and their statistical modelling, data, and interpretation issues'. York |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | February 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | 'Secondary school performance measures and league tables in England: An overview of the official measures and their statistical modelling, data, policy, and interpretation issues', Statistics and Probability Group, University of York |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | February 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | 'Should the Government modify Progress 8 to account for school intake differences in pupil background?' Afternoon event on 'Contextual Data: The benefits and the pitfalls for schools and society', Co-Op and Northern Powerhouse Partnership, Manchester |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | March 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | 'Should the Government modify Progress 8 to account for school intake differences in pupil background?' One day conference on 'Redefining progress for disadvantaged students'. Education Observatory, University of Wolverhampton |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | January 2020 |
| Year(s) Of Engagement Activity | 2020 |
| Description | A talk of presentaton: 'Accountability and school differential effects' Department of Learning and Leadership, IOE UCL, London. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Schools |
| Results and Impact | November 2018 |
| Year(s) Of Engagement Activity | 2018 |
| URL | https://mediacentral.ucl.ac.uk/Play/15633 |
| Description | A talk or presentation: 2018 AERA Annual Meeting, New York |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Apri 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | A talk or presentation: Bristol Cathedral School, Bristol |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Schools |
| Results and Impact | September 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | A talk or presentation: Educational Effectiveness EARLI SIG 18 & 23 meeting. Groningen: 25 years of school league tables, accountability and choice: Lessons from England |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | August 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | A talk or presentation: Q-step Seminar Series, University of Exeter |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Postgraduate students |
| Results and Impact | May 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | A talk or presentation: Royal Statistical Society Annual Conference, Cardiff |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | September 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | BERA Podcast recorded seminar. 'The importance of adjusting for pupil sociodemographics in Progress 8 and school accountability in England' |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Recorded seminar for BERA podcast on adjusting for pupil background characteristics in Progress 8 school performance measure. |
| Year(s) Of Engagement Activity | 2020 |
| Description | Blog post: Leckie G, Prior L, Goldstein H (2019) School accountability by Progress 8: A critique of Conservative and Labour proposals |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | Blog post written for BERA blog in run-up to 2019 General Election, focusing on potential implications for school accountability of Conservative and Labour standpoints. Intended to spark discussion. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.bera.ac.uk/blog/school-accountability-by-progress-8-a-critique-of-conservative-and-labou... |
| Description | Blog post: Leckie G, Prior L, Goldstein H (2020). School accountability by Progress 8: A critique of Conservative and Labour proposals. School of Education Blog, University of Bristol. |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | Blog post written for the School of Education focused on school accountability in reference to proposals and standpoints of the Conservative and Labour parties. Intended to spark discussion. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://schoolofeducation.blogs.bristol.ac.uk/2020/01/23/school-accountability-by-progress-8-a-criti... |
| Description | Calculating intraclass correlation coefficients in multilevel models for count responses. 12th International Amsterdam Multilevel Conference, Utrecht, April 2019 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | A presentation to an international audience of academics and postgraduate researchers interested in use of multilevel models. |
| Year(s) Of Engagement Activity | 2019 |
| Description | Catherine Peckham Symposium/Otto Wolff Lecture 2019, London |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | October 2019. |
| Year(s) Of Engagement Activity | 2019 |
| Description | DfE funded NPD User Group meeting (1 day), Bristol. Co-organiser with Rebecca Allen |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Third sector organisations |
| Results and Impact | September 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | Discussion with Director and Chief Economist of Northern Powerhouse Partnership |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Policymakers/politicians |
| Results and Impact | 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | Discussion with Education Campaigns Manager at Humanists UK about Progress 8 |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Third sector organisations |
| Results and Impact | August 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | Discussion with Head of Co-Op Academies about Progress 8 |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Schools |
| Results and Impact | April 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | Discussion with Head of Co-Op Academies about Progress 8 |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Schools |
| Results and Impact | February 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | Discussion with Headmaster of school in North West about Progress 8 |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | Local |
| Primary Audience | Schools |
| Results and Impact | October 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | EARLI SIG 18, Freiburg, 'School value-added models for multivariate academic and non-academic outcomes in school accountability systems', presentation, September 2022 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Presentation at EARLI SIG 18 conference (Educational Effectiveness and Improvement) |
| Year(s) Of Engagement Activity | 2022 |
| Description | EARLI SIG 18, Freiburg, 'Student mobility: extent, impacts and predictors of a range of movement types', presentation, September 2022 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | Presentation at EARLI SIG 18 conference (Educational Effectiveness and Improvement) |
| Year(s) Of Engagement Activity | 2022 |
| Description | G. Leckie discussion with Warwick Mansell around challenges of applying to use DfE NPD, access via ONS SRS, and delays to DfE NPD data release, May 2022. |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | Discussion with Warwick Mansell, educational journalist, around challenges of applying to and accessing DfE NPD data. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Impact case study: Secondary school accountability in England (2022). ADR UK. |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Worked with the Office for National Statistics SRS/IDS Impact team to create a ADR UK impact case study focused on our work around secondary school accountability in England. |
| Year(s) Of Engagement Activity | 2022 |
| URL | https://www.adruk.org/our-mission/our-impact/secondary-school-accountability-in-england/ |
| Description | International Multilevel Modelling conference. Key note presentation. 'Multilevel modelling of school effects on student achievement and their application in school accountability systems', Utrecht. April 2022. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Keynote conference presentation |
| Year(s) Of Engagement Activity | 2022 |
| Description | Is it worth calculating Progress 8 anymore? Policy Briefing, PolicyBristol 109:11/2021 |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Policymakers/politicians |
| Results and Impact | Policy Briefing report summarising a review of Progress 8 with policy recommendations. Shared with key policy and practitioners and available publicly. |
| Year(s) Of Engagement Activity | 2021 |
| URL | http://www.bristol.ac.uk/policybristol/policy-briefings/calculating-progess-8/ |
| Description | KU Leuven, Guest Lecture, 'A tour of Government produced value-added school effects in England', Educational Effectiveness, May 2022 |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Guest lecture at KU Leuven on Educational Effectiveness |
| Year(s) Of Engagement Activity | 2022 |
| Description | KU Leuven, Research Presentation. 'Avoiding bias when estimating the consistency and stability of value-added schools effects', May 2022. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Research presentation at KU Leuven. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Leckie, G., Prior, L., Goldstein, H. (2019). School accountability by Progrress 8: A critique of Conservative and Labour proposals. BERA Election Blog post. 29/11/2019 |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Blog post considering the potential implications of policy proposals by the Conservatives and Labour on the Progress 8 school performance measure. |
| Year(s) Of Engagement Activity | 2019 |
| Description | Leckie, G., and Goldstein, H. (2020). Progress 8 school league tables punish and reward the wrong schools. BERA's Research Intelligence magazine, 143. |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | Article for magazine on Progress 8 and the implications of adjustment for student background characteristics. |
| Year(s) Of Engagement Activity | 2020 |
| Description | National Pupil Database (NPD) User Group meeting. Co-organiser (with Dave Thomson). January 2020. London. |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | January 2020. Conference organisation for NPD user group. |
| Year(s) Of Engagement Activity | 2020 |
| Description | Newspaper coverage: Bristol Post, Daily Express, The Guardian, The Independent, Leigh Journal, The Star, The Times, Times Educational Supplement, Warrington Guardian, Wigan Today, Yorkshire Post |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | January 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | November 2020: Frank Norris: "Why the Co-op is involved in the academy programme". Bristol Conversations in Education. School of Education, University of Bristol. Organiser G. Leckie. |
| Form Of Engagement Activity | A talk or presentation |
| Part Of Official Scheme? | No |
| Geographic Reach | Regional |
| Primary Audience | Postgraduate students |
| Results and Impact | Organised talk by Frank Norris on Co-op's involvement in the academies programme for the Bristol Conversations in Education series. |
| Year(s) Of Engagement Activity | 2020 |
| Description | Other media coverage: BBC website, Chronical Live, FE News |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | Other media coverage: Ekklesia, Greensheets, Humanists UK, Local Gov, My Science, Reclaiming Schools, Schools Improvement, Schools Week, World News Live 4 |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | January 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | Policy Briefing: Leckie, G., Goldstein, H. (2019). The Government's Progress 8 school performance measure needs to account for pupil background. PolicyBristol. Policy Briefing 66: 01/2019. |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | January 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | Postgraduate short course: Introduction to Multilevel Modelling (1 day). Bamberg. Course Instructor |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | October 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | Postgraduate short course: Multilevel Modelling in School Effectiveness Research (0.5 days). EARLI Special Interest Group 18 Educational Effectiveness, Groningen. Course Instructor |
| Form Of Engagement Activity | Participation in an activity, workshop or similar |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Postgraduate students |
| Results and Impact | August 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | Prof. G. Leckie to host academic visit from Koen Aesaert, Associate Professor, Centre for Educational Effectiveness and Evaluation, KU Leuven (two weeks). Working together on school effectiveness and school value-added models. October 2023 |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Prof. G Leckie to host visiting academic Koen Aesaert, Associate Professor, from the Centre of Educational Effectiveness and Evaluation, KU Leuven and to work collaboratively on school effectiveness and school value-added models research. |
| Year(s) Of Engagement Activity | 2023 |
| Description | Prof. G. Leckie visit to Koen Aesaert, Associate Professor, Centre for Educational Effectiveness and Evaluation, KU Leuven (two days). Working together on school effectiveness and school value-added models. May 2022. |
| Form Of Engagement Activity | Participation in an open day or visit at my research institution |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | Prof. George Leckie underwent an academic visit to Associate Professor Koen Aesaert, Centre of Educational Effectiveness and Evaluation, KU Leuven for two days, and worked collaboratively on research on school effectiveness and school value-added models. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Professor George Leckie, SIG members in the spotlight. EARLI SIG 18 Educational Effectiveness and Improvement Newsletter. December 2022. |
| Form Of Engagement Activity | A magazine, newsletter or online publication |
| Part Of Official Scheme? | No |
| Geographic Reach | International |
| Primary Audience | Professional Practitioners |
| Results and Impact | EARLI SIG 18 Newsletter focused on Professor George Leckie and his work in the field of educational effectiveness and improvement. |
| Year(s) Of Engagement Activity | 2022 |
| Description | Radio coverage: Heart radio (pre-recorded interview); Love sport radio (live interview); BBC World at One (invited to live interview, declined as I was abroad). |
| Form Of Engagement Activity | A press release, press conference or response to a media enquiry/interview |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Media (as a channel to the public) |
| Results and Impact | January 2019 |
| Year(s) Of Engagement Activity | 2019 |
| Description | Roundtable discussion on Progress 8 and school accountability measures with 15 others Held online. |
| Form Of Engagement Activity | A formal working group, expert panel or dialogue |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Professional Practitioners |
| Results and Impact | A roundtable discussion about Progress 8 and school accountability measures with academics, and others from the Fischer Family Trust, Education Policy Institute, Centre for Education and Youth, and the Behavioural Insights Team, as well as representatives from the DfE. |
| Year(s) Of Engagement Activity | 2020 |
| Description | Video recorded seminar: 'Accountability and school differential effects', Department of Learning and Leadership, IOE UCL, London. Speaker |
| Form Of Engagement Activity | A broadcast e.g. TV/radio/film/podcast (other than news/press) |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Postgraduate students |
| Results and Impact | 2018 |
| Year(s) Of Engagement Activity | 2018 |
| Description | Website: Fairer Schools Index 2018. Northern Powerhouse Partnership collaboration. |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Schools |
| Results and Impact | In joint work with the Northern Powerhouse Partnership we produced a website where schools could look up their Progress 8 and Adjusted Progress 8 scores. This sparked media interest and enquiries from those in the schools sector about scores and the change under adjustment for pupil background characteristics. |
| Year(s) Of Engagement Activity | 2018 |
| URL | http://orthernpowerhousepartnership.co.uk/publications/latest-research-shows-league-tables-punish-an... |
| Description | Website: Fairer Schools Index 2019. Northern Powerhouse Partnership collaboration. |
| Form Of Engagement Activity | Engagement focused website, blog or social media channel |
| Part Of Official Scheme? | No |
| Geographic Reach | National |
| Primary Audience | Schools |
| Results and Impact | In joint work with the Northern Powerhouse Partnership we produced a website where schools could look up their Progress 8 and Adjusted Progress 8 scores. This was accompanied by media interest and sparked further interest from those involved in the schools sector. |
| Year(s) Of Engagement Activity | 2019 |
| URL | https://www.arcgis.com/apps/dashboards/1741a670cfcb493eb2cb20f14af8a064 |