Multilevel Modelling of the Government's New School Performance Measures, 'Floor Standards' Target and 'Narrowing the Gap' Priority

Each year, the Government publishes school performance tables that report the achievement and progress of pupils in English secondary schools. These tables are subsequently republished by the media, on websites and as supplements in national newspapers. The Government's aim is to hold schools accountable and to enable parents to make meaningful choices about where to send their children to school.

The Government's 2010 Schools White Paper outlined major changes to these tables which have now been implemented in the 2011 school performance tables. First, they have changed the way they measure pupil progress. Second, they have introduced a new government target which requires all schools to pass 'floor standards' in these measures. Third, they have made it a new government priority that all schools narrow the performance gaps between their most disadvantaged pupils and their peers. These changes have important implications: 'underperforming' schools which fail the new government target will risk being placed in 'special measures', take-over, or even closure.

The high-stakes nature of these changes make it imperative that they are critically reviewed and that innovative techniques are developed and applied to explore potential improvements to the way the Government measure school performances. This proposal will do this by pursuing a programme of methodological and substantive research based on analyses of the highly detailed pupil and school level performance data held by the Government.

The main objectives of this proposal are to:

1. CRITICALLY REVIEW SCHOOL PERFORMANCE MEASURES IN ENGLAND
Despite the high-stakes nature of the new school performance measures, target and priority no such review exists. I will therefore carefully explain the methodology behind each measure, explore the conclusions that each measure allows users to draw, and explain how different measures are relevant to different users (e.g. parents and government) and for different purposes (e.g. school choice and school accountability).

2. RESEARCH POTENTIAL IMPROVEMENTS TO SCHOOL PERFORMANCE MEASURES
I will use statistical models to explore the new school performance measures and government target and the extent to which they might suffer from potential statistical limitations. For example, the measured performances of small schools are known to fluctuate greatly from year to year simply due to the small number of students upon which they are based. However, the government target appears not to recognise this issue or the important implications it has for schools' and ultimately pupils' futures.

I will then extend the proposed models to explore the new government priority and to statistically identify whether schools are narrowing the performance gaps between their most disadvantaged pupils and their peers. Such analyses are a necessary first step in attempts to identify those school policies and practices that can bring about such changes.

3. SKILLS DEVELOPEMENT
I will develop my statistical modelling and software skills through a programme of reading guided by my mentor and will complement this by attending external training workshops and a one-week academic visit to an expert in this area. I will develop my knowledge exchange, media and public engagement skills through training courses provided by my university.

4. DISSEMINATE THE RESEARCH TO ACADEMICS AND NON-ACADEMICS
I will publish the research in academic journals and present it at national and international conferences. I will collaborate with a third sector charity through a one-week placement. I will publicise the research to the Department for Education and non-academics through short research briefings and press releases. Finally, I will train academic and other researchers in the methods used in the research through running a workshop and by providing online training materials.

WHO WILL BENEFIT FROM THE RESEARCH?

GROUP 1
The users of school performance tables, who need to be able to understand and correctly interpret the published statistics, will benefit from the research. This group includes: education policy makers, who use these tables to set government targets and to select schools for policy initiatives; third sector organisations, such as the Education Endowment Foundation (EEF) and the Sutton and Impetus Trusts, who use these tables to select schools for charitable interventions; local authorities and schools, who use these tables for performance monitoring and resource allocation; the media, who present and explain these data to the public; and the wider public themselves, who use these tables to choose which schools to send their children to and to understand how their children's schools are performing locally and nationally.

GROUP 2
The producers of the school performance tables, who need to be aware of the different statistical issues in designing and presenting school performance measures, will benefit from the research. This group includes: the DfE who produce the official school performance tables; and the Fischer Family Trust (FFT) who provide a range of additional performance data to LAs and schools to support target setting and self evaluation.

GROUP 3
The producers of other public sector performance measures, both within the UK and globally, will also benefit from the research; the issues the research will address apply to measuring institution performance in general. Within the UK, this group includes: The Higher Education Funding Council for England (HEFCE) and The Higher Education Statistics Agency (HESA) who both publish higher education performance indicators; The Department of Health (DoH) and Dr Foster who publish hospital waiting times, mortality rates, and readmission rates; and The Ministry of Justice and regional police forces who publish reoffending rates and crime maps.


HOW WILL THEY BENEFIT FROM THE RESEARCH?

GROUP 1
The critical review will benefit the users of school performance tables by carefully explaining: the methodology behind each school performance measure; the conclusions that each measure allows users to draw; how different measures are relevant to different users and are required for informing different types of decisions; and the notion of statistical uncertainly and risk in over interpreting small differences in schools' performances.

This group will also benefit from the substantive findings into how successful schools are at passing the government 'floor standards' target and how successful schools are at the government priority of 'narrowing the gap' between schools' most disadvantaged pupils and their peers.


GROUPS 2 & 3
The methodological research will benefit the producers of school and other public sector performance measures by demonstrating how their, typically descriptive, measures can be formulated as multilevel models, the advantages of doing this, and how a wide range of modelling extensions can then be applied to measure many different aspects of institutions' performances. All of this research will promote the importance of communicating the statistical uncertainty in performance measures to the end user.

This group will benefit from the online training materials which will provide step-by-step instructions on how to specify and interpret multilevel models for performance data and on how to fit these models using MLwiN and other statistical software packages.

This group will also benefit from the one-day training (morning) and research (afternoon) workshop on 'Statistical Modelling of School Performance Measures' that I will run this in the final year of the award.

The FFT will benefit from the one-week knowledge exchange placement by engaging with academic research and collaboratively working towards a co-production of knowledge in measuring schools performance.