The moderating role of reading proficiency in pupils' performance on national and international assessments of primary mathematics

I propose three phases of research aimed at addressing three main research questions. The intentions here are to improve and extend Mullis et al.'s coding approach, apply this to existing performance data for pupils in England, and assess whether language modifications to these test-items improves the performance of pupils with weaker reading skills. Phase 2 focuses on research question 2, while Phase 3 focuses on question 3. All three phases address research question 1.
Phase 1, in combination with a wider literature review, aims to identify potential weaknesses in Mullis et al.'s coding approach, by identifying cases in which the outcome of their coding is incongruent with pupils' actual comprehension difficulties, and identifying why this is the case. This will involve presenting children with a variety of TIMSS 2011 test-items, and using a cognitive interview approach to discuss their interpretation of the question. The child might be asked to rephrase the question in their own words, discuss their method for tackling the question, or identify which of a given set of questions asks the same question as the original. The interviews will be conducted in a format similar to that used by Martiniello (2008), though are anticipated to be shorter, at approximately 15-20 minutes per session, rather than 60-90 minutes. I intend to recruit approximately 40 children, just less than double the number used by Martiniello; given the difference in session length, 40 is a feasible number of cognitive interviews to conduct.
Phase 2 requires the use of the National Pupil Database to explore item-level performance of England's TIMSS 2015 cohort. These students were among those who sat the first wave of the new KS2
assessments in 2016, and this analysis aims to evaluate how strongly performance in KS2 reading moderates performance on both KS2 and TIMSS mathematics test-items. I will explore how this relationship is moderated by the item's reading demands. In accordance with the findings of previous research, it is hypothesised that pupils with lower performance in the reading assessments will have had less success on high reading demand mathematics test-items in both the TIMSS and KS2 assessments than pupils, whereas the performance gap will be smaller on low-reading demand items. Items will be classified according to a newly developed categorisation scheme based on that by Mullis et al., but incorporating the findings from Phase 1 and the wider literature review.
In Phase 3, approximately 400 pupils in Years 5 and 6 will take part in a class level test of mathematics. The test-items that are chosen will include both original and modified versions of recent KS2 and TIMSS questions with varying mathematical and reading demands. Pupils' performance on these test-items would then be compared to teacher judgements of reading proficiency, as well as, ideally, historical data such as their KS1 reading performance. Modified questions will attempt to reduce the reading demands of items previously identified in Phase 2 as having had the largest variation in performance by pupil reading proficiency. The data would be collected as a class level (i.e. conducted like a class test), and last 40 minutes, the same as a KS2 reasoning paper. Hypotheses for these are likely to emerge from the findings of the secondary analysis, though it is again suspected that weaker readers will perform particularly poorly on high reading-demand items. It is hoped that the performance gap between stronger and weaker readers will be reduced on modified versions of high reading-demand items.