Improving Numeracy Outcomes for 'At Risk' Children: A cluster-Randomised Control Trial of Precision Teaching Methods.
Lead Research Organisation:
Bangor University
Department Name: Sch of Psychology
Abstract
Despite an evidence-base supporting early intervention many students in Wales still reach high school without necessary basic numeracy skills (OECD, 2014). Academic underachievement relating to skill deficits has been conceptualised in the 'cumulative dysfluency' paradigm (Binder, 1996). That is, underachievement is a result of a student not acquiring multiple prerequisite skills to a level of mastery before progressing onto complex skills. Evidence-based approaches have been applied within education to remediate the cumulative dysfluency effect (e.g., Gallagher, 2006; Hughes et al., 2007; Hunter, et al., 2016; Beverley et al., 2016).
PT is an example of an 'accelerated learning' system derived from the basic principles of learning (Fredrick & Hummel, 2004). This helps children learn skills faster than would be typically expected by employing mastery-based methods (Kubina, et al., 2002). PT has been shown to be effective at improving basic skills across a wide range of domains including literacy (e.g., Cavallini et al., 2010) and numeracy (e.g., Brady & Kubina, 2010; Hayden & McLaughlin, 2004). PT interventions have typically adopted small n designs, aimed at children diagnosed with specific difficulties. More recent class-wide experimental studies have extended the evidence-base for effectiveness in mainstream classrooms (e.g., Hunter et al., 2016; Beverley et al., 2009, 2016). However, no research has evaluated PT using gold standard RCT methods, which offer more robust evidence for interventions effectiveness (Thornicroft, 2011). This would represent an important addition to the evidence-base for effective early interventions.
The primary aim of this research is to extend the current evidence-base for the effectiveness of Precision Teaching methods in schools across North Wales. We aim to do this by conducting a large-scale cluster-Randomised Control Trials (c-RCTs) of Precision Teaching methods. This will be aimed at reducing the academic attainment gap between children 'at risk' of poor academic outcomes (i.e., pupils who are eligible for Free-School Meals [eFSM] or come from deprived backgrounds) and their average peers. Specifically, this project aims to improve attainment in numeracy.
Closing this attainment gap is a key target for Welsh Government (2016) and the Well- being and Future Generations Act (2015). It is also a priority for GwE (Regional School Effectiveness and Improvement Service for North Wales), as set out in their regional business plan (GwE, 2015). The proposed research will build on a systematic research and implementation programme currently being conducted as part of the Collaborative Institute for Education Research and Impact (CIEREI).
PT is an example of an 'accelerated learning' system derived from the basic principles of learning (Fredrick & Hummel, 2004). This helps children learn skills faster than would be typically expected by employing mastery-based methods (Kubina, et al., 2002). PT has been shown to be effective at improving basic skills across a wide range of domains including literacy (e.g., Cavallini et al., 2010) and numeracy (e.g., Brady & Kubina, 2010; Hayden & McLaughlin, 2004). PT interventions have typically adopted small n designs, aimed at children diagnosed with specific difficulties. More recent class-wide experimental studies have extended the evidence-base for effectiveness in mainstream classrooms (e.g., Hunter et al., 2016; Beverley et al., 2009, 2016). However, no research has evaluated PT using gold standard RCT methods, which offer more robust evidence for interventions effectiveness (Thornicroft, 2011). This would represent an important addition to the evidence-base for effective early interventions.
The primary aim of this research is to extend the current evidence-base for the effectiveness of Precision Teaching methods in schools across North Wales. We aim to do this by conducting a large-scale cluster-Randomised Control Trials (c-RCTs) of Precision Teaching methods. This will be aimed at reducing the academic attainment gap between children 'at risk' of poor academic outcomes (i.e., pupils who are eligible for Free-School Meals [eFSM] or come from deprived backgrounds) and their average peers. Specifically, this project aims to improve attainment in numeracy.
Closing this attainment gap is a key target for Welsh Government (2016) and the Well- being and Future Generations Act (2015). It is also a priority for GwE (Regional School Effectiveness and Improvement Service for North Wales), as set out in their regional business plan (GwE, 2015). The proposed research will build on a systematic research and implementation programme currently being conducted as part of the Collaborative Institute for Education Research and Impact (CIEREI).
Organisations
People |
ORCID iD |
| J. Carl Hughes (Primary Supervisor) | |
| Kaydee Owen (Student) |
Publications
Owen K
(2021)
Assessing the Social Validity of the SAFMEDS Strategy From the Perspective of Teachers and Children
in Frontiers in Education
Owen K
(2021)
Implementation Support Improves Outcomes of a Fluency-Based Mathematics Strategy: A Cluster-Randomized Controlled Trial
in Journal of Research on Educational Effectiveness
Watkins R
(2020)
Defnyddio Dull Rhuglder Cyfarwyddiadol i Addysgu Sgiliau Adio mewn Uned Cyfeirio Disgyblion: Astudiaeth Beilot
in Cylchgrawn Addysg Cymru / Wales Journal of Education
Studentship Projects
| Project Reference | Relationship | Related To | Start | End | Student Name |
|---|---|---|---|---|---|
| ES/P00069X/1 | 01/10/2017 | 30/09/2027 | |||
| 1945874 | Studentship | ES/P00069X/1 | 01/10/2017 | 30/09/2020 | Kaydee Owen |
| Description | The purpose of this PhD was to add to the existing literature on evidence-based practice within education. I focused on two education strategies: Precision Teaching (PT) and Direct Instruction (DI). Study 1: "Implementation Support Improves Outcomes of a Fluency-Based Numeracy Strategy: A Cluster-Randomised Controlled Trial" The Say-All-Fast-Minute-Every-Day-Shuffled (SAFMEDS) strategy derives from PT has a growing evidence-base with previous research demonstrating that it can be used to promote fast and accurate fact recall. In this study, we used a cluster randomised controlled trial (c-RCT) to assess the impact of implementation support during a teacher-led SAFMEDS numeracy intervention. We analysed the mathematical fluency data from 575 children, across 60 schools in North Wales. The results suggest that ongoing support, in the form of three 1-hour visits and ongoing email contact with a researcher, has a small effect on children's ability to answer addition, subtraction, multiplication, and division questions fluently (Mathematics Fluency and Calculation (MFaCTs): Grades 1-2, d = 0.23, 95% CI: 0.06 to 0.39; MFaCTs: Grades 3-5, d = 0.25, 95% CI: 0.08 to 0.43). We also found evidence to suggest that the addition of implementation support does not impact children's attitudes towards maths (Test of Early Mathematical Ability (ToMA-3): Attitudes Towards Math, d = 0.13, 95% CI: -0.03 to 0.29). Study 2 & 3: "Assessing the Social Validity of the SAFMEDS Strategy from the Perspective of Teachers and Children" Few published studies have reported the social validity key stakeholders associate with using the SAFMEDS strategy in schools. We disseminated an online survey to teaching staff across North Wales; all of whom have used the strategy to promote fluency amongst their students (N = 55). Using thematic content analysis, we identified five themes amongst their responses: (1) factors that promote progress; (2) factors that limit progress; (3) impact of competition; (4) confidence; and (5) inherent advantages of the SAFMEDS strategy. These themes provide insight into how practitioners and teachers can yield the benefits of the SAFMEDS strategy in schools. We also interviewed children to gain insight into their views on the SAFMEDS strategy. We interviewed 26 children who had used the strategy with their teacher to promote fast and accurate recall of maths facts. This analysis revealed five further themes relating to the strategy: (1) enjoyment; (2) data; (3) sense of achievement; (4) skills; and (5) home use. Study 4: "Supporting Literacy and Numeracy within Inner-City London: A Charity-Led Fluency-Building Programme" The XLP numeracy and literacy project (XL-LAN) aims to support children who are unable to perform age-expected literacy and/or numeracy skills fluently. XLP recruited the expertise of researchers at Bangor university to help train their staff in the use of the Say-All-Fast-Minute-Every-Day-Shuffled (SAFMEDS) strategy. XLP and Bangor University continued to collaborate in order to support the programme in educational environments across inner city London; this includes mainstream schools, pupil referral units, and mentoring services.Using a mixed-method approach, we were able to identify key quantitative differences relating to the children's in-session SAFMEDS scores as well as qualitative themes pertaining to the children's experience of using the strategy to support their learning. The final quantitative dataset included SAFMEDS session from 263 children. Although children studying literacy and numeracy subjects showed gains, these were greater and more stable for children using the literacy packs. Interviews with 38 children who had used the strategy revealed seven themes: (1) skill development, (2) procedure, (3) scoring, (4) revision tool, (5) positive effects, (6) family involvement, and (7) withdrawal from class. Collectively, these data provide useful feedback which may shape the future delivery of the XL-LAN project and other projects utilising the SAFMEDS strategy. Study 5: "Using an Instructional Fluency Approach to Teach Addition Skills in a Pupil Referral Unit: A Pilot Study" Pupil referral units (PRUs) in Wales accommodate children who present with a range of difficulties that cannot be managed within a mainstream setting. Many children attending PRUs in Wales do not develop the numeracy skills that they need to support their learning across the curriculum. In an effort to teach and assess addition skills, the authors assessed the effects of using a combination of DI and PT in a PRU. The results from this study provided evidence to support the use of an instructional fluency approach in a PRU setting to help children develop early mathematics skills, particularly for children who engaged in the sessions regularly. Due to the small sample size, the results of this study have limited generalisability but may help shape future research investigating effective strategies for teaching mathematics in PRUs. |
| Exploitation Route | This research has helped to shape the SAFMEDS training that we deliver to schools across North Wales. We have been able to identify some of the challenges schools face when implementing the strategy and documented the effects of providing low-intensity support following training. Through the c-RCT, qualitative research, and impact work with XLP we have been able to identify the need for additional research that is needed within the field; particularly with regards to assessing and maintaining procedural integrity within teacher led SAFMEDS programmes. Study 5 helped us to explore whether an instructional fluency approach could help improve addition skills within a PRU. This was a small-scale study that produced some interesting and positive results. We hope that this will help shape further research in the field to promote the use of evidence-based pedagogical teaching approaches with children who attend PRUs. |
| Sectors | Education |