The "Irie Classrooms Toolbox": a cluster randomised trial of a universal violence prevention programme in Jamaican preschools.

Lead Research Organisation: Bangor University
Department Name: Sch of Psychology

Abstract

Violence is a leading worldwide public health problem with a very high prevalence in Jamaica. Interventions in early childhood are an important component in the primary prevention of violence. Training young children's caregivers in behavioural strategies to reduce child aggression and promote child social skills can 1) prevent the early development of antisocial behaviour in children and 2) reduce violence against children by caregivers. Corporal punishment is widely used in schools across Jamaica despite efforts by the Ministry of Education to eliminate it. Although banned by law in early childhood institutions, corporal punishment is still common in Jamaican preschools. The prevalence of conduct problems in young Jamaican children has not been accurately determined but estimates suggest it is between 12%-21%. Preschool teachers report child behaviour problems as a major concern and feel ill equipped to deal with them.
We have shown that teachers trained to use evidence-based child behaviour management strategies provide a more emotionally supportive classroom environment and use less corporal punishment. This translates to improvements in class-wide child behaviour; reductions in high-risk children's conduct problems and increases in their social skills at school and at home. Through several years working with preschool teachers in Jamaica, we have developed an intervention that involves guiding teachers in the use of a "toolbox" of key behaviour management strategies that are relevant, easy to use, effective and flexible. The intervention is delivered through participatory and practical methods and has been specifically designed to be scalable, low cost and suitable for use in low resource settings with teachers with limited formal teacher training.
We now plan to evaluate this intervention in 76 preschools (228 classrooms) to determine if benefits are obtained when the training is disseminated on a large scale. The preschools will be randomly assigned to a group in which all teachers receive the intervention in year 1 (38 preschools, 114 classrooms) and a second group that will receive the intervention a year later (38 preschools, 114 classrooms). The intervention will involve 5 full-day workshops, monthly in-class support and weekly text messages over the course of a school year. We will evaluate the effect of the intervention on class-wide measures of child aggression and teachers' use of corporal punishment and verbal abuse. We will also evaluate whether the intervention benefits the quality of the classroom environment, class-wide measures of children's prosocial behaviour, teachers' mental health and children's self-regulation, mental health and attendance. We will examine issues surrounding implementation of the intervention including measuring the degree to which teachers adopt the strategies in the classroom, factors affecting teachers' uptake of the intervention and teachers' views on the components of the toolbox and the training. This will help us to refine the toolbox to ensure its' suitability for implementation at scale. An economic evaluation will be conducted to ascertain the costs associated with delivering the intervention to teachers and the ratio of costs to the benefits obtained.
This study has the potential to make an enormous public health impact in Jamaica with benefits to reduced aggression and improved child mental health at the population level. We have close links with the government agencies and training institutions responsible for early childhood education in Jamaica and are confident that, if this study shows positive results, the intervention will be integrated into on-going teacher training initiatives. Furthermore, the intervention would be suitable for use at the preschool and early primary school level in other low and middle-income countries with potential to be an important component of global initiatives to prevent violence against children and to promote child mental health.

Technical Summary

Aim: To determine the effects of school-based violence prevention programme, implemented in Jamaican preschools, on the levels of aggression among children in school and violence against children by teachers.
Study Design: Cluster randomized controlled trial
Sample: 76 preschools in urban areas of Kingston, Jamaica will be randomly assigned to intervention (n = 38) or control (n = 38). Each preschool comprises three classrooms (one 3, 4 and 5 year old class) and all teachers and classrooms will be included in the study (228 teachers/classrooms: 114 in each group). In addition, a random sample of 6 children in the 4 and 5-year old classes will be selected for evaluation of child-level outcomes (912 children: 456 children in each group).
Intervention: The intervention involves training teachers in classroom behaviour management and in strategies to promote children's social-emotional competence. Training is delivered through 5 full-day workshops, monthly in-class coaching over two school terms and weekly text messages.
Measurements: The primary outcome measures are: i) observed violence against children by teachers and ii) observed levels of child aggression in the classroom. Secondary outcomes include observations of the quality of the classroom environment, teachers' reports of their mental health, teacher reported child mental health, direct tests of child self-regulation, child reports of teachers' use of corporal punishment and child attendance.
Economic, Social, Qualitative Measures: Implementation science is incorporated throughout the study to determine the factors affecting the quality of implementation and how implementation is associated with outcomes. An economic evaluation will also be conducted.
Implications: This study involves integrating an intervention into an existing service that is universally accessed by young Jamaican children. The intervention thus has the potential to make an enormous public health impact at the population level.

Planned Impact

The most direct beneficiaries of this research will be the participant teachers and children. In addition, if teachers continue to use these strategies, they will continue benefiting new cohorts of children over time.
There is a high level of interest and concern around the issue of corporal punishment in schools in Jamaica and we anticipate wide interest in the results of this study. Our dissemination activities (as listed in the communications plan) should raise awareness in Jamaican society that alternative behaviour management strategies are available and effective. There is also great concern about the increasing levels of violence and aggression by Jamaican children in school and this study will inform strategies to reduce this by demonstrating the value of appropriate behaviour management and explicit teaching of social skills for appropriate child behaviour.
If the intervention is effective, our greatest potential impact within Jamaica would result from wide-scale adoption by the MOE, the ECC, the teacher training colleges and NCTVET. These institutions are actively seeking appropriate strategies to eliminate the use of corporal punishment in schools and this intervention is suitable for integration into pre-service and in-service teacher training. Integrating the intervention into existing teacher training initiatives has the potential for a large impact in terms of reducing violence against children and improving child mental health at the population level in Jamaica. Given our close links with these organisations, we anticipate that uptake of this intervention should occur over the short to medium term with benefits seen within five years from the end of the study.
The intervention has been designed to be suitable for implementation in low-resource settings with teachers with low levels of training in education. If the intervention implemented at scale is effective at reducing violence against children by teachers and improving child behaviour in the Jamaica early childhood setting, we will work with colleagues in the UNICEF regional office and the Inter-American Development Bank (IDB) to disseminate the intervention more widely across Latin America and the Caribbean. The intervention also has the potential to be adapted for use with early childhood practitioners in other low and middle income countries as it uses practical and participatory training methods to teach key behaviour principles that we have found to be relevant, easy to use, effective and feasible for poorly trained teachers, with limited resources and high child-staff ratios. Professor Walker is currently PI on a grant from Grand Challenges Canada to produce and disseminate an evidence-based parenting intervention package suitable for use in LMIC on the web. If the proposed intervention is effective, we plan to add the teacher-training package to the website, thus making it available to the international community.
TMRI has an international reputation for conducting high quality research addressing important public health problems for the region and beyond and is a prominent training institute for public health researchers in the Caribbean region. This study will support several graduate students in Jamaica who have been working with Dr. Baker-Henningham to develop and evaluate this intervention. Building the research capacity of junior Jamaican staff will facilitate the continuation of this programme of research in the future.
The training is based on sound theoretical principles with demonstrated effectiveness in the Jamaican setting. We are closely monitoring implementation dosage and quality throughout the trial through qualitative and quantitative methods to evaluate how this affects behavioural outcomes. This aspect of the study will contribute to the literature on implementation and dissemination of evidence-based interventions in service settings regardless of the outcome of the trial on teacher and child behaviour.

Publications

10 25 50
 
Description Through this work, we developed, implemented and evaluated the Irie Classroom Toolbox: a low-cost, violence prevention, teacher-training programme suitable for us in low- and middle-income countries.
1. The award facilitated the development of the Irie Classroom Toolbox teacher-training package. This training package includes materials for teachers and materials for facilitators. Teacher materials include a theory book, an activity book of lesson plans, games and activities, fourteen social problem-solving stories and a resource book of visual aids, behaviour plans and photocopiable resources. Facilitator materials include a fully scripted training manual and a coaching manual for use in in-class support sessions. These teacher and facilitator materials are freely available through a creative commons license to the global community.
2. We developed an observational schedule to measure teachers' use of violence that was reliable and sensitive to change. With use of this observational schedule, the study provided rigorous data on the use of violence against children in early childhood classrooms in Jamaica thus providing an important advocacy tool.
3. There were large reductions in teachers use of violence against children immediately after programme implementation and at one-year follow up for teachers trained in the Irie Classroom Toolbox. There were no benefits of intervention to observations of class-wide child aggression. Other benefits of intervention included improved quality of the classroom environment, increases in class wide prosocial behaviour, improved teacher well-being and reduced teacher-turnover. Children taught by teachers who participated in the programme scored higher on a battery of inhibitory control tests and were less likely to be in the clinical range for behaviour problems than children in the comparison group.
4. Although we found large reductions in teachers' use of violence, the majority of teachers continued to use violence as a behaviour management strategy at times. Through a qualitative evaluation of the programme, we identified the need for additional training aimed at building teachers' self-regulatory capacities and changing teachers attitudes and beliefs relating to the use of corporal punishment. We are currently developing and testing this additional content.
Exploitation Route The Irie Classroom Toolbox has been adopted by the Early Childhood Commission (ECC), Jamaica as their early childhood teacher-training programme in classroom behaviour management. We are currently working with the ECC to disseminate the Irie Classroom Toolbox nationally within Jamaica through ongoing in-service training initiatives. This school year (2019-2020), we are training and supporting the senior team from the ECC to implement the Toolbox with selected teachers. This team comprises the early childhood inspectors and supervisors in all six educational regions in Jamaica. In the 2020-2021 school year, the senior team will train and support all early childhood development officers and inspectors in their respective educational regions in the Toolbox so that it can be fully integrated into in-service training.
All resources are available through a Creative Commons License and the teacher resources are available for download from our website (www.irietoolbox.com) .

We continue to conduct workshops and professional seminars for early childhood educators in Jamaica as part of the Professional Development Institute of the Early Childhood Commission. Through these workshops we raise awareness of the Toolbox, create demand for the training, and direct teachers to the website.

We have close links with UNICEF and the Inter-American Development Bank and we plan to leverage these links to disseminate the intervention to policy makers and practitioners in the wider Caribbean and Latin America.

The work also contributed to building the capacity of eight junior Jamaican research staff in the design, implementation and evaluation of early childhood interventions. These staff are continuing to work on research related to early childhood teacher and parent-training programmes within Jamaica and the aim is that they will become independent researchers who will lead this programme of research in the future. In addition, every year for four years we trained up to 25 researchers in data collection methods including observational assessments, direct child testing and administering face-to-face interviews. The majority of these researchers are currently engaged in or have completed research degrees (MPhil and PhD) and this study increased their competence related to data collection and data management skills.

The work has led to professional development training and upskilling of paraprofessionals and professionals in Jamaica. This includes training 35 government staff from the Ministry of Education and Early Childhood Commission to deliver interventions using evidence-based behaviour change techniques and directly upskilling over 300 early childhood practitioners.
Sectors Education

 
Description Within Jamaican Primary Schools: the research team have trained 15 technical staff from the Ministry of Education in Jamaica to deliver a one-day teacher training workshop in psychosocial support based on the IRIE Classroom Toolbox. These technical staff delivered the training to all grade one, two and three teachers across Jamaica (100% saturation) as an integral part of in-service teacher training at the national level. 5,100 primary school teachers were trained reaching 120,000 children aged 6-8 years every year. Hard copies of the IRIE Classroom Toolbox teacher resources have been distributed to all primary schools across Jamaica to ensure teachers can fully utilise this training. Within Jamaican Preschools: In collaboration with UNICEF and the Early Childhood Commission in Jamaica, the research team are facilitating the national dissemination of the IRIE Classroom Toolbox for early childhood practitioners. Sixteen senior staff and 4 members of the executive from the Early Childhood Commission have been trained to deliver the programme. These staff are implementing the programme with over 150 teachers during the 2019-2020 school year. In July 2020, these staff will in turn train all (100%) of the development officers and inspectors responsible for in-service teacher-training in Jamaica (approx. 100 persons). These persons will train over 1,000 teachers per year in the programme benefitting over 20,000 children aged 3-6 years every year. Impact on Policy and Programming: The IRIE Classroom Toolbox has been adopted as the national training programme to train early childhood practitioners in classroom behaviour management in Jamaica. The core content of the Toolbox has also been integrated into the services provided to primary schools by the Counselling Unit within the Ministry of Education. Through these training initiatives, the quality of early childhood education provision across Jamaica will improve and teachers use less violence against children. When teachers utilise the skills taught through the programme, benefits are found for child mental health and social-emotional competence. Furthermore benefits are found for teachers' well-being and teacher retention. Reductions in teacher-turnover has economic benefits for Jamaica.
First Year Of Impact 2016
Sector Education
Impact Types Policy & public services

 
Description Early Childhood Commission has adopted the Irie Classroom Toolbox as part of their in-service training initiatives
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact The Irie Classroom Toolbox is being disseminated through in-service training initiatives to early childhood educators in Jamaica. This school year (2019-2020) we are working in all six educational regions across Jamaica. We have trained the senior ECC staff in each region to conduct teacher training workshops and approximately 100 early childhood practitioners are being reached through this initiative. In July 2020, all early childhood development officers and inspectors will be trained to deliver the Irie Classroom Toolbox workshops so that all early childhood educators across Jamaica can access the training.
 
Description Integrated aspects of Irie Classroom Toolbox into national training initiatves
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact The PI developed a one day training suitable for use with teachers of children in grades 2-3 based on the content of the 'Irie Classroom Toolbox' The training was delivered to all grade two and three teachers in government schools in Jamaica (approx 3,500) teachers reaching up to 80,000 children/year) as an integral part of in-service teacher training.
 
Description Integrated aspects of the Irie Classroom Toolbox into national in-service training initiatives
Geographic Reach National 
Policy Influence Type Influenced training of practitioners or researchers
Impact The PI and senior members of the research team trained 15 technical staff from the Ministry of Education, Youth and Information to deliver a one day training in classroom behaviour management/psychosocial support to Grade 1 teachers in Jamaica. We designed the training based on the content of the 'Irie Classroom Toolbox' and produced a training manual and all training materials (e.g. charts, hand-outs) for the one-day training. The training was delivered to all grade one teachers in government schools in Jamaica (approx 1,700 teachers reaching up to 40,000 children/year) as an integral part of in-service teacher training.
 
Description Task Force to address Corporal Punishment in Schools in Ministry of Education
Geographic Reach National 
Policy Influence Type Participation in a advisory committee
 
Description Dissemination of the Irie Classroom Toolbox with the ECC
Amount $150,000 (USD)
Organisation UNICEF 
Sector Public
Country United States
Start 07/2019 
End 08/2020
 
Description ESRC IAA Impact Grant
Amount £25,000 (GBP)
Organisation Bangor University 
Sector Academic/University
Country United Kingdom
Start 08/2018 
End 01/2019
 
Description Global Mental Health
Amount $268,922 (CAD)
Funding ID 0592-04 
Organisation Government of Canada 
Department Grand Challenges Canada
Sector Public
Country Canada
Start 10/2014 
End 09/2016
 
Description Joint Global Health Trials scheme - call 8
Amount £180,852 (GBP)
Funding ID MR/R018421/1 
Organisation Medical Research Council (MRC) 
Sector Public
Country United Kingdom
Start 09/2018 
End 11/2019
 
Description Preparing the IRIE Classroom Toolbox for National Dissemination in Jamaica
Amount £31,775 (GBP)
Organisation Bangor University 
Sector Academic/University
Country United Kingdom
Start 01/2020 
End 07/2020
 
Description Saving Brains
Amount $250,000 (CAD)
Funding ID 1707- 08326 
Organisation Government of Canada 
Department Grand Challenges Canada
Sector Public
Country Canada
Start 08/2017 
End 07/2019
 
Title Concurrent and longitudinal impact of violence prevention programme 
Description The database consists of pre, post and follow-up data collected in 76 preschools in Kingston and St Andrew, Jamaica. Data includes observations of the quality of the preschool classroom environments using the CLASS Pre-K observation schedule, structured observations of teachers' use of violence against children (physical violence, verbal abuse and other abuse), measures of teachers depression, burn-out and teaching self-efficacy by teacher self-report, and measures of inhibitory control, behaviour difficulties and prosocial skills for a random sample of 4 year old children. Data includes information on 229 teachers at baseline, 233 at post-test and 230 teachers at one year follow-up. 864 children were tested at baseline and 783 at post-test. A further cohort of 838 children were tested at follow-up. 
Type Of Material Database/Collection of data 
Year Produced 2018 
Provided To Others? No  
Impact The Irie Classroom Toolbox has been adopted by the Early Childhood Commission in Jamaica for in-service training in classroom behaviour management for early childhood practitioners. It is also being used to address the national strategic plan of action for an integrated response to violence and children. The government have made plans to disseminate the Toolbox nationally to prevent the use of violence against children by teachers in schools. The Ministry of Education have delivered a one day training in the The Irie Classroom Toolbox to all teachers of children in grades 1-3 in primary schools at the national level. Each primary school has a set of the resources required to implement the training. 
 
Title Effect of the Irie Classroom Toolbox in the wait-listed control group 
Description The database consists of pre and post measures collected in 37 schools in Kingston and St Andrew, Jamaica. Data includes observations of the quality of the preschool classroom environments using the CLASS Pre-K observation schedule, structured observations of teachers' use of violence against children (physical violence, verbal abuse and other abuse), measures of teachers depression, burn-out and teaching self-efficacy by teacher self-report 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? No  
Impact Data was used to further refine the Irie Classroom Toolbox prior to national roll out. 
 
Title Process evaluation of the Irie Classroom Toolbox 
Description We conducted an ongoing process evaluation of the implementation of the Irie Classroom Toolbox with 230 teachers over two rounds of implementation. Qualitative and quantitative data was collected. Qualitative data consists of facilitator record forms documenting enablers and barriers at the level of the teacher, children, facilitator and context after each in-class support session. Facilitator record forms were completed monthly for each teacher over 8 months of implementation in each round. The data consists of 1840 facilitator record forms. Quantitative data consists of teacher attendance, participation and engagement in the intervention, and measures of teacher satisfaction and teacher-facilitator relationships. 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? No  
Impact The data has been used in the design of the technical guides used to train the facilitators of the Irie Classroom Toolbox. These technical guides are currently being used to train government early childhood education supervisors to implement the toolbox with early childhood practitioners in their region. 
 
Title Qualitative evaluation of the Irie Classroom Toolbox 
Description We conducted qualitative interviews with 38 early childhood educators who had participated in the Irie Classroom Toolbox. Interviews lasted between 45-60 minutes. All interviews were audiotaped and transcribed and the data thus consists of 38 transcripts. 
Type Of Material Database/Collection of data 
Year Produced 2019 
Provided To Others? No  
Impact The results of the qualitative evaluation were used to inform the further roll-out of the Irie Classroom Toolbox across Jamaica in collaboration with the Early Childhood Commission. The results of the evaluation are also being used to develop new content including content to promote teachers' self-regulatory skills. 
 
Title Quality of preschool classroom environments 
Description The database consists of data collected in 76 preschools (229 classrooms) in Kingston and St Andrew, Jamaica. Data includes observations of the quality of the preschool classroom environments using the CLASS Pre-K observation schedule, structured observations of teachers' use of violence against children (physical violence, verbal abuse and other abuse), measures of teachers depression, burn-out and teaching self-efficacy by teacher self-report and measures of inhibitory control and school attendance for 864 children age 4 years. 
Type Of Material Database/Collection of data 
Provided To Others? No  
Impact The data has been used to highlight the prevalence of teachers' use of violence against children in school at the early childhood level. A task force to address the matter of corporal punishment in schools has been formed within the Ministry of Education following a meeting that we had with the Minister of Education, Chief Education Officer and other Executive members of the MOEY in January 2016. 
 
Description Collaboration with UNICEF and the Early Childhood Commission 
Organisation UNICEF
Country United States 
Sector Public 
PI Contribution We are training the senior early childhood education supervisors and inspectors in each educational region in Jamaica in the Irie Classroom Toolbox. This involves training the ECC team through a series of workshops and providing support as the team conduct workshops with the teachers. We are also preparing technical guides for use by the ECC team to maximise the quality and fidelity of the training. We have also provided full kits of the Irie Classroom Toolbox to the ECC to facilitate the trainings.
Collaborator Contribution UNICEF are providing funds to support the training and the ECC are allocating staff time to the collaboration. The ECC are also covering costs of transportation and consumables.
Impact The collaboration is multi-disciplinary and includes the disciplines of early childhood education, psychology, child health and epidemiology. The outputs include: 1) a revised training manual, 2) a supervisory checklist, 3) a guide documenting the alignment of the Irie Classroom Toolbox with the Jamaican early childhood curriculum and the Operational Standards of early childhood institutions, and 4) technical guides to assist early childhood education supervisors to incorporate the Irie Classroom Toolbox into school development plans.
Start Year 2019
 
Title Enhanced version of the Irie Classroom Toolbox 
Description The Irie Classroom Toolbox consists of materials for teachers and materials for facilitators. Materials for teachers include: 1) a theory book describing all the strategies introduced through the Toolbox, 2) an activity book giving lesson plans, games, and activities, 3) a resource book of photocopiable resources including visual aids, behaviour planning forms and Irie notes home and 4) fourteen social problem-solving stories to use with young children. Materials for facilitators include: 1) a training manual, 2) a coaching manual and 3) a series of video vignettes 
IP Reference  
Protection Protection not required
Year Protection Granted 2019
Licensed Yes
Impact The Irie Classroom Toolbox has been adopted by the Early Childhood Commission as their in-service training programme to train early childhood educators in classroom behaviour management and to promote children's social and emotional competence.
 
Title The Irie Classroom Toolbox 
Description We have developed The Irie Classroom Toolbox: a behavioural intervention to train preschool teachers in low and middle-income countries in classroom behaviour management and in strategies to promote child mental health. The following aspects of the toolbox are now available through a Creative Commons License: a) Two manuals for teachers: a toolbox book giving simple and clear instructions of how to use each behaviour management strategy and an activity book of songs, games and lesson plans to promote children's executive function skills and social-emotional competence, b) Three sets of picture cards illustrating classroom rules, friendship skills and emotion faces to facilitate the teaching of key skills, and c) Fourteen picture stories of common classroom problems faced by children with potential solutions (e.g. the need to share materials, wait for a turn etc.). 
IP Reference  
Protection Protection not required
Year Protection Granted 2017
Licensed Yes
Impact Some of the materials have been given to the MOEYI in Jamaica to be distributed nationally. We are currently in discussion with the MOEYI and the Early Childhood Commission to make these materials available more widely (e.g. on DVD and on the internet.
 
Description Interview on radio 
Form Of Engagement Activity A press release, press conference or response to a media enquiry/interview
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Media (as a channel to the public)
Results and Impact The PI was interviewed on the radio programme 'Power Talk' on Power 106 Jamaican radio on the 'Irie Classroom Toolbox' including the rationale for the intervention, the impacts on teacher practices and the policy implications.
Year(s) Of Engagement Activity 2018
 
Description Presentation at local conference 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The Irie Classroom Toolbox team gave a panel presentation at the Caribbean Child Research Conference. This conferences targets high school children and early childhood practitioners. The presentations led to a great deal of discussion and interest in the teacher training programme.
Year(s) Of Engagement Activity 2017
 
Description Teacher training workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact The Irie Classroom Toolbox Team conducted 2 workshps for early childhood practitioners to introduce them to evidence-based strategies to manage child behaviour in school. Each workshop involved approx 55 participants. The workshops were conducted as part of of the annual Dudley Grant Early Childhood Conference. Teachers were interested in being trained in the intervention and we are working with the Early Childhood Commission to disseminate the intervention.
Year(s) Of Engagement Activity 2017
 
Description Workshop at the Professional Development Institute of the Early Childhood Commission, Jamaica 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Professional Practitioners
Results and Impact We conducted two workshops with early childhood educators attending the Professional Development Institute of the Early Childhood Commission in Jamaica.
The workshops led to increased use of our website by early childhood educators in Jamaica and created demand for the programme.
Year(s) Of Engagement Activity 2019