Case Study I: How to improve an ECE Program A Good Start Program - Chile Juan Baron Yoshikawa et al (2015). Experimental Impacts of a Teacher The World Bank Professional Development Program in Chile on Preschool Classroom Quality and Child Outcomes. Developmental Psychology, 2015, Vol. 51, No. 3, 309–322 Chile Colombia Juan Chile Learning crisis (WDR, 2018) • Children need to be better prepared to learn: Quality pre-school programs (one aspect) • Early learning opportunities -> later educational attainment/$$ (Evidence) • Important expansion of pre-school programs in Latin America (however: snack, recess, behavior management) • Quality is the biggest challenge: no expansion without quality! • Chile - > Improve children development through classroom quality Chile Un Buen Comienzo (A Good Start; UBC) Look at available evidence from other countries: • Classroom Instruction to Support Language and Literacy Development ü Teachers better supported to improve children’s language and literacy: coaching and focus on language • Targeting Classroom Behavior Management üChildren benefit from instruction when they are better at self-regulating emotions/behaviors, less distracted, low disruptive behaviors, high levels of prosocial behaviors Piloted in 4 schools in 2007 – implementation checks Chile Un Buen Comienzo (A Good Start; UBC) Standards and Curricula in Preschool • Learning standards of some dimensions • Broad standards - flexibility for teachers • No specified frequency of instructional or classrooms strategies Innovations a) intensity of the professional development, including the implementation of in-classroom coaching b) training of all adults in the classrooms (teachers and aides) in the same topics and strategies c) collaborative nature of the development of the program. Chile Un Buen Comienzo (A Good Start; UBC) Expected Outcomes • quality of classroom practices in prekindergarten classrooms after 1 school year and kindergarten classrooms after 2 UBC years • children’s language, literacy, and socioemotional skills at the end of year 2, among children recruited at the beginning of prekindergarten Chile Did this program work? Design of evaluation: • 64 schools (why 64? Power analysis) • Schools randomly assigned within each municipality (criteria) • Two groups: full UBC vs. Comparison Posttest Pre-test End-of- End-of- assessment prekindergart. kindergart. (child outcomes and classroom quality) assessment assessment (Classroom quality) (child outcomes and classroom quality) Chile The UBC Program (Detail) Full UBC group: • 12 modules (6 per year, 6 months each) • Didactic (content-focused) activities, coaching, library 100 books x class • Each module: • 4 weekly activities • Initial 4-hour didactic workshop to introduce the topic/introduce strategies • 2 coaching session over the following 2 weeks of each module (2 visits per month) • Coaching session: (a) meeting coach/teacher to plan, (b) implementation of plan, (c) feedback meeting +/- • Domains: (i) oral language and early literacy development; (ii) socio-emotional development; (iii) coordination ECE and health Comparison: • 10 books; one workshop on self-care and stress reductions for teachers Chile Sample • 64 schools, 90 classrooms, 119 teachers, 94 aides and 1,876 children (4 year-olds) Full UBC group • 32 schools, 51 classrooms, 66 teachers, 54 aides, 1,033 children Comparison group • 32 schools, 39 classrooms, 53 teachers, 40 aides • 843 children Chile Un Buen Comienzo (A Good Start; UBC) Expected Outcomes • quality of classroom practices in prekindergarten classrooms after 1 school year and kindergarten classrooms after 2 UBC years • children’s language, literacy, and socioemotional skills at the end of year 2, among children recruited at the beginning of prekindergarten Chile UBC on quality of classroom practices End of Prekindergarten Impact Posttest Impact 6 6 5 5 4 4 3 3 2 2 1 1 0 0 Emotional Instructional Classroom Emotional Instructional Classroom support support organization support support organization Full UBC Comparison Full UBC Comparison Chile UBC on children’s outcomes Impact of UBC on Children's Outcomes 30 25 20 15 10 5 0 Langua & Letter-word Early writing Socio-emotional vocabulary identification skills Full UBC Comparison Chile Un Buen Comienzo (A Good Start) Expected Outcomes • quality of classroom practices in prekindergarten classrooms after 1 school year and kindergarten classrooms after 2 UBC years • children’s language, literacy, and No socioemotional skills at the end of year 2, among children recruited at the beginning effect of prekindergarten So what? • UBS changed the behavior of teacher/aides in the classroom • UBS did NOT change children’s outcomes • Someone wrote a paper with the IE result Program is being adjusted Case Study I: How to improve an ECE Program A Good Start Program - Chile Juan Baron Yoshikawa et al (2015). Experimental Impacts of a Teacher The World Bank Professional Development Program in Chile on Preschool Classroom Quality and Child Outcomes. Developmental Psychology, 2015, Vol. 51, No. 3, 309–322 To consider … 1. That you select the school doesn’t mean teachers and aides will participate (agreement to participate) 2. Working sample 3. Take up during implementation (actual participation) 4. How to collect information (baseline, follow-ups) - logistics 5. How to measure outcomes (children’s outcomes, classroom quality) and other socio-demographic variables 6. How to analyze data if issues arise (e.g. changers, attrition) • Heterogeneous effects 7. Statistical significance vs. economic significance (size) – indices 8. Cost-effectiveness of interventions