The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) Improving the Quality of Initial and Primary Education in Uruguay (P159771) LATIN AMERICA AND CARIBBEAN | Uruguay | Education Global Practice | IBRD/IDA | Investment Project Financing | FY 2017 | Seq No: 12 | ARCHIVED on 02-Feb-2023 | ISR53869 | Implementing Agencies: ANEP - Administracion Nacional de Educacion Publica, Oriental Republic of Uruguay Key Dates Key Project Dates Bank Approval Date: 16-Dec-2016 Effectiveness Date: 08-May-2017 Planned Mid Term Review Date: 15-Jun-2020 Actual Mid-Term Review Date: 11-Oct-2019 Original Closing Date: 31-May-2022 Revised Closing Date: 28-Feb-2023 pdoTable Project Development Objectives Project Development Objective (from Project Appraisal Document) The objectives of the Project are to improve the teaching practices and the learning environment in early and primary education, as well as the internal efficiency in primary education, all in Full-Time Schools, and to strengthen the evaluation capacity of the education system. Has the Project Development Objective been changed since Board Approval of the Project Objective? No Components Table Name Early Education:(Cost $5.53 M) Primary Education:(Cost $26.98 M) Transition between Primary and Secondary Education:(Cost $1.00 M) Monitoring and Evaluation System:(Cost $6.39 M) Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO Moderately Satisfactory Satisfactory Overall Implementation Progress (IP) Satisfactory Satisfactory Overall Risk Rating Moderate Moderate Implementation Status and Key Decisions The Project will close on February 28, 2022. In its last year of implementation, the Project Implementation Unit (PIU) mainly focused on completing the construction of four remaining schools, providing funding for school maintenance, providing physical and education equipment, and ensuring that existing infrastructure is adapted to guarantee COVID-19 sanitation measures. In terms of teacher training, the PIU has provided training and support for teachers to tackle the challenges raised by the COVID-19 pandemic, which particularly increased repetition and absenteeism at the national level. Evaluation activities also continued with the implementation of the Inventario Nacional de Desarrollo Infantil (INDI, the locally-developed early childhood development assessment), Prueba de Lectura, Escritura y Oralidad (LEO, an evaluation applied to a sample of 2nd grade students to assess reading and writing skills)), and Sistema de Evaluación de Aprendizajes (SEA+, an adaptative learning assessment applied to a sample of 3rd and 6th grade students) evaluations in 2022, whose results have yielded updates on the status of education in the post- pandemic context to continue informing future policymaking in the sector. 2/2/2023 Page 1 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) The Project achieved or exceeded three of its four objectives. Progress by Project Development Objective (PDO) Indicators is as follows:  PDO Indicator 1: Increase in the teaching practices score in FTS based on class observation. The Teach classroom observation tool and self-reported data showed an improvement between 2019 and 2021, from 2.6 to 2.7. This indicates that initial changes are occurring in the classroom, which would lead to improved teaching practices in the long term.  PDO Indicator 2: Number of additional students enrolled in schools that meet required quality standards. This indicator surpassed the end target of 4,500, with an additional 9,474 students enrolled in schools that received works to meet the required quality standards by December 2021 when considering Project, counterpart, and public-private financing.  PDO Indicator 3: First grade repetition rate in FTS. This indicator achieved the end target of 9.00%, reaching 9.30% in December 2021.  PDO Indicator 4: Share of schools that use national student-level assessment results for decision-making. This indicator achieved 81.4% in the sample of schools surveyed following the application of the LEO evaluation in 2022, exceeding the end target of 35%. The Results section presents data for the academic year ending in December 2021, which corresponds to the final measurement scheduled under the Project. Risks Systematic Operations Risk-rating Tool Risk Category Rating at Approval Previous Rating Current Rating Political and Governance Low Low Low Macroeconomic Moderate Moderate Moderate Sector Strategies and Policies Low Low Low Technical Design of Project or Program Moderate Moderate Moderate Institutional Capacity for Implementation and Moderate Moderate Moderate Sustainability Fiduciary Moderate Moderate Moderate Environment and Social Low Low Low Stakeholders Low Low Low Other Low Moderate Moderate Overall Moderate Moderate Moderate Results PDO Indicators by Objectives / Outcomes Improve teaching practices in early and primary education IN01216567 ►PDO1. Increase in the teaching practices score in FTS based on class observation. (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 2.60 2.70 2.70 2.91 2/2/2023 Page 2 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) Date 26-Oct-2018 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target not achieved. The Teach classroom observation tool and self-reported data did, however, show an improvement between 2019 and 2021, from 2.6 to 2.7. This indicates that initial changes are Comments: occurring in the classroom, which would lead to improved teaching practices in the long term. IN01216646 Sub-PDO1. Increase in the teaching practices score in FTS based in self-reports. (Percentage, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 69.90 71.70 71.70 80.40 Improve the learning environment in early and primary education IN01216568 ►PDO2. Learning environment. Number of additional students enrolled in schools that meet required quality standards. (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 8,073.00 9,474.00 4,500.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed. The actual value corresponds to the total number of students in schools that meet the standards within the broader Project framework, including those financed with counterpart and public- Comments: private funding. 5,543 additional students were enrolled in schools financed solely from the Project. Improve the internal efficiency in primary education IN01216569 ►PDO3. Internal efficiency. First grade repetition rate in FTS. (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 11.00 9.30 9.30 9.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target achieved. Comments: IN01216647 Sub-PDO3. First grade repetition rate gender gap in FTS. (Percentage, Custom Breakdown) Baseline Actual (Previous) Actual (Current) End Target Value 4.30 3.40 3.40 3.30 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 Strengthen the evaluation capacity of the education system IN01216570 ►PDO4. Evaluation Capacity. Share of schools that use national student-level assessment results for decision-making (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target 2/2/2023 Page 3 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) Value 0.00 35.00 81.40 35.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed. 81.4% of teachers surveyed reported that they used the LEO student results reports to make decisions. The LEO evaluation was applied to 9,436 students in 2022, belonging to 844 Comments: groups in 2nd grade located in 610 schools, which represents 27% of primary schools in the country. Intermediate Results Indicators by Components Early Education IN01216648 ►IRI1. Number of additional classrooms for 3-year-olds resulting from Project interventions (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 18.00 19.00 20.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target achieved. The one remaining classroom to achieve the end target is expected to be Comments: completed by Project closing. IN01216652 ►IRI2. ECD teachers of FTS use mobile libraries at least once a week (Percentage, Custom, PBC) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 65.20 65.20 70.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed (pending final update). The actual data reported corresponds to the measurement taken in December 2018. Preliminary results from the December 2021 measurement show Comments: that 76% of ECD teachers of FTS use the mobile libraries at least once a week, surpassing the end target of 70%. Final results are expected to be available for the ICR. IN01216656 ►IRI3. Increase in the share of 5-year-old students in FTS with an "expected" development level. (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.50 0.81 0.94 1.50 Date 24-Sep-2019 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target not achieved. The drop in the reported development levels from the value of 3.14% achieved in 2019 reflects the interruptions in schooling caused by the pandemic, which disproportionately affected Comments: children in early education (ages 3 to 5). IN01216659 ►IRI4. ECD assessment results inform first grade teachers (Text, Custom, PBC) Baseline Actual (Previous) Actual (Current) End Target 2/2/2023 Page 4 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) ECD assessment results Local ECD (EIT) integrated into Local ECD assessment is GURI's database and assessment developed ECD assessment has developed and Value local ECD assessment and approved by the 50% coverage. approved by the developed and approved Project Implementing Project Implementing by the Project Entity. Entity. (DLI 6) Implementing Entity. Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target achieved. Results of the Evaluacion Infantil Temprana (EIT) were integrated into GURI's database and a local ECD assessment (the INDI) was developed and approved by the Project Comments: Implementing Entity. IN01216662 ►IRI5. FTS ECD teachers apply protocols for at-risk students (Text, Custom, PBC) Baseline Actual (Previous) Actual (Current) End Target 70% of ECD teachers of all FTS carry out ECD teaching guidelines 61.7% of FTS ECD interventions according and protocols for at-risk teachers apply Value Not started to the guidelines for students included in the protocols for at-risk students that do not ECD in-service training students. achieve the expected level of development Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target partially achieved. The pilot to measure teachers' application of the protocols, originally planned for May 2020, was postponed due to school closings as a result of the COVID-19 pandemic. To address this, the self-reported survey was made available to teachers online and the interviews were Comments: conducted by phone/video. The application of the protocols was scaled up and measured in the second semester of 2022, and preliminary data shows that 61.7% of FTS teachers apply the protocols for at-risk students. IN01216664 ►IRI6. ECD teachers of FTS accredit in-service training courses (Text, Custom, PBC) Baseline Actual (Previous) Actual (Current) End Target 350 certified ECD Value Not started 202 certified teachers 202 certified teachers teachers Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed. Accreditation per year is as follows: 2017-18: 265 2019: 293 2020: 226 2021: 202 An additional 120 teachers were enrolled in 2022 and are pending certification. The combined total of Comments: certifications would potentially reach 1,106 by Project closing. It is important to note, however, that accreditation refers to the completion of a one-year course, so teachers accrediting more than 1 course could be counted more than once in the final measurement. The unique number of teachers receiving at least 1 certification will be calculated as part of the Implementation and Completion Results Report to ensure that teachers are not double counted in the final measurement. Primary Education IN01216649 ►IRI7. Number of additional primary level classrooms built or rehabilitated in FTS resulting from Project intervention. (Number, Custom) 2/2/2023 Page 5 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 327.00 356.00 160.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed. The current value represents all classrooms built and rehabilitated within the broader framework of the Project, including those financed with counterpart funds. 197 classrooms were Comments: financed solely with Project funds. IN01216653 ►IRI8. Number of FTS that are equipped with physical and educational equipment. (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 29.00 32.00 28.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed An additional 3 FTS were equipped under the Project in 2022 following the Comments: extension of the Project’s closing date. IN01216657 ►IRI9. Number of FTS that annually receive resources and technical assistance for preventive maintenance. (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 100.00 211.00 211.00 211.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target achieved. 44 FTS received resources in 2021 and 167 FTS received resources in 2022. Comments: IN01216660 ►IRI10. Primary teachers, principals and inspectors accredit in-service training courses. (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target 3,417 teachers, 2400 teachers, 3,417 passed the first of principals and Value Not started principals or inspectors two cycles inspectors passed the certified. first of two cycles Date 30-Dec-2016 07-Dec-2021 11-Jan-2023 30-Dec-2022 End target surpassed. Accreditation per year: 2017-18: 1,296 2019: 852 2020: 778 Comments: 2021: 491 An additional 411 were enrolled in training courses in 2022 and are pending accreditation. Once accredited, the total number of teachers, directors, and inspectors passing the first of two cycles would be 3,828. Transition between Primary and Secondary Education IN01216650 ►IRI11. Number of FTS that receive support targeted for teachers with students at risk in the sixth grade (Number, Custom) 2/2/2023 Page 6 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 24.00 36.00 30.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed. The value represents 9 FTS that received support in 2018, 7 in 2019, 8 in 2021, Comments: and 12 in 2022. IN01216654 ►IRI12. Pilot experience linking FTS with secondary schools. (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Activities began in 8 FTS in 2018 and 1 additional Work carried out in a FTS was incorporated in 10 FTS-EMB pairs Value Not started total of 18 CEA (FTS- 2019. 9 FTS have been fully aligned EMB pairs) included in the plan of activities for 2022. Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed. This values corresponds to work in 8 pairs in 2018, 1 in 2019, and 9 in 2022. Due to the pandemic, the Centros Educativos Asociados pilot interventions were not carried out in 2020 or Comments: 2021. Monitoring and Evaluation System IN01216651 ►IRI 13. Creation of a monitoring system for students' academic trajectories (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Results report of the pilot Results report of the Results report of the Value System not available delivered pilot and its expansion pilot and its expansion Date 31-Dec-2017 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target achieved. The SEA+ evaluation is planned to be continued and has been included for support in the Strengthening Pedagogy and Governance in Uruguayan Public Schools Project (P176105). Under Comments: this Project, the SEA+ evaluation was expanded to 473 schools (105,850 evaluations) in 2022. IN01216655 ►IRI14. Number of census-based standardized student assessments carried out at the ECD or Primary levels. (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 2.00 2.00 2.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target achieved. The two evaluations carried out to meet this target are the EIT-INDI and the LEO. Comments: IN01216658 ►IRI15. First grade repetition rate in quintile 1 and 2 FTS. (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target 2/2/2023 Page 7 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) Value 12.40 10.90 10.90 10.00 Date 31-Dec-2015 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target achieved. Comments: IN01216661 ►IRI 16. Beneficiary Feedback. Share of FTS benefiting from Project interventions that use teacher or parent apps to carry out beneficiary satisfaction surveys (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 65.00 65.00 50.00 Date 30-Dec-2016 09-Jun-2022 11-Jan-2023 30-Dec-2022 End target surpassed. 65% of FTS use teacher or parent apps to assess satisfaction with Project Comments: interventions in the second year of the schools' functioning. IN01216663 ►Teachers recruited or trained (Number, Corporate) Baseline Actual (Previous) Actual (Current) End Target Value 693.00 693.00 693.00 693.00 Date 07-Dec-2021 09-Jun-2022 11-Jan-2023 30-Dec-2022 End targets achieved. The current value represents teachers who have completed in-service (611), Comments: preschool (226), and management (170) trainings as of December 2020. IN01216665 Number of teachers recruited (Number, Corporate Breakdown) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 0.00 Date 21-May-2021 07-Dec-2021 11-Jan-2023 30-Dec-2022 IN01216666 Teachers recruited or trained - Female (RMS requirement) (Number, Corporate Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 0.00 IN01216667 Number of teachers trained (Number, Corporate Breakdown) Baseline Actual (Previous) Actual (Current) End Target Value 693.00 693.00 693.00 693.00 Date 07-Dec-2021 07-Dec-2021 11-Jan-2023 30-Dec-2022 Performance-Based Conditions 2/2/2023 Page 8 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) IN01216668 ►PBC 1 IRI4. ECD assessment results inform first grade teachers (Text, Process, 500,000.00, 100.00%) Baseline Actual (Previous) Actual (Current) June 30, 2021 ECD assessment results ECD assessment results (EIT) integrated into (EIT) integrated into GURI's database and GURI's database and ECD assessment has 50% Value local ECD assessment local ECD assessment -- coverage. developed and approved developed and approved by the Project by the Project Implementing Entity. Implementing Entity. Date -- 09-Jun-2022 11-Jan-2023 -- PBC 1: "ECD assessment results integrated into GURI's database" was achieved and fully disbursed Comments: (US$500,000). IN01216669 ►PBC 2 IRI6. ECD teachers of FTS accredit in-service training courses (Text, Output, 500,000.00, 100.00%) Baseline Actual (Previous) Actual (Current) June 30, 2021 Value Not started 202 certified teachers 202 certified teachers -- Date -- 09-Jun-2022 11-Jan-2023 -- PBC 2: "Guidelines for ECD teaching practices are revised and published by the Project Implementing Entity" Comments: was achieved and fully disbursed (US$500,000). IN01216670 ►PBC 3 IRI2. ECD teachers of FTS use mobile libraries at least once a week (Percentage, Intermediate Outcome, 400,000.00, 100.00%) Baseline Actual (Previous) Actual (Current) June 30, 2021 Value 0.00 65.20 65.20 -- Date -- 09-Jun-2022 11-Jan-2023 -- PBC 3: "60% of ECD teachers (from a baseline of 30 ECD teachers) of FTS which have installed a mobile Comments: library use mobile libraries at least once a week during 10 consecutive months prior to January 1, 2019" was achieved and fully disbursed (US$400,000) IN01216671 ►PBC 4 IRI5. FTS ECD teachers apply protocols for at-risk students (Text, Intermediate Outcome, 400,000.00, 0.00%) Baseline Actual (Previous) Actual (Current) June 30, 2021 2/2/2023 Page 9 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) ECD teaching guidelines 61.7% of FTS ECD and protocols for at-risk Value Not started teachers apply protocols -- students included in the for at-risk students. ECD in-service training Date -- 09-Jun-2022 11-Jan-2023 -- Verification of PBC 4 "60% of ECD teachers of all FTS (operating as of January 1, 2019) apply protocols for at- risk students" (US$400,000) was carried out by the Instituto Nacional de Evaluación Educativa in November Comments: 2022 and the verification report is pending finalization and submission to the Bank (expected in February 2023). Preliminary data indicate that the PBC has been achieved and will be disbursed before the end of the grace period. IN01216672 ►PBC 5 IRI4. ECD assessment results inform first grade teachers (Text, Process, 600,000.00, 0.00%) Baseline Actual (Previous) Actual (Current) June 30, 2021 ECD assessment results ECD assessment results (EIT) integrated into (EIT) integrated into GURI's database and GURI's database and ECD assessment has 50% Value local ECD assessment local ECD assessment -- coverage. developed and approved developed and approved by the Project by the Project Implementing Entity. Implementing Entity. Date -- 09-Jun-2022 11-Jan-2023 -- PBC 5 "60% of first grade teachers of all FTS (operating as of January 1, 2020) have digital access to Comments: students' ECD assessment results" (US$600,000) has been achieved and is pending the Borrower's request for disbursement. IN01216673 ►PBC 6 IRI4. ECD assessment results inform first grade teachers (Text, Output, 600,000.00, 100.00%) Baseline Actual (Previous) Actual (Current) June 30, 2021 ECD assessment results ECD assessment results (EIT) integrated into (EIT) integrated into GURI's database and GURI's database and ECD assessment has 50% Value local ECD assessment local ECD assessment -- coverage. developed and approved developed and approved by the Project by the Project Implementing Entity. Implementing Entity. Date -- 09-Jun-2022 11-Jan-2023 -- PBC 6: "Local ECD assessment is developed and approved by the Project Implementing Entity" (US$600,000) Comments: was achieved and disbursed. Data on Financial Performance Disbursements (by loan) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed % Disbursed 2/2/2023 Page 10 of 11 The World Bank Implementation Status & Results Report Improving the Quality of Initial and Primary Education in Uruguay (P159771) P159771 IBRD-86750 Effective USD 40.00 40.00 0.00 39.00 1.00 98% Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P159771 IBRD-86750 Effective 16-Dec-2016 22-Mar-2017 08-May-2017 31-May-2022 28-Feb-2023 Cumulative Disbursements Restructuring History Level 2 Approved on 03-Dec-2019 ,Level 2 Approved on 11-Apr-2022 Related Project(s) There are no related projects. 2/2/2023 Page 11 of 11