Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00006217 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF0B4486 ON A GRANT IN THE AMOUNT US$6.77 MILLION TO THE REPUBLIC OF NICARAGUA FOR THE NICARAGUA COVID-19 EDUCATION SECTOR RESPONSE PROJECT April 27, 2023 Education Global Practice Latin America And Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective April 27, 2023) Nicaraguan Currency Unit = Cordoba NIO $36.57 = US$1 FISCAL YEAR July 1 - June 30 Regional Vice President: Carlos Felipe Jaramillo Country Director: Michel Kerf Regional Director: Luis Benveniste Practice Manager: Emanuela Di Gropello Task Team Leader: Marcelo Becerra ICR Main Contributor: Suzana Nagele de Campos Abbott ABBREVIATIONS AND ACRONYMS ACE Alliance for Education Quality (Alianza por la Calidad de la Educación) Project E&S Environmental and Social FM Financial Management FY Fiscal Year GLE Local Education Group (Grupo Local de Educación) GoN Government of Nicaragua GPE Global Partnership for Education GRM Grievance Redress Mechanism ICT Information and Communication Technology IP/AD Indigenous Peoples and Afro-descendants IRI Intermediate Results Indicator ISR Implementation Status and Results Report LAYS Learning-Adjusted Years of Schooling M&E Monitoring and Evaluation MINED Ministry of Education (Ministerio de Educación) PDO Project Development Objective RF Results Framework TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 1 A. CONTEXT AT APPRAISAL .........................................................................................................1 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .......................................7 II. OUTCOME ...................................................................................................................... 8 A. RELEVANCE OF PDOs ..............................................................................................................8 B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................8 C. EFFICIENCY ........................................................................................................................... 13 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 14 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 14 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 15 A. KEY FACTORS DURING PREPARATION ................................................................................... 15 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 18 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 18 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 19 C. BANK PERFORMANCE ........................................................................................................... 20 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 21 V. LESSONS AND RECOMMENDATIONS ............................................................................. 21 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 23 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 35 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 37 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 38 ANNEX 5. BORROWER COMMENTS ON THE ICR ................................................................... 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P174677 Nicaragua COVID-19 Education Sector Response Project Country Financing Instrument Nicaragua Investment Project Financing Original EA Category Revised EA Category Organizations Borrower Implementing Agency Republic of Nicaragua Ministry of Education (MINED) Project Development Objective (PDO) Original PDO The Project Development Objective (PDOs) is to support MINED’s COVID-19 response programs conducive to reducing learning losses and developing socioemotional skills in selected public preschools and basic education schools. Page 1 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 6,770,000 6,770,000 6,770,000 TF-B4486 Total 6,770,000 6,770,000 6,770,000 Non-World Bank Financing 0 0 0 Total 0 0 0 Total Project Cost 6,770,000 6,770,000 6,770,000 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 03-Dec-2020 11-Dec-2020 19-Nov-2021 30-Apr-2022 31-Oct-2022 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 18-Apr-2022 6.77 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Highly Satisfactory Satisfactory High RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 21-Mar-2021 Satisfactory Satisfactory .59 02 14-Oct-2021 Satisfactory Satisfactory .59 03 24-Dec-2021 Satisfactory Moderately Satisfactory 6.77 Page 2 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) 04 29-Jun-2022 Satisfactory Satisfactory 6.77 05 29-Oct-2022 Highly Satisfactory Satisfactory 6.77 SECTORS AND THEMES Sectors Major Sector/Sector (%) Education 100 Early Childhood Education 33 Public Administration - Education 1 Primary Education 33 Other Education 33 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 78 ICT 78 ICT Solutions 78 Human Development and Gender 100 Disease Control 100 Pandemic Response 100 Education 100 Access to Education 78 Teachers 78 Education Governance, School-Based 24 Management Standards, Curriculum and Textbooks 78 Education Facilities 100 ADM STAFF Role At Approval At ICR Regional Vice President: Carlos Felipe Jaramillo Carlos Felipe Jaramillo Page 3 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Country Director: Andrea C. Guedes Michel Kerf Director: Luis Benveniste Luis Benveniste Practice Manager: Emanuela Di Gropello Emanuela Di Gropello Enrique O. Alasino Massetti, Task Team Leader(s): Marcelo Becerra Alonso Sanchez Suzana Nagele de Campos ICR Contributing Author: Abbott Page 4 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) . I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. By 2016, Nicaragua had made progress in poverty reduction following a period of Gross Domestic Product (GDP) growth (4.6 percent in 2016), but approximately one-quarter of the population still lived below the national poverty line, with the majority concentrated in rural areas and remote communities with constrained access to basic services . However, health and education services were free, gradually improving the quality of care for the population. The economy contracted by 4.0 and 3.9 percent in 2018 and 2019, respectively in response to the sociopolitical context. This contraction generated significant inequalities, especially in terms of employment opportunities. The unemployment rate was 6.8 percent in 2019, and both the informal economy and unemployment were high among youth. To respond to these challenges, the Government of Nicaragua (GoN) had committed to important reforms to raise productivity and boost shared prosperity, including strategies for reducing poverty and improving education, employment and innovation. 2. At the time of appraisal of the COVID-19 Education Sector Response Project (the Project) in September 2020, the Coronavirus Disease 2019 (COVID-19) pandemic had impacted economic activity in Nicaragua. Following the first confirmed COVID-19 case in March 2020, Nicaragua had recorded 5,725 confirmed cases and 159 deaths. The GoN had not closed schools, but had implemented protocols for preventative measures against the virus, including mandating the use of masks, distancing and following up on student health. The private sector had also adopted protective measures, including introducing flexible working arrangements, cancelling events, and closing stores and services. Consequently, the pandemic had led to job losses, a fall in consumer and business confidence, and a decline in labor-intensive sectors, such as construction, commerce and services, further reducing the progress in poverty reduction since 2005. Growth was expected to contract sharply in 2020, exacerbating the country’s significant inequalities in terms of income, employment opportunities, and access to quality basic social services. 3. The pandemic was expected to increase existing vulnerabilities that stemmed from Nicaragua’s own geography. The country is highly vulnerable to natural disasters, including hurricanes and storm surges in coastal areas, as well as extreme precipitation and earthquakes. Its ability to withstand these risks was still limited, however, leading to a variety of unintended consequences, and putting already vulnerable communities and physical structures at risk, including schools, negatively impacting access to education for children in these communities. 4. At the time of appraisal, Nicaragua was already facing learning challenges that could undermine human capital formation and productivity, particularly for the poor, Indigenous People and Afro- descendants (IP/ADs), girls and young women. Student outcomes on standardized assessments, had improved from 2013 to 2019 in Mathematics but continued to fall short when compared to other countries in the region.1 Learning achievements showed significant statistical differences by gender, even 1Nicaraguan students performed below their regional peers across all subject areas in the 2013 Third Regional Comparative and Explanatory Study (Tercer Estudio Regional Comparativo y Explicativo, TERCE). Page 1 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) after controlling for socioeconomic status.2 5. The quality of the learning environment, as well as teacher’s knowledge and skills, were main contributors to the education system’s overall low performance. A diagnostic study carried out by the Ministry of Education (Ministerio de Educación, MINED) among the country’s eight public teacher training institutes highlighted the following areas for improvement: (i) teachers’ mastery of core subjects; (ii) length of teaching and student mentoring in the classroom; and (iii) teaching of pedagogical and evaluation strategies. Also, limitations in school infrastructure, including schools lacking water and sanitation facilities and digital constraints affected the availability of safe and adequate learning conditions, especially in rural areas. 6. The COVID-19 pandemic was threatening the country’s education outcomes, especially for students from the most vulnerable households, and was expected to widen the learning gaps among students from different socioeconomic groups. Although schools were never closed during the pandemic, public school attendance rate had decreased from 70.1 percent in March 2020 to 45.3 percent in June, as parents opted out of sending their children to school, before recovering to 70.6 percent in September 2020. The pandemic was impacting households and communities differently depending on their socioeconomic level and resilience. Middle- and high-income families whose children attended private schools were better able to cope with challenges posed by the crisis by sustaining learning at home or through remote learning modalities. Vulnerable students coped poorly due to irregular attendance and parental support, as well as a lack of access to digital devices and connectivity. 7. MINED was making strong efforts to cope with the crisis, but greater investment and coordinated interventions were needed to mitigate the pandemic’s impact on learning outcomes of vulnerable students. To sustain short-term education continuity and learning recovery, MINED had developed actions to strengthen and reinforce teaching and learning throughout public schools, including intensive training for primary and secondary level teachers using webinars, videoconferences, massive open online courses and a repository of digital resources hosted on MINED’s portal. MINED was also promoting remote learning through educational television and the use of communication platforms, such as WhatsApp and Messenger. The GoN was preparing a prioritized curriculum, and related materials, that emphasized the main content of core subjects to ensure development of basic literacy and numeracy skills through a blended learning model. 8. Rural primary schools lacked the digital technologies to ensure learning continuity and recovery. A total of 742 secondary educational institutions were (and continue to be) equipped with digital classrooms to enrich the learning experience by diversifying instructional modalities and teacher training processes. However, primary schools in rural areas which some of the most vulnerable students attended, lacked access to adequate equipment and connectivity to benefit from MINED’s technology solutions. MINED planned to design and implement a comprehensive strategy to ensure that multi-grade primary schools in rural areas had access to and made use of digital tools and technological platforms with capacity to ensure accelerated learning. 9. The COVID-19 crisis was also affecting children’s and adolescents’ emotional health and well- being, especially those from the most vulnerable households. Several factors led to increased anxiety 2 Boys generally scored better in Mathematics and Science, while girls outperformed boys in Literature. Page 2 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) and depression: (i) loss of relatives and friends; (ii) economic pressures on many households; and (iii) stress caused by containment measures like physical distancing. Girls and young women often bore a double or triple burden, caring for the elderly and other children, being responsible for housework and being subjected to greater domestic and sexual violence as tensions increase in households triggering aggression and violence against them. International research has shown that increased psychological stress upon children and youth could translate into learning losses threatening their future development. 10. The GoN prepared broad strategies to respond to the challenges posed by the COVID-19 pandemic. MINED’s COVID-19 response was framed within a broader strategy: the National Strategy for Basic and Secondary Education to Respond to the Challenge of COVID-19 (the National Strategy). Also, based on its strong experience in promoting the socioemotional development of vulnerable students and their families, the MINED developed a strategy, “Growing in Values” to respond to COVID-19 by focusing on relevant socioemotional skills, such as copping with stress management. The National Strategy articulated four overlapping stages: (i) Stage I: Preventative education to promote healthcare during face-to-face schooling; (ii) Stage II: Support to remote education with emphasis on equity and inclusion; (iii) Stage III: Mitigation of the impact generated by the pandemic on the return to school; and (iv) Stage IV: Resilience—learn and prepare for future challenges. It was also organized in three results areas that were to be supported by the Project: (i) didactic workbooks; (ii) digital classrooms; and (iii) socioemotional strategies. provide them with tools to identify and address negative shocks. Theory of Change (Results Chain) 11. The Project’s Theory of Change was based on the premise that the delivery of customized materials and technological tools that relied on a prioritized curriculum, coupled with teacher training and the involvement of the broader school community in promoting the students’ well -being, would support teaching and learning practices conducive to reducing learning losses and improve the understanding of and competencies in socioemotional support among school staff and parents to help develop socioemotional skills in the education community, and particularly among vulnerable students. This would, in turn, help mitigate the medium- to long-term impact of the COVID-19 shock on the education system, ultimately reducing the expected loss in human capital due to the pandemic. Given the Project’s short implementation period, its Results Framework (RF) measured mostly outputs/intermediate outcomes, under the expectation that achievement of the planned output and intermediate outcome targets would lead to achievement of longer-term outcomes. The TOC is presented below: Figure 1: Theory of Change PDO: to support the Ministry of Education’s (MINED’s) COVID -19 response program conducive to reducing learning losses and developing socioemotional skills in selected public preschools and basic education schools Outputs Intermediate Outcomes Longer-term outcomes Component 1 Programs to reduce learning losses in public preschools and basic education schools • Didactic workbooks for students in Support of teaching and learning public preschools and basic education practices conducive to reducing schools predominantly serving learning losses in schools vulnerable groups. predominantly serving vulnerable students Page 3 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) • Teacher training on the use of the didactic workbooks and on pedagogical practices to reduce learning losses. Negative impact of COVID-19 on • Distribution of didactic workbooks to education outcomes and well-being of targeted schools predominantly students mitigated through reduction serving vulnerable groups. in learning losses and development of • Mobile Digital Classrooms for selected socioemotional skills. multi-grade primary schools predominantly serving vulnerable students. • Development of an Adaptive Learning Tool. • Adaptive Learning Program pilot. Component 2: Program to develop socioemotional skills in selected public preschools and basic education schools • Socioemotional support program Improved understanding of and developed and implemented among competencies in socioemotional school staff, students, and parents. support among school staff and parents conducive to support students with socioemotional stress and develop socioemotional skills. Assumptions • School staff and students will adapt to using the new technology • The technology will be used as intended • School staff will apply their pedagogical and socioemotional training in the classroom and school environment • Student dropout rate during COVID-19 will be lower as a result of the interventions • MINED will provide the resources, capacity, and motivation to repair/replace any damaged equipment Mitigation Efforts • The Project will build the MINED’s capacity as well as that of pedagogical advisors and school leaders to monitor project activities and provide school staff with support. • The Project interventions will be designed as cost-effectively as possible to ensure its sustainability after the Project closes Project Development Objectives (PDOs) 12. The Project Development Objective was to support the Ministry of Education’s (MINED’s) COVID-19 response program conducive to reducing learning losses and developing socioemotional skills in selected public preschools and basic education schools. Key Expected Outcomes and Outcome Indicators 13. The Key Expected Outcome was to support programs to help tackle two of the most immediate negative impacts of the COVID-19 pandemic: the worsening of education outcomes and the decline in well-being for the most vulnerable. 14. The Outcome Indicators selected to measure progress towards the PDO were as follows: Page 4 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) 15. PDO Indicator 1: Students supported by the Project participating in MINED’s COVID-19 programs conducive to reducing learning losses (Target (number): 420,946),3 disaggregated by gender (Target: 50% female), rural (Target: 55% rural), IP/AD groups (Target: 8% IP/AD) and preschool (Target: 60,500 students). 16. PDO Indicator 2: Students supported by the Project participating in MINED’s COVID-19 programs conducive to developing socioemotional skills (Target (number): 103,751), disaggregated by gender (Target: 50% female), rural (Target: 55% rural), IP/AD groups (Target: 8% IP/AD). Components 17. The Project was designed to support MINED’s National Strategy to mitigate the impact of the COVID-19 crisis on education outcomes and on the well-being of students from selected public preschools and basic education schools predominantly serving vulnerable students.4 It was to support programs conducive to reducing learning losses and develop socioemotional skills, through two components aligned with the National Strategy’s Stage IV. Its interventions and the activities it financed (workbooks, digital tablets for Digital Classrooms, teacher training, etc.) were all supportive of in person (or at most hybrid) delivery of teaching and learning, not distance learning since schools never closed. Workbooks were used at home as well, as an important tool to help preclude students from losing pace in their studies or dropping out altogether. Component 1: Programs to reduce learning losses in public preschools and basic education schools (US$5.15 million total cost) 18. The activities under this Component would focus on schools predominantly serving vulnerable students and would: (i) support the delivery of a prioritized curriculum, and (ii) equip primary schools with electronic devices and digital content to complement the learning experience. To accelerate implementation, the activities supported were to build upon work already underway and already-proven approaches by MINED. Component 1 comprised two subcomponents. 19. Subcomponent 1.1: Supporting the implementation of a prioritized curriculum for public preschools and basic education schools predominantly serving vulnerable students. This subcomponent would build upon MINED’s already prioritized curriculum, development of didactic workbooks, and on the existing teacher training mechanisms. It would support reproducing and distributing the didactic workbooks for students and teachers based on the recently prioritized curriculum and providing training to teachers on delivering the prioritized curriculum using the workbooks. About 495,231 students and 15,800 teachers from around 10,300 public preschools and basic education schools lacking access to connectivity and electronic devices (both at home and at school) would be supported. 20. Subcomponent 1.2: Supporting the use of digital tools in public primary schools predominantly serving vulnerable students. This subcomponent would build upon MINED’s existing strategy of Mobile 3 The target for PDO Indicator 1 in the Project Appraisal Document was 420,946. However, the Project’s Results Framework contains a very similar Intermediate Result Indicator (IRI 11), that reads “Students benefiting from direct interventions to enhance learning” that has a different target: 426,210 students. 4 Basic education comprises first to ninth grade and covers primary education and lower-secondary education in all its modalities. Page 5 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Digital Classrooms,5 allowing it to mitigate risks by implementing faster, more efficiently, and with an approach that would consider implementation lessons. It aimed to increase access to digital educational content for students in selected multi-grade and regular primary schools predominantly serving vulnerable students by: (i) providing targeted schools with educational technologies to help them address the needs of individual students in a more personalized way to reduce learning losses and accelerated longer-term learning, and (ii) providing teachers with the necessary pedagogical training to support students on the use of these new platforms and tools. It would target a total of 80 multi-grade primary schools with no or limited connectivity and no access to electronic devices for an updated Mobile Digital Classrooms strategy, and 15 primary schools (11 regular and four multi-grade) that already had Mobile Digital Classrooms and connectivity. Most of the content developed under this subcomponent would be made available online for all public primary schools, teachers and students. Activities would include: (i) design and development of a system of digital learning platforms and technological tools aligned with the prioritized curricular content and learning objective indicators; (ii) design and implementation of an updated Mobile Digital Classrooms strategy to increase access to digital learning content by vulnerable students in primary rural schools with no or limited connectivity; (iii) implementation of a small pilot of an adaptive learning program through an online platform which customizes content for students based on their current learning level and dynamic performance with the system (in 15 primary schools with technological equipment and connectivity and 10 additional multi-grade primary schools); (iv) designing a training strategy and training of teacher and other relevant personnel on the use of the digital tools to complement and strengthen their instruction; and (v) increasing the capacity of the technological infrastructure at the central and departmental levels to develop and carry out all of the above activities, through the provision of servers and communication networks for the use of digital platforms and resources, as well as complementary videoconferencing equipment to support related teacher training and learning communities. Component 2: Program to develop socioemotional skills in selected public preschools and basic education schools (US$1.52 million total cost) 21. This Component was designed to help mitigate the negative impact of the COVID-19 pandemic on the well-being of the education community in selected schools (selected to be representative of the country’s departments). Component 2 comprised two subcomponents. 22. Subcomponent 2.1: Developing a socioemotional support program for public preschools and basic education schools’ communities. Based on the MINED’s “Growing in Values” socioemotional strategy, this subcomponent would support the development of a program to provide socioemotional support to the education community to cope with the challenges posed by the COVID-19 pandemic. Activities would include: (i) design of a socioemotional support strategy for school staff, students and their families in targeted preschool, primary and secondary schools; and (ii) design of training materials, including course guidelines and manuals for schools directors and teachers on how to promote self-care, how to provide socioemotional support for families, how to develop socioemotional skills in students through school activities, and how to identify risks for girls and young women through MINED’s existing Early Warning System, as well as recommendations for their mitigation. 5 The Mobile Digital Classrooms provide tablets to be used by students in the classrooms. When students are finished using them, the tablets are stored in a suitcase in the classroom. The classrooms are not mobile; the tablets are mobile, but always inside the school’s classrooms. Page 6 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) 23. Subcomponent 2.2: Implementing a socioemotional support program for public preschools and basic education schools’ communities. This subcomponent would support the first implementation phase of the program in about 900 selected public preschools and basic education schools distributed among the 19 departments and/or regions according to enrollment, ensuring the public schools in urban and rural areas were equally included. Activities were to include: (i) delivery of an online socioemotional support course for pedagogical advisors, counselors, directors, deputy directors and teachers; (ii) provision of “self-care spaces” in the form of bi-weekly meetings for participating directors, deputy director and teachers to foster socioemotional skills through practical activities; (iii) provision of monthly sessions for families (complying with biosafety protocols) on how to provide socioemotional support to their children at home, to raise awareness of the socioemotional risks associated with the COVID-19 pandemic and other similar shocks, and how to mitigate and alleviate threats and dangers or girls and young women; and (iv) delivery of sessions for students in dedicated time slots during the school day, guided by teachers, to encourage students’ expression of emotions and feelings through play, art and creativity, and promote their emotional well-being and learning, utilizing materials and recreational kits financed by the Project. Component 3: Project management, monitoring and evaluation (US$0.10 million total cost) 24. This Component would focus on strengthening MINED’s teams working on fiduciary management, social and environmental standards, and project implementation and monitoring, including financing for technical assistance. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 25. Neither the PDOs nor the Outcome Targets were revised. Revised PDO Indicators 26. The PDO Indicators were not revised. Revised Components 27. The Project’s components were not revised. Other Changes 28. The Grant’s Closing Data was extended by six months, from April 30, 2022 to October 30, 2022. Rationale for Changes and Their Implication on the Original Theory of Change 29. The Rationale for the Closing Data extension was to provide additional time required to complete the procurement of project inputs due to supply chain issues in world markets resulting from the pandemic. The extension did not impact the Theory of Change. Page 7 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) II. OUTCOME A. RELEVANCE OF PDOs 30. The Project continues to be aligned with the World Bank Group’s Country Partnership Framework (CPF) for Nicaragua for the period FY2018-2022 discussed by the Board of Executive Directors on March 15, 2018.6 By supporting the MINED in implementing activities during the recovery and resilience stages of the pandemic by remediating and strengthening learning and promoting the well-being of vulnerable students, the Project contributed directly to two of the CPF’s three strategic pillars: Pillar I, Investing in human capital in particular for vulnerable groups, through improved access to quality education and health services and by reducing learning disparities, and Pillar III, Improving institutions for resilience and sustainability, which supports the Government in ensuring sustainability and growth by addressing challenges related to natural hazards, external shocks and institutional fragility. The Project was designed as an emergency response to an unprecedented pandemic, and its components were and continue to be fully aligned with the GoN’s National Strategy for Basic and Secondary Education to Respond to the Challenge of COVID-19 (Section IA). Given Nicaragua’s susceptibility to adverse disasters, the programs developed by the Project, including those aimed at reducing learning losses and providing socioemotional support to students impacted by adverse events will continue to be relevant in the post-pandemic environment. The Government aims to sustain and expand upon project-financed activities to institutionalize them as a mechanism of disaster response in the education sector. Assessment of Relevance of PDOs and Rating 31. Relevance is rated High, in view of the Project’s full alignment with the Bank’s CPF at closure. B. ACHIEVEMENT OF PDOs (EFFICACY) 32. For purposes of evaluating achievement of the Project’s PDO (to support the Ministry of Education’s (MINED’s) COVID-19 response program conducive to reducing learning losses and developing socioemotional skills in selected public preschools and basic education schools), and as implied in the project’s design and PDO indicators, the word “conducive” means that its activities and investments would support basic inputs and outputs required to, over the longer-term, reduce learning loses and develop socioemotional skills. In other words, the Project’s very short-term emergency nature and the actual priority activities and investments it financed were designed to support programs, directly in response to the pandemic, with characteristics that would help reduce learning losses and increase socioemotional skills in the longer-term, and should not be seen in the broader context of a long-term sectoral program that could produce marked improvements in learning and socioemotional skills as a short-term impact of the project. Assessment of Achievement of Each Objective/Outcome Support the MINED’s COVID-19 response program conducive to reducing learning losses in selected public preschools and basic education schools. PDO Indicator 1: Students supported by the project participating in MINED’s COVID-19 programs 6 Report No. 123026-NI. Page 8 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) conducive to reducing learning losses (Target (number): 7 420,946), disaggregated by gender (Target: 50% female), rural (Target: 55% rural), IP/AD groups (Target: 8% IP/AD) and preschool (Target: 60,500 students). 33. The achievement of this objective is rated High. The Project achieved its objective of supporting the MINED’s COVID-19 response program conducive to reducing learning losses in public preschools and basic education schools, as reflected by the overachievement of targets for the PDO Indicators defined to measure progress. It achieved this objective through implementation of activities under Component 1 that were designed to support teaching and learning practices conducive to reducing learning losses in schools predominantly serving vulnerable students. These included the implementation of a prioritized curriculum, and supporting materials, and the use of digital tools, as described below. The Project surpassed the targets for PDO Indicator 1 due to the reduced unit costs in the purchase of workbooks. The Project’s activities supported a total of 822,053 students8 in programs to reduce learning losses;9 of these, 50 percent were female, 94 percent resided in rural areas (higher than the 55% planned, since finally mostly rural multi-grade schools were selected), and 9 percent self-identified as IP/AD. A total of 83,522 preschool students also participated in programs to reduce learning losses. 34. Prioritized Curriculum. The Project supported the design and distribution of 64 different student and teacher didactic workbooks based on prioritized curricular content (Mathematics, Natural Sciences, Social Sciences and Language and Literature) and learning objective indicators, that were aimed at reducing learning losses for students lacking access to connectivity and electronic devices both at home and at school, and to guide their teachers.10 Workbooks were designed for preschool, primary and secondary education, and for regular, special education, multi-grade and distance education modalities, for the following subjects: (i) Mathematics; (ii) Natural Sciences; (iii) Social Sciences; and (iv) Language and Literature. Of these, 15 student and teacher workbooks for preschool and primary students were 7 Key activities included: (i) Training for socioemotional support course for pedagogical advisors, counselors, directors, deputy directors, and teachers; (ii) Provision of “self-care spaces” in the form of bi-weekly meetings for participating directors, deputy directors and teachers to foster socioemotional skills through practical activities, (iii) Ludic kits, including hygiene products (e.g. liquid soap and hand sanitizer) and fungible play materials (e.g. scissors, glue, markers, toys) for students; (iv) Provision of bi- monthly sessions for families (complying with biosafety protocols) on how to provide socioemotional support to their children at home, to raise awareness of socioemotional risks associated with the COVID-19 pandemic and other similar shocks, and how to mitigate and alleviate threats and dangers for girls and young women. 8 882,053 students were beneficiaries of the didactic workbooks (64 different didactic booklets in Mathematics, Natural Sciences, Social Sciences and Language and Literature and a document of didactic guidelines for teachers distributed nationwide). Out of those, 6,052 students benefited as well from mobile digital classrooms in the 80 selected schools, and 2,215 students from the adaptive learning platform in 25 selected schools. 9 Of these 822,053 students, 560,414 (68 percent) were at the primary level, 174,672 (22 percent) at the secondary level and 83,522 (10 percent) in pre-primary education. In terms of the Project’s scope, its beneficiaries reached 62 percent of the total primary level enrollment (2022); 32 percent of the total secondary enrollment including youth and adults (2022); and 33 percent of the total preprimary enrollment. 10 Several studies show that didactic workbooks and other interventions are highly instrumental in making up for loss of classroom time. OECD’s Overcoming School Failure: Policies that Work (2010) concludes that distribution of free textbooks is a support for learners especially those who do not have access to them. Several WB studies, for instance in Pakistan (2012) proved that free textbook distribution helped increase retention rates, decrease dropouts, raise enrollment, improve daily attendance, increase passing rates of learners and enhance the quality of education. A recent diagnostic on the impact of the pandemic in Latin America (The World Bank’s Two Years After (2022) report) provides several other examples of the positive impact of materials on student retention in the region. Page 9 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) contextualized and translated into six languages.11 A total of 10,382 schools (Intermediate Results Indicator/IRI 1, target: 8,759 schools) benefitted from 2,577,660 copies of the didactic materials distributed. Of these, 26 schools were Special Education Schools (IRI 2, target: 25 schools); 2,938 children with disabilities benefited from the workbooks. Training on the use and management of didactic workbooks was delivered to 91 pedagogical advisors and school directors at the regional and municipal levels, followed by training of 2,561 technical coordinators and pedagogical advisors for various modalities (Special Education, Early Childhood Education, Regular and Multi-Grade Primary, etc.), and then, in a third phase for 14,000 teachers of different modalities at the national level (IRI 3, target: 13,915 teachers). These teachers were all provided with workbooks. A learning diagnostic assessment was applied to 421,200 students at the national level for different subjects and educational modalities (IRI 4, target: 420,946 students)12. Its objective was to analyze the content and results achieved by students in the first semester of 2020 as input to design a strategy to provide continuity in the second semester of 2020 and the following school year. Its results and recommendations were summarized in a report, Diagnóstico Sobre Aprendizajes Alcanzados (Diagnostic of Results Achieved).13 Moreover, a survey was applied to schools’ directors, teachers and parents in 30 educational centers to assess the effectiveness and level of satisfaction of beneficiaries with the didactic booklets program.14 The results were highly encouraging confirming its usefulness to support the teaching and learning process and therefore, eventually, reduce learning losses. More than 90 percent of schools’ principals and teachers considered that the workbooks: (i) facilitated and helped maintain the pace of learning for those students that assisted classes irregularly during the pandemic; and (ii) provided key pedagogical support to help teachers maintain students’ attention, ensuring continuity, while helping them to present the contents in an attractive and pedagogically-sound fashion. Similarly, a vast majority of parents reported finding the orientations given by teachers for the use of the workbooks very appropriate and helpful, especially for those students who could attend school regularly. 35. Use of Digital Tools. Building upon the MINED’s Mobile Digital Classrooms strategy, the Project designed digital learning platforms and tools aligned with the prioritized curriculum15 and learning 11 Miskitu, Tuahka, Ppanamahka, Ulwa, English and Creole. 12 The subjects included: Language, Mathematics, Social Sciences, Natural Sciences, Biology, English, Physics and Chemistry. The modalities included: regular primary, regular secondary, multi-grade primary, distance learning primary, distance learning secondary, and adult secondary. 13 Ministerio de Educación, Diagnostico Sobre Aprendizajes Alcanzados, October, 2020. 14 The methodology and results of the survey were compiled in a special report “ Informe de Resultados de la Encuesta de Satisfacción de Actividades Realizadas en la Implementación del acompañamiento Socioemocional y el Uso de Cuadernillos didácticos” (MINED, 2023). The survey instrument was structured as follows: (i) the survey of directors was structured with six items, whose purpose was to evaluate three variables--assessment of the usefulness of the booklets to assist students in any emergency situation and perception of their implementation to guarantee educational continuity, reinforcement, consolidation and leveling of student learning assessment of their usefulness to carry out a formative evaluation; (ii) the teacher survey was structured with eight items, whose purpose was to evaluate three variables--assessment of the usefulness of the booklets to assist students in any emergency situation and perception of their implementation to guarantee educational continuity, reinforcement, consolidation and leveling of student learning assessment of their usefulness to carry out a formative evaluation; (iii) The parents survey (mothers, fathers or guardians) was structured with five items, whose purpose was to evaluate two variables: assessment of the usefulness of the booklets as support material for the accompaniment of the learning of their children at home and perception of their implementation to guarantee educational continuity, reinforcement, consolidation and leveling of learning. 15 McEwan (2015) in “Improving Learning in Primary Schools of Developing Countries: A Meta -Analysis of Randomized Experiments,” documents consistently positive effects of different interventions on learning outcomes. The largest mean effect sizes included treatments with computers or instructional technology (0.15). Page 10 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) objectives and expanded access to digital educational content (IRI 5, target: Yes) that now benefit 6,052 students in 80 multi-grade primary schools, of which 7 schools include students that identify as IP/AD (IRI 4, target: 68 schools). The 80 schools were selected out of a universe of 7,322 multi-grade primary schools based on criteria including: (i) availability of electricity; (ii) accessibility for delivery of equipment; (iii) representative student enrollment; (iv) total student enrollment; and (v) basic structure of school and security. The physical spaces of the 80 schools were prepared to store the Digital Mobile Classrooms and each was equipped with: (i) 20 tablets; (ii) a laser projector; (iii) a printer; (iv) a router; and (v) two laptop computers. The physical spaces all provide safekeeping of equipment, including adequate electric and digital connections, and materials to protect against surges in electrical currents. Some of the schools were provided linkage to Access Point, to ensure connectivity. Awareness-raising workshops on the use, maintenance and protection of technology equipment were carried out in each of the 80 schools, with participation of 203 teachers, 438 students, 813 parents, 22 community leaders, 46 directors and 16 pedagogical advisors (of which 186 identified as IP/AD). The Project developed and implemented an online adaptive learning program, that corresponds to the curriculum of regular and multi-grade primary education for Mathematics, in 10 regular primary and 15 multi-grade primary schools. Training on the use and management of the adaptive learning platform was provided to 2,125 students and 89 teachers in these 25 Educational Centers. The platform customizes content to students’ knowledge and performance, allowing them to catch up with the curriculum content. This platform is aligned with other adaptive learning programs in the region which have shown to have significantly improved learning.16. By completion, 2,214 students had used the adaptive learning platform 9,379 times (IRI 6, target: 102,928 times). However, based on more recent usage and estimates, MINED expects that an additional 63,002 students will have used the platform, bringing the total usage to almost 72,381 accesses by August 2023, expecting the final target (102, 928) would be achieved by the end of 2023. Support the MINED’s COVID-19 response program conducive to developing socioemotional skills in selected public preschools and basic education schools. PDO Indicator 2: Students supported by the project participating in MINED’s COVID-19 programs conducive to developing socioemotional skills (Target (number): 103,751), disaggregated by gender (Target: 50% female), rural (Target: 55% rural), IP/AD groups (Target: 8% IP/AD). 36. The achievement of this objective is rated High. The Project achieved its objective of supporting the MINED’s COVID-19 response program conducive to developing socioemotional skills in public preschools and basic education schools, as reflected by the overachievement of targets for the PDO Indicators defined to measure progress. It achieved this objective by developing and implementing a socioemotional support program for public preschools’ and basic education schools’ communities to provide them with tools to understand and address negative shocks, as described below. The Project surpassed the targets for PDO Indicator 2. The Project’s activities17 supported a total of 133,351 students 16 For instance, an EdTECH pilot based on adaptive technology in Dominican Republic to improve the math skills of third year secondary school students has shown students achieve 7% more of a grade curricular unit of study (based on grade standards). 17 Key activities included: (i) Training for socioemotional support course for pedagogical advisors, counselors, directors, deputy directors, and teachers; (ii) Provision of “self-care spaces” in the form of bi-weekly meetings for participating directors, deputy directors and teachers to foster socioemotional skills through practical activities, (iii) Ludic kits, including hygiene products (e.g. Page 11 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) in programs conducive to developing socioemotional skills; of these, 69,551 (52 percent) were female, 72,713 resided in rural areas (55 percent) and 10,137 (8 percent) self-identified as IP/AD. 37. Developing and Implementing a Socioemotional Support Program. The Project provided technical assistance for the development of a Socioemotional Support Strategy and related materials. MINED developed an online socioemotional course to support students by helping them, their parents and teachers identify and address the negative impacts resulting from adverse events, such as the pandemic. The socio-emotional support materials contain a chapter called “Approaches to the Strategy”, with one of them being the gender approach and how gender is transversal to all of its contents (IRI 12, target: yes). Training in the application of the strategy and use of materials (ludic kits and triptych for families18) was provided to 3,910 School Directors and Deputy Directors, and teachers (IRI 8, target 3,220 staff) of 900 educational centers nationwide (IRI 7, target: 765 centers) that are now implementing the socioemotional support strategies aimed at students and teachers, and awareness raising programs were delivered to students’ parents through bimonthly meetings to also help them identify and address socioemotional impacts, including how to accompany their children in the learning process, contributing to school attendance and permanence, shared responsibility, comprehensive sexuality education, in a framework of respect of human rights, gender equity, inclusiveness, interculturality, environment and promotion of values. A total of 57,739 parents (41,562 women and 16,234 men) participated in socio- emotional activities, of which 36,386 received triptychs and provided feedback on support materials (IRI 11, target: 2,700 parents). A survey was also applied to assess the effectiveness of and level of satisfaction with the socioemotional support program from the program’s beneficiaries: schools‘ directors, teachers and parents in 30 educational centers.19 The survey’s results show a high level of fulfillment of the program’s objectives : 65 percent of school principals and teachers found the program “excellent” and a 23 percent found it “very good,” pointing to its conduciveness to developing socioemotional skills. They valued especially the training and orientation received to implement the strategy in a pandemic context, the usefulness of the kit developed to deliver the program “I feel good expressing myself,” the “self-care” liquid soap and hand sanitizer) and fungible play materials (e.g. scissors, glue, markers, toys) for students; (iv) Provision of bi- monthly sessions for families (complying with biosafety protocols) on how to provide socioemotional support to their children at home, to raise awareness of socioemotional risks associated with the COVID-19 pandemic and other similar shocks, and how to mitigate and alleviate threats and dangers for girls and young women. 18 The "Classroom with joy" kit contains: (i) support documents “Guide for the use and handling of the Play Kit ;” (ii) material to promote hygiene habits and prevent COVID 19 and other diseases, such as: alcohol and liquid soap; (iii) consumables for art, through crafts, drawing, painting, writing, such as: Liquid silicone, Scissors, Foami, Acrylic Markers, Permanent Markers, Crayons, Watercolors, Plastiline, Ream of bond paper, Glue; (iv) traditional ludic material of Nicaraguan context: Ula – Ula, Jakets, Pelotas. The Ludic Kit is oriented to the development of activities that are carried out through games where children and adolescents can move and express themselves freely, allowing them to explore, create, have fun, share, learn, strengthen affective bonds and practice values for harmonious coexistence. These spaces are just as important as toys, and they should be safe and comfortable. Triptychs for families are a consultation tool for mothers, fathers and guardians who attend the sensibilization meetings, they contain guidelines so that they can strengthen their knowledge and skills to know, manage and express their emotions, as well as those of their children who are experiencing certain situations, including COVID 19. The triptychs are delivered in the bimonthly meetings that take place in the educational centers. 19 The methodology and results of this survey were also part of the report “Informe de Resultados de la Encuesta de Satisfacción de Actividades Realizadas en la Implementación del acompañamiento Socioemocional y el Uso de Cuadernillos didácticos” (MINED, 2023). The survey instrument was structured as follows: (i) the survey of teachers and directors was structured with 17 items, whose purpose was to evaluate three variables: assessment of the course, appreciation of the self-care meeting, and perception of the usefulness of the implementation of self-care spaces with students, (ii) the survey of parents was structured with 12 items, whose purpose was to assess the usefulness of the knowledge shared in the meetings with mothers, fathers and guardians, for their daily life and coexistence in their home. Page 12 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) workshops20 carried out and the ludic spaces. The vast majority of the parents reported that the program helped to improve communication within the family and helped their children express feelings and sentiments more freely as well as to cope adequately, in what was a difficult situation. Parents also valued the triptych “In my family we protect and care ourselves.” Justification of Overall Efficacy Rating 38. Overall efficacy is rated High. The Project achieved its objectives, exceeding the targets for its PDO and IRIs. C. EFFICIENCY Assessment of Efficiency and Rating 39. Economic Analysis. The Project’s economic efficiency is High. The Project’s ex-post economic analysis was prepared following the same methodology used at Appraisal, a Cost-Benefit Analysis. The approach is based on the monetary benefits derived from the decrease of learning losses, the prevention of a rise in dropouts, and the increase in quality of education. Specifically, the Project mitigates learning losses by enabling access to education for children who otherwise might risk not attending school disrupting their learning. Without the strengthening of education delivery, students would not be able to complete the academic year, which could result in increased dropout rates. The Project prevented this by ensuring an appropriate delivery of workbooks to all primary students in targeted schools, who thus benefited from uninterrupted education throughout the year. Lastly, the addition of technology to teaching practices, the digital tools in targeted schools, the adaptive learning pilot, and the socioemotional support offered to teachers, students and their families, all increased the quality of education received by the students in the short and medium term. As a result, the updated cost-benefit analysis estimates that the activities supported by the Project have a cost-benefit ratio of 17.49, with an internal rate of return of 11 percent, higher than the original estimates of 11.8 and 8.80 percent, respectively. The better results are due to the higher number of beneficiaries resulting from project implementation: student and teachers’ workbooks, students using digital mobile classrooms, and receiving socioemotional support. This estimation is considered more realistic than the Appraisal estimate since it uses more accurate data about the number of project beneficiaries. The full economic analysis is provided in Annex 3. Finally, official MINED data reveal that overall enrollments were not affected much by the COVID 19 pandemic at primary level and secondary levels: at primary level, enrollments were just around 1 percent lower in 2021 and 2022 than in 2019 (pre-crisis), while at the secondary level enrollments were 3 percent higher in 2022 than in 2019. This may suggest that project’s interventions were helpful to tackle disruptions in attendance while also stabilizing enrollment.21 40. Implementation Efficiency. The Project’s implementation efficiency was High. Despite a short six- month Closing Date extension, the Project was fully and efficiently implemented in 18 months. Delays were due to several exogenous factors: (i) volatility in world markets for goods and services, including 20 MINED staff organized “Personal Growth” meetings, developed for directors and teachers, whose the purpose was to generate spaces for self-care. These wworkshops were not financed by the Project (but by MINED) but they were part of Project activities. 21 These numbers are provided for background only since, in the absence of a counterfactual, it is not possible to precisely evaluate the Project’s impact on enrollments during the pandemic, especially in view of the Government’s policy of keeping schools open. Page 13 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) important delays in maritime transportation (affecting imported goods), reduced supply and increased demand, that affected availability, and delivery dates of project inputs; (ii) two hurricanes that impacted Nicaragua’s Caribbean coast; and (iii) staff absences following their contracting of COVID-19. However, even in this context, by the original Closing date (April 30, 2022), the Project had already disbursed and contracted 100 percent of the funds, with most contracts executed well before the final Closing Date (October 30, 2022). More importantly, the Project was able to exceed planned targets for most PDO and IRIs also thanks to efficiency in procurement—the Project delivered more within the allocated budget. Cost savings were all utilized to expand the Project’s outputs and expected intermediate outcomes. Cost savings were incurred since the cost of workbooks had been overestimated in view of the increased international costs resulting from pandemic supply issues that in the end did not materialize since the unit cost of offers received were below estimates. MINED had expected to acquire about 1.8 million workbooks, but was able to acquire almost 2.6 million as a result of savings. Several positive factors during preparation and implementation impacted the Project’s implementation efficiency and outcome as described in Sections III A and B. The final allocation of proceeds and costs is provided in Table 1. Table 1: Final Project Costs Allocated Disbursed Category Category Description US$ US$ Goods, Civil Works, Non-Consulting 1 Services, Consultancy, Training, 6,770,000 6,770,000 Operational Cost Assessment of Efficiency and Rating 41. Efficiency is rated High, in view of the Project’s economic and implementation efficiency, both of which exceeded expectations. D. JUSTIFICATION OF OVERALL OUTCOME RATING 42. The Project’s Overall Outcome is rated Highly Satisfactory in view of its High Relevance, High Efficacy and High Efficiency ratings. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 43. Although the Project did not target gender specifically, its gender achievements (Section II B) are expected to have had a positive impact on women and girls as they are the most likely to be affected by the impacts of the pandemic. First, females are more likely to be charged with care for the elderly or other children. Then, they are subject to greater domestic and sexual violence as household tensions increase due to pandemic-related stresses. Finally, the pandemic was expected to impact the already high fertility rates, especially among indigenous women, and continued learning supported by the Project was expected to mitigate this. The socioemotional strategy and materials developed and delivered under the Project include a gender perspective, and learning modules for directors, teachers and parents include information about gender-related risks and mitigation measures. Page 14 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Institutional Strengthening 44. The Project did not have institutional strengthening as an objective. Nevertheless, the development and implementation of digital learning platforms, alternative learning programs and socioemotional programs, along with training of school directors and teachers, and the activities coordinated by the MINED’s various units, has built capacity for implementing these programs in the event of a natural or other disaster. Continued involvement with implementation of World Bank-financed projects has strengthened MINED’s capacity over the years, and implementation of this Project expanded this experience. Representatives of the ministry stated that they are fully comfortable with implementing Projects following both Government and World Bank requirements. Mobilizing Private Sector Financing 45. N/A Poverty Reduction and Shared Prosperity 46. Although not specifically stated in the PDO, the Project focused on vulnerable students, mostly those in multi-grade schools in rural areas and by those that identify as IP/AD. Several key indicators monitored these project beneficiaries. Poverty in Nicaragua continues to have a spatial dimension with the Central and Caribbean regions lagging other regions and half of Nicaragua’s poor living in the Central region (that is mostly rural). The Caribbean region contains a sizable proportion of indigenous peoples and Afro‐descendants, which according to the latest 2005 Census represent six percent of the population. The census identified a large gap between non‐indigenous and indigenous Nicaraguans in terms of basic service provision. The Project’s achievements in terms of reaching beneficiaries in both of these groups— those vulnerable students both in rural areas and those self-identifying as IP/AD exceeded expectations. Other Unintended Outcomes and Impacts 47. N/A III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 48. The Project was prepared in response to the Government’s submission of its Application and Program Document for COVID-19 Accelerated Funding to the Global Partnership for Education (GPE). It was prepared following the GPE Guidelines, and the World Bank was designated by GPE as Grant Agent. Several factors during preparation impacted the Project’s implementation and outcome. These included: (i) Government commitment; (ii) long-standing World Bank support to Nicaragua’s education sector; (iii) an ongoing, well-performing portfolio; (iv) strong sectoral coordination with donors and affected groups; (v) its design as an emergency operation, focused on vulnerable groups in rural multi-grade schools; and (vi) adequate identification of risks. 49. The Government’s commitment to the PDO was strong; in fact, the PDO was defined around the Government’s National Strategy. The National Strategy was supported by national resources and by ongoing projects financed by international organizations and donors, coordinated under the framework of a Local Education Group (Grupo Local de Educación, LGE) that supported preparation of the Strategy. The international and national organizations under the LGE also contributed actively to the COVID-19 Page 15 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) response and to the Project’s design, following GPE guidelines. 50. The World Bank has a long history of support to Nicaragua’s education sector . As a part of that support, the Nicaragua education portfolio comprised well-performing operations, including the then ongoing Alliance for Quality Education Project (ACE, P161029), for which additional financing under a GPE Education Support Implementation Grant was under preparation (P167443, since approved in April 2021), and two operations that had recently closed: Second Support to the Education Sector Project PASEN II (P126357) and the Education Sector Strategy Support Project (133557). To support emergency preparation under condensed procedures, the Project utilized and built upon the institutional arrangements and instruments, including environmental and social instruments, that were developed and under implementation for the ACE Project. Its design also incorporated lessons of experience from previous and ongoing World Bank-supported projects. These included the importance of the Project: (i) building upon national strategies and institutional mechanisms to ensure effective implementation; (ii) focusing on a small number of activities with relevant results; (iii) requesting providers to distribute materials to Municipal Delegations and coordinating with local actors to facilitate the distribution to schools as part of the shared responsibility model promoted by the Government, or by using other strategies that have been supported and improved by MINED under ACE. The Project was also based upon international experience, especially in the use of socioemotional support tools to respond to crises. 51. The Project was designed as an emergency operation, and focused (correctly) on a small number of priority activities that could be implemented in a short period. The activities included those that: (i) could be rolled out to all education centers; (ii) were targeted at vulnerable groups in rural areas (mostly multigrade schools) with limited or no connectivity and no access to electronic devices; and (iii) would be piloted in schools with existing internet connections. It also ensured coordination across all activities that involved school directors, teachers, students and parents. 52. The risk analysis at appraisal correctly identified the Project’s risk as Substantial, and, to the extent possible, incorporated mitigation measures in its design. Extensive consultations in the framework of the Local Education Group (Grupo Local de Educación, GLE) during preparation, close coordination with MINED technical staff and reliance upon existing institutional arrangements for ongoing World Bank-financed project, coupled with close implementation support were expected to mitigate some of the key risks. Its focus on vulnerable student populations, together with complementarity with support provided by other donors was expected to ensure the sustainability and priority of project- supported investments in the event of an economic downturn. B. KEY FACTORS DURING IMPLEMENTATION 53. The design factors described above impacted the Project’s implementation positively . Nevertheless, there were several factors that impacted implementation both positively and negatively: (i) delays in start of implementation; (ii) supply chain issues; (iii) experienced staff in MINED; and (iv) procurement procedures. The first three of these are described below; challenges with procurement procedures are described in Section IV B. The impact of these challenges was addressed during the Mid- Term Review in November 19-25, 2021, whereupon a six-month Closing Date extension was agreed to allow for completion of remaining activities. Page 16 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) 54. Although the financing agreement became effective shortly after signing (December 11, 2020) there was a short delay in the start of implementation. This delay was due to delays in adopting the Project’s Operations Manual that was only finalized in April 2021 (this was to have been prepared no later than 30 days after the Effectiveness Date) and the Procurement Plan was only approved in May 2021. While several processes suffered delays, there were synergies with the ACE Project, and several activities were implemented during this period including: (i) conducting the final review of didactic workbooks, and (ii) reaching the final stage of procurement for the Mobile Digital Classrooms and for the carrying out of the Socioemotional Strategy framework. 55. The Project was implemented by the regular staff of MINED, supported by a Project Implementation Unit with vast experience in supporting World Bank-financed education projects. The continuity and stability in staffing at MINED—many staff have been employed at the ministry for over 20 years and most have been promoted through the system--was a positive factor in the Project’s efficient implementation. Most staff have experience with implementation of eight World Bank-financed operations approved since 2005, and this continuous experience has led to a virtuous cycle of implementing, learning, adjusting, etc. 56. Nevertheless, several project activities were complex, and required high quality technical assistance. As an example, the Mobile Digital Classrooms and Adaptive Learning Platform required the involvement of several specialized experts in the preparation and review of documents and plans. This review process required more time than had been planned during preparation and led to the delays in the target dates for some processes, particularly the drafting and revision of technical specifications. 57. The COVID-19 pandemic, and its impact on the world economy, also delayed implementation. First, in 2021, several staff of the MINED contracted COVID-19 with consequent staff absences while they recovered before returning to work (in the case of the Project Management Unit). Then, the pandemic led to volatility in the supply of goods and services that affected procurement of inputs required for the Project. This resulted in, for example, differences in estimated prices and offers received and difficulties in establishing precise dates for purchase and delivery of goods. It also delayed the delivery of goods. 58. Nicaragua’s Caribbean Coast was affected by Hurricanes Eta and Iota in November 2020. Both of these natural disasters affected implementation of project activities, such as the pre-investment studies for the small infrastructure investments for schools scheduled to receive Mobile Digital Classrooms. 59. The Project’s implementation was advancing well by the time of the Mid-Term Review (MTR) in November 2021, but earlier delays led to a proposal for the Closing Date extension. The MTR reviewed carefully the activities remaining, and procurement processes associated with those activities. Delays related mostly to the production and distribution of didactic workbooks, provision of equipment for Mobile Digital Classrooms, design of the Socioemotional Strategy, and purchase of recreational kits. As of March 2022, all contracts had been procured, but the six-month Closing Date extension was approved to allow for a completion of activities and achievement of the PDO. Page 17 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 60. The Project’s TOC and associated RF was simple, clear, concise and well-designed for an emergency operation (Annex 1). The PDO indicators were output indicators, and this is reasonable given that the Project was to be implemented in 18 months so it would have been impossible to define, monitor and measure and especially achieve improvement in learning outcomes during such a short period. Impacting learning outcomes positively requires many different interventions (beyond those financed by the Project) to converge over a period longer than that of the Project’s implementation. In any event, measuring improvements in learning outcomes would have required a proper baseline that was not available given the emergency nature of the preparation process. The experienced Project Support Team in MINED, responsible for monitoring and evaluation of ongoing projects was to be responsible for day- to-day project monitoring. MINED’s General Directorate of Coordination of Programs and Projects was to be responsible for collecting administrative data, monitoring results, and assessing progress towards the PDO. 61. The Project utilized the Geo-Enabling Initiative for Monitoring and Supervision (GEMS) method that enables project teams to use open-source tools for in-field collection of structured digital data that automatically feeds into a centralized M&E system. The integrated data can include any kind of indicators, based on tailor-made forms; photos, audio, videos; time and date stamps; and GPS coordinates that allow for automated geo-mapping of the information. Using GEMS systematically allows operations to enhance the transparency and accuracy of M&E, and the accountability of third-party monitoring. Moreover, it provides a platform for remote supervision, real-time monitoring of risks during implementation & safeguards monitoring, and portfolio mapping for coordination across projects and partners. M&E Implementation 62. M&E was implemented as planned, with progress reports being presented semiannually as per the Legal Agreement and the Project’s Operation Manual, describing progress as well as the definition of the next steps during each period. 63. Minor works to accommodate the suitcases and equipment for the Mobile Digital Classrooms were monitored as follows. Works in the first 25 schools followed normal supervision procedures in view of the rapid implementation process. The remaining 55 multi-grade schools were accompanied by the GEMS method based upon standard M&E forms and two field visits; the information containing details on both construction and electrical work are archived in a central KoBo Toolbox. M&E Utilization 64. M&E was utilized to track the Project’s progress, keep implementation on track, and address possible delays. Justification of Overall Rating of Quality of M&E 65. The Quality of M&E is considered High, especially since it was appropriately designed for an Page 18 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) emergency project with a short duration, relied upon existing implementation capacity, and was used to routinely track and inform project progress. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social 66. The Project’s combined Environmental and Social (E&S) risk rating was considered Substantial, in view of: (i) work with multiple and diverse indigenous groups dispersed nationally; (ii) implementation challenges in remote and difficult to access areas; (iii) the transition in to the World Bank’s Environmental and Social Framework (ESF) and MINED’s lack of experience since it was following the new framework for the first time; and (iv) an unstable sociopolitical context exacerbated by the challenges posed by the COVID-19 pandemic. 67. The Project adopted a two-phased approach to preparation of E&S instruments. First, MINED developed, consulted (with the GLE and through virtual meetings with IP/AD representatives and organizations working with persons with disabilities) and disclosed a Stakeholder Engagement Plan (SEP), including a project-level Grievance Redress Mechanism (GRM) and an Environmental and Social Commitment Plan. Results of the consultations were documented in the SEP and integrated into the Project’s design. Second, and in line with the preparation’s condensed procedures and to ensure consistency with relevant environmental and social standards (ESS) – namely, ESS1, ESS2, ESS3, ESS4, ESS7 and ESS10 -, by effectiveness, MINED had updated, consulted and disclosed: (i) ACE’s Environmental and Social Management Framework, including labor management measures with a dedicated GRM for project workers; (ii) ACE’s Indigenous Peoples and Afro-descendants Plan; and (iii) ACE’s Indigenous Peoples and Afro-descendants Planning Framework. 68. During implementation, compliance with E&S requirements was consistently rated Satisfactory in the Project’s Implementation Status and Results Reports (ISRs). Implementation of the ESF was supported by two social and two environmental officers contracted to work with the MINED’s staff responsible of social and environmental compliance under World Bank-financed projects. MINED developed and consulted with each community and disclosed “environmental and social diagnostics”, including mitigation measures, Code of Conduct and GRM description for each of the 80 schools that were supported with digital classrooms; these diagnostics were included in bidding documents. MINED continued to carry out consultation and monitoring visits during implementation with authorization of local authorities and recorded the COVID-19 prevention measures that were taken during those engagements. The main activities implemented were: (i) workshops in Managua, Bilwi, Bluefields, Nueva Guinea and Siuna to contextualize didactic workbooks with the participation of IP/AD teachers and a closing meeting of the contextualization process with the Regional Secretariats of Education of the Regional Autonomous Governments and teachers of students identifying as IP/AD in the country’s Caribbean region; (ii) the translation of the booklets into indigenous languages; and (iii) monitoring of contractors to ensure they complied with the provisions of environmental, social and labor management measures and code of conduct for workers. The main results of the Indigenous Peoples Plan’s implementation were: (i) adaptation of learning methods to indigenous cosmology, indigenous languages and culture; (ii) the translation of didactic workbooks into indigenous languages; and (iii) the adaptation of learning assessment in bilingual intercultural education environments. The Project’s GRM (which is utilized by other World Bank-financed education projects), is widely disseminated, and formats have been Page 19 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) shared with departmental and municipal delegates and directors of the 80 project schools. A total of two complaints and three concerns were received with respect to issues of waste management and paint spills and concerns regarding the progress of works; all were addressed and resolved satisfactorily and there are no outstanding complaints. Complaints within the scope of the Labor Management Procedures and Codes of Conduct were monitored through monitoring sheets for contractors of school infrastructure. No complaints were filed under the contractors’ GRM for workers. 69. The adaptation and translation of didactic workbooks into mother tongues contained in the Indigenous Peoples Plan were not planned or financed by the Project; however, these activities began with the project in compliance with the commitment assumed by the MINED, and it is planned to continue even beyond its completion. Fiduciary 70. Responsibilities for compliance with the Project’s financial management (FM) and procurement functions was assigned to the same units responsible for these processes under the ACE Project and its additional financing. The Project financed additional staff only to strengthen the Project’s technical capacity and safeguards management to avoid overburdening existing staff—fiduciary functions were carried out by existing staff of the unit. Given experience acquired with these functions under other World Bank-financed operations, and continued training by the World Bank for project-financed staff, the Project’s ISRs consistently rated FM and procurement compliance as Satisfactory. The main issues with procurement related to the: (i) procurement processes to be followed were based on traditional procurement procedures, as required by the World Bank despite the emergency nature of the operation and were more time-intensive than expected during preparation, and (ii) volatility in world markets for goods and services, including reduced supply and increased demand, that affected availability, prices and delivery dates of project inputs. C. BANK PERFORMANCE Quality at Entry 71. The World Bank helped the Government prepare a project, in coordination with the GLE, that responded to immediate needs to mitigate the pandemic’s impact on learning outcomes on an emergency basis. The Project’s design was concise, well-focused with activities responding to the realities of the different beneficiary students. The Project’s design correctly assigned responsibility for project management, including M&E, FM, procurement and safeguard compliance, to the existing unit that has extensive experience working with the World Bank and is implementing well similar functions under the ongoing ACE project. This not only ensured support by MINED staff familiar with World Bank requirements, but also helped secure coordination of the Project’s activities with those of other World Bank- and other donor-financed projects. The Project’s RF was tailored to its emergency nature and extremely short implementation period. Preparation correctly identified risks and mitigation measures, to the extent possible. Two areas that preparation could have addressed were: (i) finalizing the Operations Manual before approval of financing (although this was likely justified due to the short, emergency nature of preparation), and (ii) securing World Bank approval for more streamlined procurement procedures in view of the Project’s emergency nature. Page 20 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Quality of Supervision 72. The World Bank’s supervision was timely, results-focused and comprehensive in that it comprised sectoral, fiduciary and environmental and safeguard specialists. The Project’s ISRs were well documented, describing in detail implementation progress and updating key indicators to monitor progress towards achievement of the PDO. The restructuring proposal was timely, and well justified to provide time for completion of all project activities. The World Bank’s team held bi-weekly follow up meetings (virtually) with MINED throughout implementation, to address any concerns need to keep implementation on track. Justification of Overall Rating of Bank Performance 73. Overall World Bank Performance is rated Satisfactory. The World Bank responded expeditiously to the Government’s request, with a project that was “right-sized” and well designed to support quick implementation in response to challenges posed by the COVID-19 pandemic. The World Bank’s support was well justified in view of the long history its support to the education sector in Nicaragua, its knowledge of the sector and government programs and the fact that it had ongoing operations whose results could be affected in the absence of this rapid assistance. D. RISK TO DEVELOPMENT OUTCOME 74. Throughout implementation, the Government has demonstrated its full commitment to the Project and its objectives. Although the COVID-19 pandemic is improving, the objectives and outputs supported by the Project are equally relevant in other crisis or shock situations, including climate-related and natural disasters to which Nicaragua is constantly exposed. Therefore, the Government’s commitment to the Project continued strong upon completion. In fact, MINED highlighted that the Project, in its response to the pandemic-related impacts on the education sector, helped design and put in place several programs and activities that the Government had planned previously but had not gotten around to implement. These include, for example, the didactic booklets the support to the multi-grade primary schools through the Mobile Digital Classrooms and the Socio-Emotional Support Program, all of which would be equally relevant in response to a natural disaster. The MINED has, over the years, developed strong implementation capacity, including a dedication to monitoring, learning, and incorporating lessons learned. Equipment maintenance costs and other operating expenses are covered by the MINED’s normal budgetary allocations, with resources allocated to regional offices and educational centers. These factors all bode well for sustainability of project-financed programs. Although the financing has now closed, for a while, MINED had continued to count upon technical and financial support for the education sector from the World Bank under the Additional Financing of the Alliance for Education Quality Project. The main question relates to the Government’s ability to expand upon the successful programs that it developed under this GPE-funded project. The Government is formulating a project to expand the use of technology to secure funding for the equipment from bilateral or multilateral sources. V. LESSONS AND RECOMMENDATIONS 75. Many of the lessons learned from design and implementation may seem obvious but are worth reiterating given the Project’s highly satisfactory outcome. These include: 76. When the World Bank has a long-standing program of support to a sector that is impacted by Page 21 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) an emergency, timely support can build upon previous assistance to hopefully help sustain previous achievements. This GPE-funded Project benefitted from decades of World Bank support to Nicaragua’s education sector, and was designed to count upon mechanisms (e.g., fiduciary and E&S compliance), capacity and institutional arrangements that had been strengthened previously to ensure smooth and efficient implementation. In this sense, it is easier to implement an emergency project when the counterparts and mechanisms are in place. Further, while there is no solid evidence to substantiate this, it is reasonable to assume that an emergency project that builds upon previous assistance should help prevent backsliding on educational achievements. 77. Continuity in staffing in an implementing agency ensures the sustainability of institutional capacity strengthening and efficient implementation. MINED’s strong implementation capacity was due not only to a long-standing program of assistance, but to a continuity of the ministry’s staff over time. As a result, previous capacity building efforts have not been lost, previous lessons are internalized, and, as a result, implementation can be timely and effective. Further, the continuity in staffing and the related capacity strengthening over the years has resulted in strong multi-disciplinary teamwork among the MINED’s various units in both preparation and implementation which helps achieve expected results. 78. As a complement, experience with implementation of environmental and social policies over time, if positive, can lead to their implementation being internalized. Under the Project, not only were MINED staff fully conversant with the World Bank’s E&S policies and GEMS, but they have almost adopted them in their internal priorities. As an example, MINED initiated, with their own resources, to translate didactic workbooks into indigenous languages, although this activity had not been included in the Project. MINED will continue with the translations, as permanent institutional policy. Also, implementation support can focus efforts on project-specific issues, and, in the case of the Project, on bringing the counterpart up-to-speed on the World Bank’s new E&S framework that was just getting underway when the Project was approved. 79. Emergency projects should focus on a small number of priority activities that can be implemented in a short period and be monitored and measured to show results during that time. The Project’s design and institutional arrangements were streamlined and its Results Framework, including PDO and IRIs, defined to measure what the Project could be reasonably expected to achieve in a two-year period. They did not attempt to measure impacts on learning outcomes, for the reasons described in Section IV A. 80. Emergency projects offer an opportunity to engage with the client and pilot programs that may be useful in diverse circumstances, which had been on the back burner until the crisis . Nicaragua is subject to various natural disasters. The MINED had planned to develop alternative learning methods, to respond to an eventual crisis, but given other priorities had not yet designed or implemented them. Emergency support from GPE offered the Government an opportunity to implement various programs on a small scale and learn from their implementation before rolling them out on a larger scale. Page 22 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Support MINED’s COVID-19 Response Programs to Reduce Learning Losses Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Students supported by the Number 0.00 420,469.00 822,053.00 Project participating in MINED’s COVID-19 programs 28-Sep-2020 28-Sep-2020 14-Feb-2023 conducive to reducing learning losses Female students Percentage 0.00 50.00 50.00 participating in MINED’s COVID-19 programs conducive to reducing learning losses Students from rural areas Percentage 0.00 90.00 94.00 participating in MINED’s COVID-19 programs conducive to reducing Page 23 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) learning losses Students who self-identify Percentage 0.00 8.00 9.00 as indigenous or afro- descendants participating in MINED’s COVID-19 programs conducive to reducing learning losses Students in preschools Number 0.00 60,500.00 83,522.00 participating in MINED’s COVID-19 programs conducive to reducing learning losses Comments (achievements against targets): Objective/Outcome: Support MINED’s COVID-19 Response Program to Develop Socioemotional Skills Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Students supported by the Number 0.00 103,751.00 133,351.00 Project participating in MINED’s COVID-19 program 28-Sep-2020 28-Sep-2020 14-Feb-2023 conducive to developing socioemotional skills Students who self-identify Percentage 0.00 8.00 8.00 Page 24 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) as indigenous or afro- descendants who are benefited by the socioemotional support program Female students who are Percentage 0.00 50.00 52.00 benefited by the socioemotional support program Students from rural areas Percentage 0.00 55.00 55.00 who are benefited by the socioemotional support program Comments (achievements against targets): A.2 Intermediate Results Indicators Component: Support Programs to Reduce Learning Losses in Public Preschools and Basic Education Schools Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Educational centers Number 0.00 8,759.00 10,382.00 benefiting from the programs conducive to 05-Aug-2020 28-Sep-2020 14-Feb-2023 Page 25 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) reducing learning losses for students without connectivity Special Education schools Number 0.00 25.00 26.00 benefiting from the strategy to reduce learning 28-Sep-2020 28-Sep-2020 14-Feb-2023 losses for students without connectivity Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teachers trained in Number 0.00 13,915.00 49,257.00 accelerated learning methodologies to mitigate 05-Aug-2020 28-Sep-2020 14-Feb-2023 learning losses due to COVID- 19 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Students whose learning was Number 0.00 420,946.00 421,200.00 Page 26 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) assessed to evaluate learning 05-Aug-2020 28-Sep-2020 14-Feb-2023 losses since the pandemic started Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Multi-grade primary schools Number 0.00 68.00 80.00 implementing the Mobile Digital Classrooms program 05-Aug-2020 28-Sep-2020 14-Feb-2023 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Availability and operability of Yes/No No Yes Yes a digital platform with adaptive learning tools in 05-Aug-2020 28-Sep-2020 14-Feb-2023 primary education, aligned with the Nicaraguan curriculum Comments (achievements against targets): Page 27 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of times students Number 0.00 102,928.00 9,379.00 use the adaptive learning platform throughout the 28-Sep-2020 28-Sep-2020 14-Feb-2023 Project. Comments (achievements against targets): By completion, the 2,214 students had used the adaptive learning platform 9,379 times (IR Indicator 7, target: 102,928 times). However, based on more recent usage and estimates, MINED expects that an additional 63,002 students will have used the platform , bringing the total usage to almost 72,381 accesses by August 2023, expecting the final target , (102, 928) would be achieved by the end of 2023. Component: Support Programs to Develop Socioemotional Skills in Selected Preschools/Basic Education Schools Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Schools implementing the Number 0.00 765.00 900.00 socioemotional support program 05-Aug-2020 28-Sep-2020 14-Feb-2023 Comments (achievements against targets): Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Formally Revised Completion Page 28 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Target Educational staff trained in Number 0.00 3,220.00 3,910.00 the socioemotional support course 05-Aug-2020 28-Sep-2020 14-Feb-2023 Comments (achievements against targets): Component: Cross-Cutting Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Communications received Percentage 0.00 90.00 100.00 through the GRM resolved according to the agreed 05-Aug-2020 28-Sep-2020 14-Feb-2023 procedure Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Students benefiting from Number 0.00 426,210.00 822,053.00 direct interventions to enhance learning 05-Aug-2020 28-Sep-2020 14-Feb-2023 Students benefiting from Percentage 0.00 50.00 50.00 Page 29 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) direct interventions to enhance learning - Female Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Mothers and fathers who Number 0.00 2,700.00 57,796.00 provided feedback on the monthly meetings and 05-Aug-2020 28-Sep-2020 14-Feb-2023 support materials received to reinforce the learning of socioemotional skills Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Socioemotional support Yes/No No Yes Yes program for students, parents, and school staff 05-Aug-2020 28-Sep-2020 14-Feb-2023 includes information on gender-related risks in situations that affect emotional well-being Page 30 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Comments (achievements against targets): Page 31 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) . B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Support Programs to Reduce Learning Losses in Public Preschools and Basic Education Schools 1. Students supported by the Project participating in MINED’s COVID-19 programs conducive to reducing learning losses (Number) i. Female students participating in MINED’s COVID-19 programs conducive to reducing learning losses (Percentage) ii. Students from rural areas participating in MINED’s COVID-19 programs Outcome Indicators conducive to reducing learning losses (Percentage) iii. Students who self-identify as indigenous or afro-descendants participating in MINED’s COVID-19 programs conducive to reducing learning losses (Percentage) iv. Students in preschools participating in MINED’s COVID-19 programs conducive to reducing learning losses (Number) 1.1. Educational centers benefiting from the programs conducive to reducing learning losses for students without connectivity (Number) 1.2. Teachers trained in accelerated learning methodologies to mitigate learning losses due to COVID-19 (Number) 1.3. Students whose learning was assessed to evaluate learning losses since the pandemic started (Number) Intermediate Results Indicators 1.4. Multi-grade primary schools implementing the Mobile Digital Classrooms program (Number) 1.5. Availability and operability of a digital platform with adaptive learning tools in primary education, aligned with the Nicaraguan curriculum (Yes/No) 1.6. Number of times students use the adaptive learning platform throughout the Project. (Number) 1. A total of 822,053 students in programs to reduce learning losses; of these, 50 percent were female, 94 percent resided in rural areas (higher than the 55% planned, since finally mostly rural multi-grade schools were selected), and 9 percent self- Key Outputs by Component identified as IP/AD. A total of 83,522 preschool students also participated in programs (linked to the achievement of the Objective/Outcome 1) to reduce learning losses. 1.1.: 10,382 Educational Centers benefited from the programs conducive to reducing learning losses for students without connectivity. Page 32 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) 1.2.: a total of 42,927 teachers were trained in accelerated learning methodologies to mitigate learning losses due to COVID-19. 1.3.: 421,200 students whose learning was assessed to evaluate learning losses since the pandemic started. 1.4.: 80 multi-grade primary schools implemented the Mobile Digital Classrooms program. 1.5.: A digital platform with adaptive learning tools in primary education, aligned with the Nicaraguan curriculum was available and operating by the end of the Project. 1.6.: the adaptative learning platform was used 9,379 times by the end of the Project. However, based on more recent usage and estimates, MINED expects that an additional 63,002 students will have used the platform, bringing the total usage to almost 72,381 accesses by August 2023, expecting the final target (102, 928) would be achieved by the end of 2023. Objective/Outcome 2: Support Programs to Develop Socioemotional Skills in Selected Preschools/Basic Education Schools 2. Students supported by the Project participating in MINED’s COVID-19 program conducive to developing socioemotional skills (Number) i. Students who self-identify as indigenous or afro-descendants who are benefited by the socioemotional support program (Percentage) Outcome Indicators ii. Female students who are benefited by the socioemotional support program (Percentage) iii. Students from rural areas who are benefited by the socioemotional support program 2.1. Schools implementing the socioemotional support program (Number) Intermediate Results Indicators 2.2. Educational staff trained in the socioemotional support course (Number) 2. A total of 133,351 students participated in MINED’s COVID-19 program conducive to developing socioemotional skills. Key Outputs by Component 2.1.: 900 schools implemented the socioemotional support program (linked to the achievement of the Objective/Outcome 2) 2.2.: a total of 3,910 educational staff were trained in the socioemotional support course. Cross Cutting Indicators 3.1. Communications received through the GRM resolved according to the agreed Intermediate Results Indicators procedure (Percentage) 3.2. Students benefiting from direct interventions to enhance learning (CRI, Number) Page 33 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) i. Students benefiting from direct interventions to enhance learning - Female (Percentage) 3.3. Mothers and fathers who provided feedback on the monthly meetings and support materials received to reinforce the learning of socioemotional skills (Number) 3.4. Socioemotional support program for students, parents, and school staff includes information on gender-related risks in situations that affect emotional well-being (Yes/No) 3.1.: All (100%) communications received through the GRM were resolved according to the agreed procedure. 3.2.: 822,053 Students benefited from direct interventions to enhance learning i. 50% of the students benefited from direct interventions to enhance learning – Key Outputs by Component were Female. (linked to the achievement of Cross Cutting Indicators) 3.3.: 57,796 mothers and fathers provided feedback on the monthly meetings and support materials received to reinforce the learning of socioemotional skills 3.4.: Socioemotional support program for students, parents, and school staff includes information on gender-related risks in situations that affect emotional well-being Page 34 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Enrique O. Alasino Massetti, Alonso Sanchez Task Team Leader(s) Monica Lehnhoff, Carlos Lago Bouza Procurement Specialist(s) Luis Barajas Gonzalez Financial Management Specialist Jorge Antonio Bastino Team Member Norma Vida Malespin Ramirez Team Member Marcela Lucia Silveyra de la Garza Team Member Adriana Cecilia Espinal Saballos Team Member Silvia Guallar Artal Team Member Antonella Novali Team Member Marta Jordao Henriques Environmental Specialist Erica Virginia Piber Social Specialist Maria Virginia Hormazabal Team Member Linda Castillo Procurement Team Violeta Arancibia Team Member Supervision/ICR Marcelo Becerra, Katia Marina Herrera Sosa Task Team Leader(s) Monica Lehnhoff Procurement Specialist Alvaro Gilberto Fernandez Trigoso Financial Management Specialist Mary Lou M. Veizaga Procurement Team Ingrid Schreuel Social Specialist Page 35 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Karla Rodriguez Meyer Procurement Team Marta Jordao Henriques Environmental Specialist Renata Pantoja Team Member Adriana Cecilia Espinal Saballos Team Member Marcela Lucia Silveyra de la Garza Team Member Norma Vida Malespin Ramirez Team Member Jorge Antonio Bastino Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY22 0 795.60 Total 0.00 795.60 Supervision/ICR FY21 2.650 62,380.08 FY22 7.500 91,480.63 FY23 4.783 74,676.69 Total 14.93 228,537.40 Page 36 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (%) Programs to Reduce Learning Losses in Public Preschools 5,15 4.69 93 and Basic Education Schools Program to Develop Socioemotional Skills in 1,52 1.96 128 Selected Public Preschools and Basic Education Schools Project Management, 0,10 .07 70 Monitoring and Evaluation Total 0.00 6.77 100.00 Page 37 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) ANNEX 4. EFFICIENCY ANALYSIS 1. The Project’s ex-post economic analysis was prepared following the same methodology used at Appraisal, a Cost-Benefit Analysis (CBA). The approach is based on the monetary benefits derived from the decrease of learning losses and the prevention of a rise in dropouts. Specifically, the Project mitigates learning losses by enabling access to education for children who otherwise might risk not attending schools and disrupting their learning. Without timely alternatives to education delivery, students would not able to complete the academic year, which could result in increased dropout rates. The Project aimed to prevent this by ensuring an appropriate delivery of workbooks to all primary students in targeted schools, who would thus benefit from uninterrupted education throughout the year. Lastly, the addition of technology to teaching practices, the digital tools in targeted schools, the adaptive learning pilot, and the socioemotional support offered to teachers, students and their families, all were designed to increase the quality of education received by the students in the short and medium term. 2. The sources of the data are the results of the project indicators; information provided by MINED (2022), and some parameters estimations used in the CBA from the Appraisal document. Main Project Interventions and their development impact 3. The COVID-19 led economic crisis disproportionately affected vulnerable students, directly impacting their human capital development. At the beginning of 2020, the education system in Nicaragua already presented bottlenecks in the quality of and access to education, leading to growing inequalities between urban and rural students. These challenges were expected to worsen equity issues and slow any progress made in the event of a widespread outbreak. The vulnerable populations already faced higher rates of job insecurity and constrained access to basic services; the pandemic thus was expected to pose a higher risk to their household income, health and nutrition, and returns to education. In addition, COVID- 19 was expected to force distance education as the alternative to face-to-face education, and rural students would inevitably face additional learning challenges with limited connectivity, likely increasing dropout rates. Furthermore, the pandemic was also expected to pose supply-side threats through the increased likelihood of a reduction in education spending because of the country’s economic deterioration. Thus, vulnerable students were expected be the most affected, with a long-lasting detrimental impact on their human capital development. 4. The Project placed a strong emphasis on mitigating learning losses and enabling access to adequate digital learning alternatives for students without connectivity. The first component aimed to mitigate the learning losses caused by the persistent low attendance rates in rural areas and the risk of disruption to access to education due to COVID-19. To accomplish this, the Project created didactic educational primary school workbooks specifically designed to enrich instruction, thus reducing learning losses for students. MINED then distributed them to selected areas and trained teachers on pedagogical practices and a prioritized curriculum so they could best support students’ learning process. This arm of the intervention aimed to mitigate learning losses by enabling rural students, who would otherwise lose access to education for the duration of outbreaks, to continue their schooling, in effect reducing their likelihood of dropping out and aiding their accumulation of human capital. This was designed to result in better student performance that would then translate into higher productivity and higher wages in the Page 38 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) labor market in future years. Notably, Montenegro and Patrinos (2014)22 report that in Latin America, private returns to education are 9.2 percent per additional year of education. Even though in Nicaragua schools remained open throughout the pandemic, for this analysis, the expected benefits come from the mitigation of the disruption in education, as a result, the difference between the time students would have lost in the absence of the Project (7 months lost) and the scenario with the Project (3 months lost). In other words, we adjust the 9.2 percent expected annual income increase by four months, which results in 3.68 percent increased wages for every year of additional schooling. 5. Teachers played a key role in students’ learning experience with the workbooks. To complement the learning process, teachers supported students by providing daily tasks to maintain their motivation. Students also enjoyed an individualized teacher interaction, which enriched the quality of education received. The beneficiaries consequently enjoyed a better learning experience and, in fact, McEwan (2015)23 estimates that instructional materials programs increase learning by 0.08. This increase in learning outcomes can be translated into an increase of annual wages of 0.067 percent, assuming the similar ratios of better education quality.24 6. Additionally, the Project supported the transition to digital learning for primary school and the provision of the appropriate equipment and tools in rural areas. Although public schools remained open, MINED has started making digital learning programs available to students. This subcomponent aimed to reduce learning losses in primary education through access to digital tools that could adapt to each student’s learning pace, which could then be used as a complement to the curriculum, and were made accessible to students nationwide both at school and at home. In areas where connectivity is an issue, digital mobile classrooms were placed with the appropriate equipment for 80 multi-grade primary schools. McEwan (Ibid) estimates that programs with computers and instructional technology increase student learning outcomes by 0.15. Similar to previous benefit calculations, we assume the same ratio of conversion to wages at the age of 28,25 and that 0.15 increase would result in 0.125 percent annually in future wages. 7. Introduction of an adaptive learning pilot program. This adaptive learning program has already proved successful in reducing learning losses and was launched for students in 10 of the 80 multi-grade primary schools receiving digital mobile classrooms, and 15 additional primary schools. These students benefited from a personalized content based on their current learning levels and the dynamics of their performance. The expected benefits, as found by Muralitdharan et al (2019)26 from similar programs, are a 0.15 change in learning outcomes. This improvement in learning for students will translate into a 0.125 percent increase in annual wages once they enter the labor market. 22 Montenegro, Claudio E., and Harry A. Patrinos. “Comparable Estimates of Returns to Schooling around the World.” World Bank Policy Research Working Paper, no. 7020 (2014). http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2491933. 23 McEwan (2015) in “Improving Learning in Primary Schools of Developing Countries: A Meta-Analysis of Randomized Experiments,” documents consistently positive effects of programs relying on instructional materials, particularly if coupled with relevant teacher training. 24 Chetty, Raj, John N. Friedman, and Jonah E. Rockoff. “Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value - Added Estimates.” American Economic Review 104, no. 9 (September 2014a): 2593–2632. doi:10.1257/aer.104.9.2593. 25 Chetty at al. (Íbid) 26 Muralidharan, Karthik, Abhijeet Singh, and Alejandro J. Ganimian. 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India." American Economic Review, 109 (4): 1426-60. Page 39 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) 8. The pandemic was expected to have a direct effect on the mental health of students and their families, with more acute effects to the well-being of vulnerable students. An added challenge for students in Nicaragua is the trauma and psychological distress resulting from the crisis, the pandemic, and the instability generated. To address this, the Project financed socioemotional support for: (i) school practitioners and teachers as they embark on new ways of teaching to be able to provide support to students; (ii) for the families of the students, who might also be facing job insecurity and have to adapt to home schooling and supporting their children; and (iii) students who would be coping with homeschooling and self-learning. Schools in rural areas are the most at risk of increasing dropout rates and face added stress of families’ job losses and economic distress. A study by Durlak et al (2011) found positive effects in many aspects of students’ lives by enhancing their social and emotional learning. Among the benefits, they estimate an increase of 0.27 in performance and learning outcomes, which we can translate with similar logic as above to 0.223 percent higher annual income. Cost Benefit Analysis Approach 9. The analysis assumes five channels by which students of targeted schools will see measurable benefits from this Project. The channels we used for the quantification of benefits are: (i) workbooks for teachers and students in all grades from preschool to lower secondary; (ii) mobile digital classrooms for students in multi-grade primary schools; (iii) digital tools and content for primary school students with connectivity; (iv) the adaptive learning pilot program for a subset of primary school students to enhance their learning with a personalized learning experience; and (v) socioemotional support provided to students to cope with the different challenges of COVID-19. While the socioemotional support is also offered to school staff and student families, we are only quantifying the benefits of the direct support to students. We then base our estimations on the relevant academic literature to determine the effect of each component on education outcomes and future wages. 10. Statistics specific to the Nicaraguan economy and education sector were used for the estimation of benefits. To estimate future wages and given the high levels of informality in the labor market in Nicaragua, we use the latest official GDP per capita US$2,029 (2018) as a proxy for average earnings. Moreover, the analysis is based on the following assumptions: (i) students benefiting from the program enter the workforce at an average age of 22 years old; (ii) following the COVID-19 simulator tool, we assume 45 average working years, which means that they retire at 67 years old; (iii) labor force participation of 33 percent; (iv) the digital classrooms and learning platform are the only arms of the component that assume to have more than one cohort of impact; and (v) a discount rate of 8 percent is used for the estimations of the benefits. Cost-Benefit Analysis Results 11. As a result, according to the updated (ex-post) cost-benefit analysis, the activities supported by the Project are expected to have a cost-benefit ratio of 17.5, with an internal rate of return of 11 percent, higher than the original estimates of 11.8 and 8.8 percent, respectively. The better results are due to the higher number of beneficiaries that the Project supported, as shown in the tables below: student and teachers workbooks, students using digital mobile classrooms, adaptative learning platform and receiving socioemotional support. This estimation is considered more realistic than the Appraisal estimate since it uses actual data for 2022 about the number of project beneficiaries. Page 40 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Table A4.1: Estimated Beneficiaries at Appraisal Beneficiaries 2020 2021 2022 2023 2024 2025 Student and Teacher workbooks - 495,231 - - - - Primary school digital content - 49,523 49,523 - - - Digital Classroom - - 4,414 4,414 4,414 4,414 - Adaptive learning program - 1,778 1,778 1,778 1,778 Social Emotional Support for students - 122,060 122,060 - - - Total: 666,814 177,775 6,192 6,192 6,192 Table A4.2: Beneficiaries at Project Completion Beneficiaries 2020 2021 2022 2023 2024 2025 Student and Teacher workbooks - 822,053 - - - Primary school digital content - 49,237 49,237 - - Digital Classroom - - 6,052 6,052 6,052 6,052 Adaptive learning program - - 2,214 2,214 2,214 2,214 Social Emotional Support for students - 133,351 133,351 - - Total: - - 1,012,907 188,640 8,266 8,266 Table A4.3: Summary Discount rate used: 8% Overall estimated cost ratio 17.5 Overall estimated Internal Rate of Return (IRR) 11 12. Without the appropriate intervention to address the risks of COVID-19, Nicaraguan students were expected to have had a reduction of 0.5 in learning adjusted years of schooling (LAYS). Partly using the World Bank Country Tool for simulating the potential impacts of COVID-19,27 it is anticipated that under an intermediate scenario in absence of the Project, Nicaraguans would see a reduction in LAYS from the pre-COVID-19 levels of 7.3 to 6.8.28 In detail, the reduction in LAYS comes from a reduction of 0.4 in expected years of schooling (EYRS) and a decrease of 11 points in the harmonized test scores (HLO). Further, this reduction of 0.4 in schooling is mitigated by the Project and the benefits estimated for this analysis stem from the difference between a scenario without the Project (a counterfactual) and one with the Project. 27 Azevedo, Joao P., Koen Geven, Diana Goldemberg, Amer Hasan, Syedah Aroob Iqbal (2020) "Country tool for simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes, Version 5". World Bank, Washington DC. 28 The optimistic and pessimistic scenarios projected a reduction in LAYS of 0.3 and 0.7, respectively. Page 41 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) Sensitivity analysis 13. Three scenarios were calculated: pessimistic, intermediate and optimistic, that serve as the sensitivity tests and ensure the viability of the Project. The analysis performed three scenarios that changed the discount rates used and size of reduction in LAYS assumed as a result of COVID-19. As shown in the table below, the main results reflect the intermediate scenario that relies on a discount rate of 8 percent and a reduction of 0.4 in LAYS. The results of the three scenarios are presented in the following table, where even in the pessimistic scenario, the estimates show the Project as viable and a good investment. Table A4.4. Sensitivity Analysis Reduction Discount Benefit Cost IRR in LAYS Rate Ratio Pessimistic 0.7 10% 13.2 9.6% Main results 0.4 8% 17.5 11.0% Optimistic 0.3 8% 30.5 13.9% Are public sector provisions or financing the appropriate vehicle? 14. Public financing is justified on both efficiency and equity grounds. First, the pandemic is disproportionately affecting vulnerable groups in the country, groups that cannot afford or have constrained access to basic services otherwise. Second, we learned from the Social Sector Expenditure and Institutional Review that 94 percent of the rural education public and rural schools with no connectivity were a focus of the Project to mitigate further inequalities between the rich and the poor. These two points justify public investment from an equity ground. The private sector generally brings better educational attainment, yet for rural inhabitants with added imperfect information due to the informality of their jobs, it means credit constraints or, in the case of Nicaragua, also poorly functioning credit markets. Furthermore, since the impacts of additional schooling have strong positive externalities on society that go beyond the accrued benefits to the individual, by increasing the overall level of productivity and growth in the economy, public investments and subsidies are justified with increased efficiency. Page 42 of 43 The World Bank Nicaragua COVID-19 Education Sector Response Project (P174677) ANNEX 5. BORROWER COMMENTS ON THE ICR A final draft version of the ICR was shared with the GoN, through MINED. Comments received either of editorial/clarificatory nature or data updates have been incorporated into this final version. Page 43 of 43