Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004285 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON A GLOBAL PARTNERSHIP FOR EDUCATION FUND II TRUST FUND GRANT (TF#13232) IN THE AMOUNT OF US$16.7 MILLION AND A EUROPEAN UNION TRUST FUND GRANT (TF#15143) IN THE AMOUNT OF EUR 26.9 MILLION TO THE THE REPUBLIC OF NICARAGUA FOR THE EDUCATION SECTOR STRATEGY SUPPORT PROJECT ( P133557 ) MAY 16, 2018 Education Global Practice Latin America And Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective April 27, 2018) Currency Unit = Nicaraguan Cordobas Oro (NIO) NIO 31.19 = US$1 US$1.21 = EUR 1 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ACE Alliance for Education Quality CPS Country Partnership Strategy ECD Early Childhood Development EMF Environmental Management Framework ESS Education Sector Strategy EU European Union FM Financial Management GDP Gross Domestic Product GDPE General Directorate of Preschool Education (Dirección General de la Educación Preescolar GDSE General Directorate of Secondary Education (Dirección General de Educación Secundaria GDSI General Directorate of School Infrastructure (Dirección General de Infraestructuras Escolares) GDTT General Directorate of Teacher Training (Dirección General de Formación Docente GPE Global Partnership for Education GoN Government of Nicaragua ICT Information and Communication Technology IP Indigenous People IPP Indigenous Peoples Plan IRR Internal Rate of Return ISR Implementation Status and Results Report M&E Monitoring and Evaluation MHCP Ministry of Finance and Public Credit (Ministerio de Hacienda y Crédito Público) MIFAN Ministry of Family (Ministerio de la Familia, Adolescencia y Niñez) MINED Ministry of Education (Ministerio de Educación) MINSA Ministry of Health (Ministerio de Salud) OM Operational Manual PAD Project Appraisal Document PAESE Education Sector Strategy Support Project (Proyecto de Apoyo a la Estrategia del Sector Educativo) PASEN Support to Education Sector Project (Proyecto de Apoyo al Sector Educativo de Nicaragua) PDO Project Development Objective PROSEN Nicaragua Education Sector Strategy Support Project (Programa de Apoyo al Sector Educación en Nicaragua) RAAS South Atlantic Autonomous Region (Región Autónoma Atlántico Sur) SD Standard Deviation SEAR Autonomous Regions Educational System (Oficina de Enlace con el Sistema Educativo de las Regiones Autónomas) SEIDI Integrated Early Childhood Development Monitoring and Evaluation System (Sistema de Evaluación Integral de Desarrollo Infantil) SOE Statement of Expenses TRP Temporary Relocation Plan UNESCO United Nations Educational, Scientific and Cultural Organization WB/Bank World Bank Regional Vice President: Jorge Familiar Country Director: Yaye Seynabou Sakho Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager: Reema Nayar Task Team Leader(s): Enrique O. Alasino Massetti ICR Main Contributor: Ingrid Bjerke TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 10 II. OUTCOME .................................................................................................................... 12 A. RELEVANCE OF PDOs ............................................................................................................ 12 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 12 C. EFFICIENCY ........................................................................................................................... 16 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 16 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 16 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 19 A. KEY FACTORS DURING PREPARATION ................................................................................... 19 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 20 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 21 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 21 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 23 C. BANK PERFORMANCE ........................................................................................................... 25 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 27 V. LESSONS AND RECOMMENDATIONS ............................................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 30 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 44 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 46 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 47 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 50 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 62 ANNEX 7. THEORY OF CHANGE WITH SUPPORTING LITERATURE .......................................... 65 ANNEX 8. REVISION HISTORY OF PROJECT INDICATORS........................................................ 68 The World Bank Education Sector Strategy Support Project ( P133557 ) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P133557 Education Sector Strategy Support Project Country Financing Instrument Nicaragua Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Republic of Nicaragua Ministry of Education (MINED) Project Development Objective (PDO) Original PDO The objectives of the Project are to: (a) increase access to preschool education in selected municipalities, and to improve preschool education learning conditions nationwide; and (b) increase access to lower secondary education in selected municipalities, and improve lower secondary education quality and completion rates nationwide. Page 1 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 16,700,000 16,700,000 16,700,000 TF-13232 34,752,263 32,688,269 32,688,269 TF-15143 Total 51,452,263 49,388,269 49,388,269 Non-World Bank Financing Borrower 0 0 0 Total 0 0 0 Total Project Cost 51,452,263 49,388,269 49,388,269 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 19-Apr-2013 02-May-2013 20-Apr-2015 30-Apr-2016 15-Nov-2017 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 19-Oct-2015 22.95 Change in Loan Closing Date(s) 24-Feb-2016 22.95 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financing Plan Change in Legal Covenants Change in Implementation Schedule 28-Oct-2016 33.13 Change in Loan Closing Date(s) 05-Apr-2017 41.68 Change in Loan Closing Date(s) 05-Sep-2017 48.02 Change in Components and Cost KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial Page 2 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 30-Oct-2013 Satisfactory Satisfactory .80 02 26-May-2014 Moderately Satisfactory Moderately Satisfactory 2.08 Moderately 03 01-Dec-2014 Moderately Unsatisfactory 6.90 Unsatisfactory 04 17-Jun-2015 Moderately Satisfactory Moderately Satisfactory 22.95 05 22-Dec-2015 Moderately Satisfactory Moderately Satisfactory 22.95 06 29-Jun-2016 Moderately Satisfactory Moderately Satisfactory 33.13 07 04-Jan-2017 Moderately Satisfactory Moderately Satisfactory 41.68 08 10-Aug-2017 Satisfactory Moderately Satisfactory 48.02 09 15-Nov-2017 Satisfactory Satisfactory 48.02 SECTORS AND THEMES Sectors Major Sector/Sector (%) Education 100 Early Childhood Education 30 Secondary Education 70 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100 Education 100 Access to Education 50 Education Financing 50 Page 3 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ADM STAFF Role At Approval At ICR Regional Vice President: Hasan A. Tuluy Jorge Familiar Calderon Country Director: Carlos Felipe Jaramillo Yaye Seynabou Sakho Senior Global Practice Director: Keith E. Hansen Jaime Saavedra Chanduvi Practice Manager: Reema Nayar Reema Nayar Task Team Leader(s): Michael Drabble Enrique O. Alasino Massetti ICR Contributing Author: Ingrid Bjerke Page 4 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Country context. Nicaragua’s economic performance and macroeconomic policy management improved in the early 2000s. After contracting 1.5 percent in the 2009 recession, Nicaragua recovered quickly and gross domestic product (GDP) grew 4.5 percent in 2010 and 4 percent in 2011. Nonetheless, Nicaragua remained the second poorest country in Latin America (after Haiti), with a GDP per capita of US$1,239 in 2011. Approximately 42 percent of the population lived below the poverty line, and one out of every five Nicaraguans lived in extreme poverty. About 300,000 families had children living in extreme poverty, and about a third of them had at least one child (age 7–12) either working or out of school. To address these issues, the Government of Nicaragua (GoN) set a pro-poor strategy that emphasized the delivery of both infrastructure and social services as part of its National Plan for Human Development for 2007-2011. 2. Sectoral and institutional context. In 2010, average education attainment in Nicaragua was 5.8 years for the population 25 years and older. Access to quality preschools in Nicaragua was low, especially for rural areas. Preschool gross enrollment was 56 percent, with rural enrollment at 47 percent, compared to urban enrollment at 70 percent. Many children did not attend preschool at all, because, among other reasons, distances to schools were prohibitive for young children. Of the children attending preschool in rural areas, 74 percent attended community-based institutions. Community-based preschools had poor infrastructure and were led by volunteers from local communities, often lacking formal pedagogical training (with 70 percent not certified as teachers). They used a different, lower quality curriculum than formal preschools and had little provision of educational materials. Formal preschools—usually located within a public primary school—offered better infrastructure, but also lacked learning materials. Preservice training for teachers was outdated, and in-service training was limited in both scale and duration. In 2010, the GoN established the Early Childhood Commission to better integrate and coordinate Early Childhood Development (ECD) among the relevant ministries and stakeholders. The commission led the preparation of an Early Childhood Policy, which was approved by the GoN in November 2011. 3. At the primary level (grades 1–6), despite significant expansion of access over the previous decade, Nicaragua was still far from achieving universal completion of six grades. Primary net enrollment was 87 percent, of which less than 75 percent finished the primary cycle. The retention rate in primary education, particularly in the first two grades, was low: only 74 percent of the children who entered grade 1 in 2010 were enrolled in grade 2 in 2011. Among the Ministry of Education’s (MINED) 25,000 primary school teachers, two-thirds worked in rural primary schools, with many lacking teaching certification (‘empíricos’). Dropout and retention rates were worse in rural schools compared to urban schools. Based on the 2010 National Learning Assessment, approximately 40 percent of students scored below what was considered to be ‘intermediate’ across all fields, with math learning levels particularly low. 4. Lower secondary education (grades 7–9) also suffered from inadequate access and low completion rates, particularly in rural areas. The overall net enrollment rate in lower secondary was 45 percent, just 28 percent in rural areas (versus 61 percent in urban areas), and as low as 20 percent in the Page 5 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) two Atlantic Autonomous Regions. Dropout rates were much higher in rural areas relative to urban areas, with the urban-rural gap for the 20 years-old cohort being the highest in Central America. The lower secondary level completion rate was 52 percent, placing Nicaragua behind Costa Rica, Panama, and El Salvador, and on level with Honduras. Low enrollment at the lower secondary level resulted from lack of access; at the secondary level, there were only 857 schools nationwide compared to 7,876 primary schools. Many of the 3,000 secondary classrooms needed significant repair or replacement, and 40 percent of secondary level teachers lacked formal training, with the vast majority of these working in rural areas. There was also a lack of basic learning materials such as textbooks and teaching guides nationwide. 5. Finally, administrative capacity across the system was weak, which reduced MINED’s ability to optimize scarce educational resources to improve education management and student learning opportunities. As such, MINED had identified critical weaknesses in planning, statistics, learning assessment, and monitoring and evaluation (M&E) at both central and regional levels. 6. In this context, MINED articulated three key priorities as part of its 2011–15 Education Sector Strategy (ESS): (a) expanded access to preschool education; (b) universal completion of six grades of primary education; and (c) universal access to lower secondary education (grades 7–9). The World Bank (WB), European Union (EU), and Global Partnership for Education (GPE) worked together to support these efforts. This Project was built on a Grant Agreement signed between the EU and GoN, focused on lower secondary education, and a Grant from the GPE to support the preschool sector. The Project was designed to complement the Bank’s Second Support to the Education Sector Project (PASEN II)1, which focused on primary education. Together, the Project and PASEN II formed the building blocks for the GoN’s 2011- 2015 ESS. 7. These Projects both build on the Bank’s longstanding engagement with the GoN in the education sector, including other operations and analytical work.2 The Project was fully aligned with the World Bank Group’s Country Partnership Strategy (CPS) 2012–17, Report No. 69231-NI, discussed by the Executive Directors on November 13, 2012, and it is consistent with the Strategic area 1, ‘Raise welfare by improving access to quality basic services’. The CPS emphasized the need to ensure continued expansion of education access and improvement in the quality of teaching and learning, as well as strengthening of institutional capacity and efficiency in the sector. 1 P126357, Cr. 5036-NI, approved by the Board of Executive Directors on January 17, 2012. 2 These are: (i) the Support to the Education Sector Project (PASEN I, P078990, Cr. 3978-NI); ii) the First Basic Education Project (APRENDE I) and the Second Basic Education Project (APRENDE II) (P050613, Cr. IDA-3281-NI; P007783, Cr. IDA-2689-NI); (iii) the first Education For All Fund (EFA) (P078990, TF057311); and (iv) the implementation of a school feeding program (Merienda Escolar) through the Japanese Social Development Fund (JSDF) (TF097212). Page 6 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Theory of Change (Results Chain) Figure 1. Overview of the Project’s Theory of Change Activities/Outputs Intermediate Project Development Long- Outcomes Outcomes Term Outcomes Increased number of Serra V. et al. • Increase access to classrooms and safety (2003) preschool education in of environment Improving/expanding selected municipalities education infrastructure Schneider • Increase access to lower Improved physical (2002) secondary education in environment for selected municipalities learning Development/ Updated curriculum alignment of Abadzi (2006) and instruments for Increase preschool curricular preschool nationwide individual instruments and social • Improve preschool benefits Improved teaching Training and Chetty (2014) education learning of practices and certification of conditions nationwide education teaching teachers effectiveness • Improve lower secondary education quality Development and Increased supply and Glewwe et al. nationwide distribution of improved materials (2013) • Improve lower secondary learning materials for learning education completion rates nationwide Improved Institutional management of strengthening of preschool and lower MINED secondary education sectors 8. The logic of the results chain of the Project is clear and well-supported by evidence in the literature. For details, see annex 7. Project Development Objectives (PDOs) 9. The objectives of the Project are to: (a) increase access to preschool education in selected municipalities, and to improve preschool education learning conditions nationwide; and (b) increase access to lower secondary education in selected municipalities, and improve lower secondary education quality and completion rates nationwide. Key Expected Outcomes and Outcome Indicators 10. The PDO was to be measured through the following five indicators: Page 7 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) (a) Increase in the gross enrollment rate for preschool (ages 3–5 years) in selected municipalities. (b) Increase in the gross enrollment rate for lower secondary (grades 7–9) in selected municipalities. (c) Learning conditions improved in preschool. (d) Increase in the grade 9 completion rate. (e) Increase in the percentage of grade 9 students achieving intermediate, advanced or excellent proficiency levels in standardized evaluations for Spanish and Mathematics. Components 11. The Project financed the following four components: Component 1: Increasing Access and Improving Learning Conditions of Preschool Education (Appraisal: US$15.9 million; Actual: US$15.85 million, funded by the GPE) 12. Component 1 aimed to contribute to increasing access to preschool education in 43 selected municipalities3 and improving preschool learning conditions nationwide. 1.1. Improving and/or Expanding Preschool Education Infrastructure. This subcomponent sought to support the construction and rehabilitation of classrooms and learning and complementary facilities (i.e. safety and sanitation) in selected preschools, including the provision of furniture and equipment. 1.2. Development and Alignment of Curricular Instruments. This subcomponent sought to support the (a) development of a unified curriculum covering the three levels of preschool; (b) alignment of instructional materials to the curriculum; and (c) development of a training curriculum for preschool teachers. 1.3. Training of Preschool Teachers. This subcomponent sought to support training activities for preschool teachers, including (a) certification-based training; (b) in-service training, including workshops, demonstration classrooms, and intra-school teacher exchanges; and (c) training materials to teachers’ workshops. 1.4. Improving Supply of Learning Materials. This subcomponent sought to support the provision and distribution of learning materials to preschool students. 1.5. Development of an Integrated ECD M&E System. This subcomponent sought to support the development and piloting of an integrated ECD M&E system (Sistema de Evaluación Integral de Desarrollo Infantil, SEIDI) and implementation of the system in selected municipalities. Component 2: Improving Access, Quality and Completion of Lower Secondary Education (Appraisal: US$33.5 million; Actual: US$30.69 million, funded by the EU) 13. Component 2 aimed to contribute to increasing access to lower-secondary education in 43 selected municipalities and improving quality and completion of lower secondary education nationwide. 3 Of 153 municipalities total in Nicaragua. Page 8 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 2.1. Improving and/or Expanding Lower Secondary Education Infrastructure. This subcomponent sought to support the (a) construction and rehabilitation of classrooms and learning and complementary facilities; (b) provision of furniture and equipment; and (c) preparation of infrastructure maintenance plans, all for selected lower secondary schools. 2.2. Improving Teacher Effectiveness in Secondary Education. This subcomponent sought to support (a) training activities through certification-based training, in-service training, information and communication technology (ICT) training, provision of training materials to teachers’ workshops; and the preparation of reference materials, teaching materials, and training of trainers’ materials; and (b) a results-focused M&E plan for the training activities. 2.3. Improving the Supply of Textbooks and Learning Materials in Secondary Education. This subcomponent sought to support the (a) development, revision, printing and distribution of textbooks and learning materials; (b) preparation of teacher guidelines and training activities for use of the textbooks and learning materials; and (c) adaptation of learning materials to integrate local cultural and linguistic aspects. Component 3: Strengthening of the Ministry of Education’s Institutional Capacity for Preschool Education (Appraisal: US$0.8 million; Actual: US$0.85 million, funded by the GPE) 14. Component 3 aimed to contribute to strengthening the institutional capacity of MINED to provide preschool education. 3.1. Strengthening of MINED’s Technical Capacity for Preschool Education. This subcomponent was designed to finance strengthening of the capacity of the General Directorate of Preschool Education (GDPE) through (a) training to preschool technical specialists; (b) provision of basic office equipment; (c) technical assistance for development of training materials; and (d) technical assistance for results-based management and M&E activities. 3.2. Project Management, Fiduciary Controls and Audits. This subcomponent was designed to finance support to the administration, financial management (FM), auditing and procurement aspects of Component 1 and Component 3 of the Project, including (a) technical assistance and training to MINED’s staff; (b) acquisition of software, information and communication hardware, and small furniture items; and (c) carrying out of the audit for Component 1 and Component 3. Component 4: Strengthening of the Ministry of Education’s Institutional Capacity for Lower Secondary Education (Appraisal: Total US$1.50 million; Actual: US$2.0 million, funded by the EU) 15. Component 4 aimed to support strengthening the institutional capacity of MINED to provide lower secondary education. 4.1. Strengthening of Capacity of MINED’s General Directorate of Secondary Education (GDSE). This subcomponent was designed to finance strengthening of the capacity of the GDSE to monitor curricula, prepare classroom observation guides, and monitor textbook distribution and utilization, through (a) technical assistance; (b) training activities; and (c) provision of basic office and ICT equipment. Page 9 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 4.2. Strengthening of MINED’s Directorate of Training. This subcomponent was designed to finance (a) technical assistance for the preparation of training; (b) provision of basic office and ICT equipment; and (c) training of staff in educational research. 4.3. Strengthening of MINED’s General Directorate of School Infrastructure (GDSI). This subcomponent was designed to finance (a) technical assistance to prepare feasibility studies and manage and supervise infrastructure construction and rehabilitation; (b) update of maintenance guidelines; (c) provision of basic office and ICT equipment; and (d) minor rehabilitation works. 4.4. Project Management, Fiduciary Controls and Audits. This subcomponent financed support to the administration, FM, auditing, and procurement of Component 2 and Component 4 of the Project through (a) technical assistance and training to staff; (b) acquisition of software, information and communication hardware, and small furniture items; and (c) the carrying out of the Project audit for Component 2 and Component 4. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets 16. There were no changes to the Project's development objectives or the overall intended outcomes. Revised PDO Indicators 17. A restructuring was approved on February 24, 2016, to update baseline data, adjust targets, and revise the wording to improve clarity of four PDO indicators (PDO Indicators 1, 2, 4, and 5). In addition, the definition of PDO Indicator 4 was updated to align with international usage. Twelve Intermediate indicators were similarly revised (see annex 8 for details on all changes). Revised Components 18. During the February 2016 restructuring, changes were made to the scope of the infrastructure activities under Components 1 and 2 to increase complementary investments in terms of water and sanitation, security, electricity, and recreational facilities, and, in the case of the core schools4, to increase the number of classrooms to rehabilitate per school. Consequently, the cost per intervention increased, and the overall number of schools and classrooms was reduced in line with the available funding, from 250 core schools with 500 classrooms to 75 core schools with 465 classrooms, and from 85 schools with 200 preschool classrooms to 60 schools with 106 preschool classrooms. Enhanced infrastructure interventions were still completed across all 43 targets municipalities, and all other activities were implemented nationwide as planned. Other Changes 19. During implementation, the Project closing date was extended for a cumulative period of 18.5 months. In addition, the budget was modified twice to reflect variations in the U.S. dollar equivalent of the financing provided by the EU (in euros), due to exchange rate fluctuations. Finally, minor revisions to 4 K-9 schools including lower secondary classes. Page 10 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) the financing plan, disbursement estimates, and implementation schedule were made in accordance with these changes. Rationale for Changes and Their Implication on the Original Theory of Change 20. The revision of PDO indicators took place to reflect more up-to-date information. The original baseline values used 2010 school and demographic projections data; however, the Project only became effective in 2013. The baseline and target values were thus revised on the basis of 2013 school year data, updated education data for the 2014 and 2015 school years, and updated 2015 demographic projections from the 2005 Population Census Data. Consequently, target values for relevant indicators were adjusted in response to revised baseline values. In addition, the wording and description of indicators were revised for clarity and to align with the (UNESCO) definition. Finally, the intermediate indicators were revised to reflect adjustments in the cost and scope of activities. 21. The infrastructure components were revised to reflect a change in strategy and to respond to issues that arose during implementation: (a) pre-investment studies undertaken during implementation showed that the need in the selected schools was much greater than was estimated during Project design with regard to school security, water, sanitation, and accessibility, among other issues; (b) the GoN requested that infrastructure investments follow a comprehensive approach, providing the full set of interventions each school needed (water, security, sanitation, etc.) to ensure a positive learning environment, rather than focalized improvements; (c) costs required revision to take into account a rise in the price of materials and labor since Project design, as well as increased transportation costs of labor and materials (specific site selection within the chosen municipalities were determined after appraisal); and (d) the total component budget had to be revised to reflect exchange rate fluctuations, which reduced available funding midway through implementation. This new approach was discussed thoroughly by the GoN, the Bank, and the two financiers. To compensate for the reduction in number of classrooms, the Project coordinated with other complementarity operations: (a) the ongoing preschool construction project funded by the Inter-American Development Bank and executed by the Ministry of Family (MIFAN) and (b) the lower secondary school construction component of the GoN’s own strategy for rural secondary education. 22. The Project closing date was extended to compensate for a delay in effectiveness of the EU grant due to a lengthy process to sign the financing agreement,5 as well as delays related to implementation of the infrastructure component. The changes in budget reflected negative and positive developments in the U.S. dollar/euro exchange rate, first necessitating cancellation of several activities and then partially reinstating them, such as minor civil works (improved drainage systems, security fences, and food storage rooms) and textbooks for rural secondary schools. 23. Since the restructurings did not entail any changes to the Project development objectives, their measurement, or the logic of the intervention, there was no impact on the theory of change. 5The Project was approved by the Bank on April 19, 2013, and the GPE grant became effective on May 2, 2013, while the EU grant became effective on September 19, 2013. Page 11 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 24. Relevance of the Project’s objectives is High. There were no shortcomings in the relevance of the Project objectives to the CPS (2012–2017) during preparation, and the objectives continued to be aligned throughout implementation. The CPS Performance and Learning Review (Report No. 98404-NI), discussed by the Executive Directors on October 14, 2015, determined that no significant changes were necessary to the CPS as the objectives remained relevant and aligned to the Government’s priorities throughout the period. The Project objectives are also fully aligned with the new Country Partnership Framework 2018– 2022 (Report No. 123026-NI) discussed by the Executive Directors on March 15, 2018. Specifically, the Project is relevant to Pillar 1, ‘Investing in human capital particularly for disadvantaged groups’ and the two associated outcomes: ‘Improved Learning Conditions and Employability’ and ‘Improved Health and ECD’. The Project is also relevant to Pillar 3, ‘Improving institutions for resilience and sustainability’ and the associated outcome: ‘Improved Government Transparency and Capacity’. 25. At the time of preparation, the Project was also aligned to the National Human Development Plan 2007–2011 and the 2011–2015 ESS. The Project objectives remain highly relevant to the priorities of the updated National Human Development Plan 2012–2016, which continues to emphasize access to free and quality health and education services. A continuation of these core themes is expected in the upcoming plan for 2017–2021, which is currently being developed. The Project also remains relevant to the objectives of the 2017–2021 ESS, which includes improving education quality, increasing equitable access, and fostering MINED’s institutional development. Relevance is also demonstrated through the Alliance for Education Quality (ACE) Project (P161029, Credit 6015-NI), approved by the Executive Directors on April 13, 2017, which builds on this Project in terms of teacher quality, learning materials, and school infrastructure. Finally, to ensure the Project’s continued relevance, the Project was adapted to address the needs of the sector throughout implementation. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 26. The PDO is assessed in five parts, using the associated PDO indicators and intermediate indicators. Although one PDO target was formally reduced during implementation6, a split rating was deemed unnecessary because the original target was nonetheless achieved. Several output targets were also revised, but these do not trigger the split as they reflected improved interventions aimed to achieve similar outcomes. Objective 1: Increase Access to Preschool Education in Selected Municipalities (Rating: High) 27. The objective of increasing access to preschools in selected municipalities was achieved. This is evidenced through the results of PDO Indicator 1 ‘Gross enrollment rate for preschool in selected 6 Through the 2016 restructuring, when the Project was 48 percent disbursed. Page 12 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) municipalities.’ In 2013, the baseline value for gross enrollment was 40 percent, and the end target for the Project was 44 percent. The Project surpassed this target, with a final rate of 50 percent gross enrollment at the end of the Project. Notably, this also surpasses the original target significantly.7 28. The intermediate indicator associated with this objective, ‘Number of preschool classrooms built/rehabilitated and equipped in selected municipalities’ surpassed its revised target of 106, with 110 classrooms built/rehabilitated. The works included complementary investments per school, including water, sanitation, security, recreational facilities, and electricity. In addition to providing adequate physical space for students, this increased the security and comfort of schools, encouraging student attraction. Even though the output target was reduced during implementation, due to the improvements in the intervention, the outcome was maintained. In addition to the improved spaces, the outcome presumably also reflects the benefits of the Project’s activities nationwide to improve preschool quality (i.e. new curriculum, teacher training and learning materials) which incentivized demand. Finally, the classrooms were built according to best practices in preschool education and served as a basis for MINED to create a new preschool classroom model that is now being implemented nationwide. Objective 2: Increase Access to Lower Secondary Education in Selected Municipalities (Rating: High) 29. The objective of increasing access to lower secondary in selected municipalities was achieved. This is evidenced through the results of PDO Indicator 2 ‘Gross enrollment rate for lower secondary (grades 7–9) in selected municipalities’. In 2013, the baseline value for gross enrollment was 47 percent, and the end target for the Project was 54 percent. The Project surpassed this target, with a final rate of 57.2 percent gross enrollment at the end of the Project. 30. The intermediate indicator associated with this objective was ‘Number of core schools (K-9) with facilities improved for lower secondary education’, and the target was substantially achieved with 74 schools built/rehabilitated against a target of 75, which represented 512 classrooms exceeding the planned 465 classrooms. The remaining school will be built with national funds outside the Project timeline.8 Objective 3: Improve Preschool Education Learning Conditions Nationwide (Rating: High) 31. The objective of improving preschool education learning conditions nationwide was achieved. This is evidenced through the results of PDO Indicator 3 ‘Learning conditions improved in preschool’. Learning conditions were considered effectively improved if 50 percent of the preschools nationwide have at least 3 out of the following 4 conditions: (a) teachers trained in the use of the unified curriculum, (b) revised classroom learning instruments available in preschools, (c) teachers certified as primary school teachers with a specialty in early child education, and (d) preschools received a package of learning materials. At end of the Project, this target was met, with 62 percent of schools achieving these conditions. 32. The Project helped MINED build, implement, and monitor a ‘Preschool Quality Model’ that is likely to have positively influenced learning conditions. The model embodied a strategic vision for improving 7 The target was reduced from 48 percent to 44 percent to reflect updated data and a reduced baseline. The project surpassed the original target by 132% and the revised target by 250%. 8 The building of one school was cancelled after receiving no eligible offers after multiple bidding processes. Given the lack of time to relaunch the bidding process, the funds were reassigned to finance complementary facilities in other works. Page 13 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) preschool education nationwide through five pillars: training of teachers, provision of learning materials, improvement of preschool infrastructure, revision of the curriculum, and M&E. By helping identify the issues in the sector, creating a quality model, and defining its components to tackle these issues, this package of support is expected to improve learning conditions. 33. The intermediate indicators associated with this objective all achieved or surpassed their targets: • The target for intermediate indicator ‘Preschool unified curriculum developed’ was achieved. One curricular framework and three preschool classroom instruction guides were integrated in one unified preschool curriculum. Overall, 1,941 preschool teachers and 4,849 pedagogical advisers and community educators were trained on the curriculum to support its effective use. This curriculum was developed by neuroscience experts, in line with the latest evidence. • The target for the intermediate indicator ‘Number of certified preschool teachers’ was surpassed, with 2,321 teachers certified versus a target of 2,300. The skill set is the first step toward better pedagogy in the classroom, and the ACE Project will build on this through classroom observations and feedback. • The target for the intermediate indicator ‘Number of preschools that receive learning material packages’ was achieved with 8,903 preschools receiving materials against a target of 8,500, which accounts for all the preschools in the country. The materials are being used in the classrooms, observed through regular supervision visits made by MINED’s Preschool Education Directorate. • The target for the intermediate indicator ‘System to evaluate early childhood development outcomes and preschool-age learning outcomes in place and operational’ was achieved. The system was designed, piloted, and validated. The system will be implemented nationwide with support from the ACE Project during 2018. Objective 4: Improve Lower Secondary Completion Rates Nationwide (Rating: High) 34. The objective of improving lower secondary education completion rates nationwide was achieved. This is evidenced through the results of PDO Indicator 4 ‘Grade 9 completion rate’. The baseline value in 2013 was 64 percent. The target for this indicator was surpassed, with 71.9 percent completion rate at Project end versus a target of 71 percent. 35. A number of Project activities were targeted towards improving completion rates. The intermediate indicators associated with this objective all substantially achieved or surpassed their targets: • The target for the intermediate indicator ‘A university-based certification program for lower- secondary teachers implemented’ was achieved, and the number of student teachers certified of 969 surpassed the target of 750. • The target for the intermediate indicator ‘Number of secondary school teachers who participate in the 22-day in-service teacher upgrading training based on the secondary curriculum’ was substantially achieved, with 1,529 teachers trained versus 1,600 targeted. • The target for the intermediate indicator ‘Number of secondary textbooks distributed’ was surpassed. Against a target of 1.5 million textbooks, the Project distributed 1.7 million textbooks. This included books to all students in public lower secondary schools (1.2 million) as well as (a) about 200,000 textbooks to all students in grades 10 and 11 in public upper Page 14 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) secondary schools, (b) about 386,700 textbooks to distance education students enrolled in secondary school living in rural areas, and (c) about 10,500 textbooks to all teachers in grades 7–11 in public secondary schools nationwide. To complement this, technical workshops on the proper use and care of these textbooks were carried out for 259 teaching mentors nationwide. These mentors in turn trained 9,875 teachers through community pedagogical circles and reference classrooms, reaching a total of 10,560 participants. The Directorate General for Teacher Training also carried out a total of 170 visits to accompany secondary school teachers in the classroom, and teachers received pedagogical counseling depending on the strengths and needs observed. • The target for the intermediate indicator ‘Number of teaching guides distributed to secondary school teachers’ was achieved, with 22,800 teachers receiving guides against a target of 16,300. These were distributed to all secondary teachers of public and semiprivate schools nationwide. To maximize impact, the training on the use of the new textbooks also included training on proper use of the teaching guides. Objective 5: Improve Lower Secondary Education Quality Nationwide (Rating: n.a.) 36. The Project intended to measure the improvement in quality in lower secondary education through PDO Indicator 5 ‘Percentage of grade 9 students achieving intermediate, advanced and above proficiency level in standardized evaluations for Spanish and Mathematics’. The baseline was measured through the National Student Learning Assessment carried out in 2010, with the expectation that changes could be measured through the results of the test when it would be carried out again in 2015 through PASEN II. However, the international consultants hired by MINED to prepare the national reports of the 2015 National Student Learning Assessment identified technical issues and recommended that the results of the assessment could not be used due to concerns regarding the quality of questions on the test overall and the reliability of data in the math section. MINED plans to undertake a review of the test and will update the test questions and methodology. Therefore, this indicator cannot be used to measure achievement of this objective. Nonetheless, the interventions that supported increases in completion rate were also designed to support quality, in line with literature on quality interventions (see annex 7), and all associated intermediate outputs were substantially achieved (see paragraph 35). Furthermore, although not completely reliable,9 as improvements in quality likely resulted in the improvements in completion rate the completion rate itself could be seen as a proxy indicator for quality. As such, the objective of improved quality is considered likely to have been achieved. However, given that there is no comparative end-line measure, the rating for this PDO is considered as Not Evaluable, although the Project likely did improve the quality of education. Justification of Overall Efficacy Rating 37. Out of the five objectives of the PDO, four have been achieved, with most targets surpassed, as measured by the PDO indicators and supporting intermediate indicators. Given the lack of direct evidence on the attainment of PDO Indicator 5, the fifth objective cannot be measured. As such, the Project’s overall Efficacy rating is Substantial. The achievement of the PDO would be the same against the original targets. 9Caution is used, as in some cases completion rate increases can occur even with a decrease in quality, as noted in the Nicaragua Completion and Learning Report for the FY13-FY17 CPS (World Bank 2018). Page 15 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) C. EFFICIENCY Assessment of Efficiency and Rating 38. The efficiency of achieving the Project’s objectives is deemed to be Substantial. Based on data collected through the Project, assuming a work life of 38 years starting at age 22, a discount rate of 8 percent, and using 2016 GDP per capita as a proxy for average wage, the internal rate of return (IRR) is estimated at 7.21 percent with a benefit-cost ratio of 5.63 (see details in annex 4). This analysis is based on activities under Components 1 and 2 (infrastructure, teacher training, updated curriculum, and learning materials). Actual gains may be even higher, as the analysis does not include activities under Components 3 and 4, as the impacts of institutional capacity building are harder to quantify, although these components are also likely to have benefits. In addition, the model uses conservative estimates of impact based on the literature, and does not include spillover effects such as benefits garnered by primary-level students, who were not targeted but attend the same schools and are able to use the improved facilities. Furthermore, the activities carried out complemented each other, providing students with safe physical space conducive to learning, with appropriate materials and qualified teachers, which may multiply the positive impacts of the Project for overall greater effect. 39. The implementing agency operated in a satisfactory manner, using existing systems and resources within MINED to facilitate implementation. The Project was implemented closely with PASEN II and increased efficiency by collaborating on joint tasks (such as pre-investment studies and audits) and by sharing key staff, thus reducing the need for additional hires and separate training. Project resources were used efficiently and largely as planned. Notably, the Project was extended by 18.5 months: 5 months due to the delay in effectiveness and 13.5 months due to delays in implementation. These delays were related to external challenges, such as inefficiencies in the civil works markets, as well as internal challenges such as staff turnover and limited capacity, especially following a reorganization within MINED in 2014. Nonetheless, appropriate action plans were followed and Project adjustments were made in a timely manner when needed, including increasing efficiency by reallocating Project savings between components to finance additional rehabilitation work. Moreover, despite reductions in some of the intermediate targets, the interventions were substantially improved during implementation and the Project met or exceeded the revised, as well as the original, PDO targets. The challenges created a delay in terms of time, but did not impact reaching the intended outcomes. Overall, efficiency was satisfactory and in line with expectations in the sector. D. JUSTIFICATION OF OVERALL OUTCOME RATING 40. The Relevance of Project objectives is High, Efficacy is Substantial, and Efficiency is Substantial. The overall outcome rating is therefore Satisfactory. E. OTHER OUTCOMES AND IMPACTS Gender 41. As the available data at both the preschool and lower secondary levels in Nicaragua show gender parity, the Project did not specifically seek to address a gender gap. Net enrollment data for 2014 –2017 disaggregated by sex shows a male:female ratio close to or at 50:50 in the 43 selected municipalities, as Page 16 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) well as in the remaining municipalities, and the rates remained steady over time.10 Unfortunately, the Project was unable to track PDO indicators disaggregated by gender because national population data disaggregated by sex are not available.11 Nonetheless, the Project did track participation in specific activities by gender (i.e. teacher training12), as well as the percentage of female beneficiaries overall, and met the target of 50 percent female beneficiaries throughout the life of the Project.13 42. Although the Project did not include specific actions focusing on gender gaps, gender inclusiveness was considered as a key element of the infrastructure investments. According to the 2014 household survey, distance to school was the main reason for lack of enrollment by 10 percent of girls versus 3 percent of the boys between 12 and 16 years in Nicaragua.14 In a 2003 study, one in four girls reported not feeling comfortable using school latrines because of, among other things, risk of harassment by boys or unhygienic conditions.15 The Project interventions reduced travel distance to school by providing secondary/preschool classrooms at schools where there were previously none and improved physical conditions within the school, including separate latrines with locks, that ensure girls privacy and safety. As such, Project activities were in line with gender inclusiveness and, as a positive externality of the Project, could contribute to increased access and an improved learning environment for girls. Institutional Strengthening 43. The Project included two components specifically aimed toward institutional strengthening, targeting MINED’s capacity for preschool and lower secondary education. These components financed support to improve technical and management capacities of MINED’s GDPE, GDSE, GDSI and the General Directorate of Teacher Training (GDTT), through technical assistance, training, and provision of minor infrastructure. In addition, in complementarity with PASEN II, the Bank supported targeted strengthening of MINED through (a) improved strategic planning and monitoring systems; (b) better statistics and information management capacities, including a new software for data management and analytics; (c) technical capacities and new protocols for the design and implementation of student assessment systems (i.e. SEIDI); and (d) a stronger infrastructure management system, including new pre-investment protocols and a new infrastructure inventory system in place. The inclusion of systemic strengthening (as opposed to only focusing on the offices and staff dedicated to the Project) solidified the gains made and mitigated risks such as high turnover of personnel and promoted sustainability. 44. The activities supporting institutional strengthening were measured through several intermediate indicators, all of which achieved their targets. This suggests that the operation had a positive impact on 10 Ratio deviates to maximum 48:52 (F:M) in 2014 for lower secondary in non-prioritized municipalities, otherwise remain at 49:51 or 50:50 each year across both sets of schools. Assuming a roughly 50:50 gender ratio in the population, access rate is inferred to be close to parity. 11 Due to delay in carrying out the 2015 national census, national population data disaggregated by sex is not available. The last census was in 2005. 12 As expected, the majority of teachers trained, in particularly at the preschool level, are female (98:2 ratio), reflecting general trends in the profession. 13 The total beneficiaries reflect students and teachers who receive intervention. As noted, teachers are majority female but the overall number is driven by the larger student enrollment numbers. 14 Household Surveys – Encuesta de Medición de Nivel de Vida 2014 – Instituto Nacional de Estadística 15 Serra V., Luis, and Marcia Castillo S. 2003 Page 17 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) institutional strengthening and lays the groundwork for the longer-term development of the country’s capacity. The improved functioning of these agencies also contributes towards the Project outcomes. • The target for the intermediate indicator ‘Directorate of Preschool Education is equipped for Project implementation and supervision activities’ was achieved. The Directorate received equipment, including computers, to strengthen management and coordination of technical teams in MINED’s 17 regional offices, as well as vehicles for supervision of Project activities. • The target for the intermediate indicator ‘General Directorate of School Infrastructure (GDSI) offices set up in two departmental education delegations and one territorial division of RAAS’16 was achieved. Three GDSI offices were created in Boaco, León, and Zelaya Central. These offices are used by the infrastructure teams to coordinate all efforts in the area related to monitoring of infrastructure works, design of pre-investment studies, and updating the school inventory database. • The target for the intermediate indicator ‘Development of infrastructure maintenance guides/norms’ was achieved and is being used to guide all infrastructure going forward. Poverty Reduction and Shared Prosperity 45. The Project contributed to reducing poverty and boosting shared prosperity. The Project supported reducing poverty through qualitative investments (teacher training and educational materials) to improve learning outcomes nationwide, contributing to increased productivity of future workers across the country. Moreover, the Project contributed to shared prosperity by targeting access-related interventions (improving infrastructure) to selected municipalities where education indicators were the worst and poverty levels the highest. Measured by the Unsatisfied Basic Needs index collected in the 2005 census, the 43 selected municipalities had a poverty rate of 59.4 percent, compared to a rate of 29.3 percent in the non-targeted municipalities.17 The latest available data show improvements in the national poverty rate since 2005; using an income-based measure, the percentage of the national population living in poverty was reduced from 27 percent in 2005 to 13 percent 2014.18 In addition, the 43 selected municipalities had a net enrollment rate of 43 percent compared to 67 percent in the other municipalities for preschool and a net enrollment rate of 24 percent compared to 55 percent at the lower secondary level. In 2017, the 43 selected municipalities had a net enrollment rate of 50 percent compared to 79 percent in the other municipalities for preschool and a net enrollment rate of 28 percent compared to 60 percent at the lower secondary level. The rate of growth in the targeted and non-targeted municipalities remain similar, presumably reflecting the benefits of the nationwide activities, as well as other projects in these areas at the time and other factors outside the Project’s control. Nonetheless, there is a positive trend in the rate of enrollment in the poorest areas and across the nation, contributing to the goals of reducing poverty and sharing prosperity. 46. In addition, the Project provided quality inputs that benefited the poorest as well. In particular, community preschools, which are attended by the poorest students, previously had a different, lower- quality curriculum. Under the Project, a unified preschool curriculum was developed and for the first time applied nationwide, and as a result, the poorest children also benefit from improved education from an 16 RAAS: South Atlantic Autonomous Region (Región Autónoma Atlántico Sur) 17 The 2005 census informed the original targeting under the Project. Due to delays in the 2015 national census, the Unsatisfied Basic Needs Index has not been updated and no data is available post-Project. 18 The income-based measure of poverty considers income of $3.20 per person per day. It is not available by municipality. Page 18 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) early age. The targeted municipalities also included areas with indigenous and Afro populations, who are among the poorest in Nicaragua. In addition to infrastructure, culturally and linguistically appropriate curriculum and learning materials were developed specifically for these areas. Other Unintended Outcomes and Impacts 47. Although the total number of schools receiving intervention declined as a result of the change in the infrastructure strategy, the quality of the intervention improved. Due to the comprehensive approach taken, as opposed to focused intervention in the targeted classrooms, the whole school benefitted, including the primary students attending these institutions. In addition, in an improved overall environment, the individual works are more sustainable over time, because the classroom investments were safeguarded by security installations and accompanied by maintenance plans. 48. While the original Project design included linguistic and cultural adaptation of the secondary textbooks for the Intercultural Bilingual Secondary Education in the Caribbean Coast region for indigenous populations and Afro descendants, during Project implementation, it was determined that a new updated curriculum was also necessary to ensure education quality for these populations. As such, MINED developed curriculum materials including 30 study programs, 12 textbooks for grades 7–9, and 15 programs and methodological guides in 5 indigenous languages (Miskitu, Kriol, Tuahka, Panamahka, and Ulwa). These will be printed and distributed as part of the ACE Project during 2018. The Project also generated other positive impacts on indigenous populations that were not anticipated, for example, the construction of a school in an indigenous community under the Project led MINED to build 4 other schools in indigenous communities. 49. Another unintended outcome was related to the certification of secondary school teachers. While the activity was financed fully by the Project, MINED seized on the opportunity to expand the training’s reach to additional beneficiaries from all over the country by using interinstitutional agreements with universities to cover the payment of professors that provided the training. The GoN also added its own funds to increase the number of teachers benefited, from the targeted 750 to 969. Furthermore, many of the teachers went beyond certification and obtained university degrees. 50. Finally, the Project had an unintended positive outcome of encouraging and increasing coordination between donors and between the GoN and donors in Nicaragua. The Project helped increase dialogue and ensure that the Bank, GPE, EU, and the GoN were working together toward the same goals. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 51. The Project was thoroughly prepared with relevant objectives and sound technical design. A key factor during preparation that positively affected implementation was that the Project was prepared in line with GoN priorities and responded directly to the ESS, ensuring ownership of the Project. In addition, the Project was designed to use existing institutional mechanisms and build on previous and adjacent operations, to increase success of implementation and in turn attainment of outcomes. Page 19 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 52. Another key factor during preparation that later affected implementation was a design issue in the infrastructure component. The original design was based on sample assessments regarding the needs of schools, and the investment per school had to be revised upward significantly during implementation. The design also did not fully consider the low infrastructure management capacities of the implementing agency, because the high volume of works overwhelmed the available staff. Another issue relates to the Indigenous Peoples Plan (IPP), which was prepared using the IPP of PASEN II as a template and was not adequately updated, causing some delays. 53. Finally, another key factor during preparation that affected implementation was that the administrative arrangements between donors were not fully defined by the time of Project approval. The Project was approved by the Bank on April 19, 2013, and the GPE grant became effective on May 2, 2013, while the EU grant did not become effective until September 19, 2013. This was due to differing legal language standards between the WB and the EU, which led to lengthy negotiations to settle the terms of the agreements. B. KEY FACTORS DURING IMPLEMENTATION 54. As identified above, one of the key factors that affected implementation on the donor side was the delay in the signing of the EU Administration Agreement. This delayed the start of both GPE- and EU- financed activities, because the team waited to launch all the operational processes at the same time. To compensate for this delay, the Bank strengthened the team’s presence on the ground to ensure the Project performed satisfactorily. To manage the Project, a Joint Steering committee, composed of MINED, EU, GPE, and the Bank was established and held semiannual meetings to ensure smooth collaboration. Regular coordination with the EU and GPE was crucial for joint decision making in key issues, such as changes in the infrastructure strategy. 55. A set of key factors with regard to the implementing agency’s capacity for fiduciary management affected implementation, especially school infrastructure related activities. MINED had high staff turnover and periods of low capacity in procurement and infrastructure, especially with regard to preparation of pre-investment studies, complying with safeguards policies, and supervision of works execution. These issues were compounded by the high volume of infrastructure works managed by MINED, both with this Project and PASEN II. Furthermore, in September 2014, the GoN carried out an organizational review of the Procurement and Infrastructure Departments of MINED to improve internal efficiencies. Following the review, key positions within these departments, including several mandated by the Legal Agreements, were left vacant. As a result, procurement processes suffered, with no new civil works processes launched during this time and several infrastructure interventions suspended due to delays in payments. 56. MINED and the Bank team worked together during this period to reestablish fiduciary controls as soon as possible. The teams built a comprehensive roadmap to tackle all existing bottlenecks and designed a revised timetable for the completion of delayed Project activities. Over the following year, Project management arrangements were enhanced by MINED, including filling key positions in procurement and infrastructure as required by the Legal Agreement. The Bank followed up closely, conducting targeted training and providing technical assistance on procurement, FM, and infrastructure. In particular, the Bank provided a simplified template for the pre-investment studies, and the quality of documents for environmental management improved markedly. Page 20 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 57. An external factor that slowed implementation was related to the local market for civil works. Specifically, few bids for certain infrastructure works were received. This was due to a lack of firms in the market capable of addressing the number of bids and few companies eligible according to Bank procurement rules, and in some cases, the distances and difficulty to access school sites, and the associated higher costs, reduced the number of firms willing to bid. MINED responded to this situation by re-launching multiple bidding processes throughout the year. An additional challenge arose when one of the firms that had been chosen to carry out several works was found to have provided falsified guarantees during the selection process. As a result, the GoN and the Bank agreed to halt the 7 works being carried out by the firm, which were at 50 percent completion at the time. New firms were selected to finish the works, and the works were completed on time. 58. Finally, other key factors that affected implementation were changes in exchange rate, cost of works, and weather issues, which were outside the GoN and Bank control. The depreciation of the euro versus the U.S. dollar resulted in a reduction in funds available and a reduction in the number of schools that could be built. The cost of works increased as salaries, transportation, and price of materials rose from the time of appraisal to implementation. The rainy season complicated and delayed the infrastructure components, particularly when other delays affected the planned schedule. MINED and the Bank team assessed each situation and made changes to activity implementation and schedules accordingly, including through formal restructuring as needed. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 59. The Project’s M&E design was clear and well defined. The objectives were clearly specified, and each outcome in the PDO statement was supported by a PDO indicator. The PDO indicators, in turn, were supported by intermediate results indicators adequate to capture the contribution of the Project activities toward the Project objectives. The indicators were specific, measurable, achievable, relevant, and time- bound, and baselines and targets were determined for all indicators at the preparation stage. The M&E arrangements were clearly identified and integrated into existing institutional arrangements with the Division of M&E at MINED responsible overall for tracking progress related to Project activities. The Division of Educational Statistics would be responsible for data collection regarding enrollment, repetition, dropout and retention. The Directorates for Preschool Education, Secondary Education, Teacher Training and Educational Infrastructure would be responsible for providing implementation monitoring data for their respective activities. Data would be collected and analyzed on a continuous basis, with implementation progress formally measured every six months. 60. There was one minor shortcoming in the M&E design, related to the PDO outcome “improve preschool education learning conditions nationwide.� This objective, as stated, was measured adequately through four related outputs concerning learning conditions in the classroom, which were directly linked to Project actives. However, the outcome itself is more of an output, and its measurement a collection of outputs rather than measurement of an outcome. This reflects the difficulty of quality measurement at the preschool level in general, the three-year time horizon planned at Project design, where quality would Page 21 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) not be expected to be affected, as well as the available data in the country at the time of design. Nonetheless, ensuring the availability of quality input and outputs is a precondition for quality and better outcomes, which the Project successfully accomplished. Moreover, the Project supported the development and piloting of an integrated ECD M&E system and implementation of the system in selected municipalities, to address this gap and increase the ability to measure child development outcomes at the preschool level going forward. Other indicators had minor shortcomings in terms of definition, baseline, or targets and required minor tweaks, but these did not alter their ability to measure their intended outputs/outcomes and were addressed appropriately throughout implementation as more information became available (see M&E implementation below). M&E Implementation 61. The PDO and intermediate indicators were measured and reported on during Project implementation as designed. MINED demonstrated adequate capacity and collected data routinely and reported accordingly in the biannual Project implementation reports. MINED also provided a narrative midterm report that was sent to the EU, and annual reports were sent to the GPE. The mechanisms used to collect data were appropriate, and the data were, in general, found to be reliable and of good quality. The Project also benefitted from a coordinated approach with PASEN II, which financed activities to strengthen the statistical capacity of the Divisions for Planning, Educational Statistics, and Institutional M&E under MINED. Given the embedded nature of the M&E arrangements in the daily functioning of MINED, these processes will be sustained after Project closing. M&E was rated Moderately Satisfactory or Satisfactory throughout Project implementation. 62. The Project actively addressed minor shortcomings in some of the indicators during implementation, formally updating as needed during a restructuring in February 2016 (see Significant Changes in Implementation section). Changes were made in terms of definition, baseline, and targets, based on new data available and other updated information. In particular, the infrastructure component targets were revised significantly, although this change reflects a preparation issue rather than an issue with M&E itself. The changes did not significantly alter the meaning of the indicators or inhibit their ability to measure the outputs or outcomes as intended. 63. There was one moderate shortcoming in the implementation of M&E, related to the measurement of PDO indicator 5 ‘Percentage of grade 9 students achieving intermediate, advanced and above proficiency level in standardized evaluations for Spanish and Mathematics.� As noted above, the results of the assessment that this indicator was based on could not be used due to concerns regarding the quality of questions on the test overall and the reliability of data in the math section. As such, the Project could not measure achievement toward the objective of improvement of quality at the lower secondary level as intended. The indicator was not updated during implementation because it had been included as part of the Legal Agreement between the EU and GoN, which was signed prior to Bank involvement, and could therefore not be altered. MINED is currently working on an improvement on the test so that this can be measured in the future. 64. Finally, the Project experienced shortcomings in terms of coordination and monitoring of the delivery of furniture and learning materials to schools. As the procurement processes for school infrastructure and furniture were separate, delivery was uncoordinated: in some cases, infrastructure was ready but the furniture was delayed, and in others, the furniture was delivered before the school was Page 22 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ready. To handle this, additional steps to ensure storage were required. With regard to learning materials, the contracted firm was required to deliver the materials to the municipal delegation, which in turn was responsible for coordinating distribution to the schools using their own funds. As such, MINED did not have direct access to verify whether the final delivery had taken place. Following requests from the Bank, MINED implemented a new verification mechanism, in which all municipalities would send confirmation of delivery directly to the Project’s M&E Unit, supplemented with physical monitoring missions to ensure that deliveries were complete. MINED and the Bank developed formats for reporting regularly on the status of deliveries, with data at the school level, which continues after Project close. MINED has also requested technical assistance from the United Nations Office for Project Services (UNOPS), financed by the PASEN II Project, to support the review and improvement of current internal processes. M&E Utilization 65. MINED and the Bank routinely assessed and used Project data to inform implementation and decision making. For example, the National Student Learning Assessment 2015 (PDO Indicator 5) results were used to inform improvements in teacher training. M&E data were used to provide evidence of achievement of outcomes through PDO Indicators 1, 2, 3, and 4, in addition to evidence of achievement of outputs through the intermediate results indicators. The GoN used Project data to engage with stakeholders, through biannual meetings to present the Project’s progress to the donors group. M&E findings were also used to inform subsequent interventions, for example, under the ACE Project, the construction firm that is responsible for school infrastructure will also provide furniture to avoid coordination issues and a detailed strategy for the distribution of learning materials is in place to ensure timely delivery to schools. The Project also coordinated with PASEN II to create sustainable systems to support M&E. Finally, the development and piloting of the integrated ECD M&E system supported under the Project strengthened institutional capacities to measure and monitor ECD and enabled the use of feedback loops to make improvements and increase quality in the preschool sector. The development of this instrument also served as the foundation for the development of a second instrument, the preschool teaching practices observation instrument, which will be finalized during 2018 and will inform in-service preschool teacher training. Justification of Overall Rating of Quality of M&E 66. The design, implementation, and utilization of the Project’s M&E system was suitable and adequate to assess the achievement of the objectives. The M&E findings were disseminated and used to inform the direction of the Project, as well as and future engagements. Given the moderate shortcomings noted above, the overall quality of M&E is rated Substantial. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 67. Environmental safeguards. The Project was rated Category ‘B’ because it involved a large number of small civil works spread across the country that were not expected to generate significant environmental impact. The Project triggered the policy on Environmental Assessment (OP/BP 4.01) to address the potential environmental and social impacts related to site selection, construction, and operation of the school facilities. An Environmental Management Framework (EMF) was prepared to screen, assess, and mitigate environmental and social impacts of Project activities. Given that Central America is a natural disaster–prone area, school infrastructure was designed to withstand natural Page 23 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) disasters and provide adequate shelter in the case of such events. A rigorous monitoring and quality assurance procedure was included in the contracts to ensure compliance with environmental health and safety standards as outlined in the EMF. Pre-investment studies for civil works were carried out in accordance with Bank policies and included appropriate environmental and safety considerations, although with some delay. Nonetheless, during the first half of the Project, the EMF was not properly implemented due to weaknesses in the instruments used during the pre-investment phase. This led to a reorganization of the Bank technical team and revision of the instruments, which contributed to delays in Project execution. Through an action plan agreed with MINED and the Bank, which included clear lines of responsibility and monitoring, performance was reestablished. As such, overall, the Project complied with the environmental safeguard policies. 68. Social safeguards. The Project triggered OP/BP 4.10 - Indigenous Peoples. The Project would benefit Indigenous Peoples (IPs) living in the North Atlantic Autonomous Region, RAAS, the North Pacific and Center Regions, the Afro-descendant populations living in Rio San Juan, and mestizo communities across the country. An IPP was prepared to ensure that IPs were not adversely affected by the Project and received its social and economic benefits in a suitable sociocultural way. MINED complied with the actions specified in the IPP, such as appropriate and inclusive distribution of education materials, capacity building on pre-school evaluation, and teacher training. However, during implementation, it was discovered that certain actions included in the IPP were not aligned with the Project's components. This was because the IPP was based on the IPP for PASEN II and had not been updated adequately (that is, it referred to printing of materials in indigenous languages that were not included in the Project, but rather took place under PASEN II). Despite these weaknesses in IPP design, the GoN went above and beyond the plan to benefit IPs. In addition to the provision of adapted secondary school textbooks for the Intercultural Bilingual Secondary Education in the Caribbean Coast, the Project also supported the development of a new adapted curriculum to provide full integration with the materials to ensure education quality for Afro descendants and indigenous populations (see section on Other Unintended Outcomes and Impacts). 69. The Project triggered OP/BP - 4.12 Involuntary Resettlement. A Resettlement Policy Framework was prepared to address issues related to land acquisition, potential displacement, or restriction of access to resources. During project implementation, it was determined that the school infrastructure works would require no permanent displacement of the population before, during or after execution, given that the civil works only entailed the temporary relocation of students. During the first half of the Project, there were some weaknesses in the Resettlement Management Plans. The plans were subsequently adjusted appropriately and a management system that delineated responsibilities was put in place, restoring performance. MINED developed an internal instrument, the Temporary Relocation Plan (TRP), to effectively and satisfactorily address relocation of students under the Project. This instrument has been institutionalized and is now applied to other construction projects undertaken by MINED. MINED also prepared a comprehensive systematization on how TRPs were implemented. MINED took appropriate measures to address all concerns raised by the Bank during field visits to construction sites (i.e. student access to latrines and small classroom spaces). Finally, a team focused on social safeguards has been created in MINED’s Infrastructure Unit, using national funds, which oversees the implementation of TRPs and addresses other social issues at construction sites. MINED has improved its institutional capacity and expertise with social safeguards during implementation. Overall, the Project complied with all social safeguard policies. Page 24 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 70. Financial Management (FM). The Project had adequate FM arrangements in general, which provided reasonable assurance that grant proceeds were used for the intended purposes and FM requirements were complied with throughout most of the implementation period. MINED’s Administrative and Financial General Division was responsible for the FM arrangements of the Project. It consolidated the financial registration in the Integrated Financial Management System and performed reconciliation of the accounts. This unit had experience managing Bank funds with a previous Project, PASEN I. The fiduciary environment was bolstered by the recruitment of a senior FM specialist, two accountants, and six controllers, as well as training of key staff. However, FM was temporarily weakened following the 2014 reorganization. The FM unit had several legally mandated positions vacant and experienced delays in consultant payments and submission of Statement of Expenses (SOEs) to the Bank and delays and qualified opinions of external audit reports. After the reorganization, MINED filled the required FM positions and improved timeliness of SOEs, and although issues with timely submission of external audits continued, MINED was proactive in complying with an action plan agreed to improve the quality of audits. In addition, the lack of adequate financial software remained an issue, because the software used did not allow collection, processing, and reporting of information in a timely and robust manner. The Bank recommended that MINED switch to the use of a financial data management system that allows the integration of data related to budget, accounting, financial execution, and financial reports. The Project Implementation Status and Results Reports (ISRs) reflected a decrease in rating to Moderately Unsatisfactory in 2014 due to these issues, before being upgraded to Moderately Satisfactory in 2015, in line with the performance improvements, for the remainder of the Project. 71. Procurement. The Project procurement arrangements and plans were adequate throughout most of the implementation period. Procurement was carried out by MINED’s Procurement Division. The unit was generally adequately staffed and equipped to undertake the standard procurement activities routinely carried out by the division. Under PASEN II, MINED hired three additional procurement analysts to carry out procurement functions for both PASEN II and this Project. The Bank provided direct training to the analysts and MINED procurement staff and contributed to quality control. Under the Project, three additional procurement analysts were hired. The implementing unit had experience on procurement aspects and Bank procedures, as it was responsible for the implementation of three preceding operations. Nonetheless, there were delays related to weak implementation capacity in comparison to the high workload and complexity of procurement to be carried out for Project implementation. Procurement was particularly hampered following the 2015 reorganization. The unit suffered from vacant legally mandated positions and out-of-date Procurement Plans and committed some errors during the bidding process. After the reorganization, MINED filled the required positions, improved Procurement Plans, conducted an independent procurement review, and received Bank training. As a result, there was an initial downgrade in rating to Unsatisfactory in 2014 and 2015, before being upgraded to Moderately Satisfactory and then Satisfactory, in line with the performance improvements for the remainder of the Project. C. BANK PERFORMANCE Quality at Entry 72. Quality at entry was satisfactory. The Bank facilitated preparation and appraised the Project adequately to achieve the planned development outcomes, in close collaboration with the GoN and the donors. The objectives were clear and strongly aligned with the national development and education strategies and consistent with adjacent operations. The Project adequately addressed economic and Page 25 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) financial issues and provided due diligence on aspects such as gender and poverty, in line with Bank priorities. At approval, the Project was ready for implementation from a technical perspective, and implementation arrangements, including M&E, FM, and procurement, were clear and well defined. 73. The Project’s technical design was partially conditioned by the content of the Legal Agreement between the EU and GoN, which was signed prior to Bank involvement. Given that EU regulations do not allow changes that could modify the substance of the Administration Agreement, there were certain design issues (such as the definition of PDO Indicator 5) that were detected after the agreement and could not be changed or updated during the Bank’s preparation (or implementation). A related minor shortcoming is the delays caused by the lengthy process to sign the EU–WB Administration Agreement. 74. Another shortcoming in quality at entry was the weakness described concerning the low estimates for infrastructure costs, as well as an overestimation of the infrastructure management capacity of the implementing agency. Finally, there were some shortcomings in the design of the IPP (see Key Factors during Preparation for details). Quality of Supervision 75. The quality of supervision was satisfactory. The Bank worked proactively with the GoN and the donors throughout implementation with a focus on achieving development impact and was commended by both for effectiveness, responsiveness, and flexibility (see annex 5, Borrower, Co-Financier and Other Partner/Stakeholder Comments). To improve availability of key staff, the leadership from the Bank side was transferred from headquarters to a specialist based in country, through a hand over mission in July 2015. The country team was further strengthened by hiring a local Infrastructure Specialist to provide constant feedback to the infrastructure unit, and a local Project Implementation Specialist to provide guidance and supervision to the client’s M&E unit. To ensure timely and close follow-up, the team held monthly meetings at the highest level, weekly executive meetings, and follow-up meetings with implementation units and organized missions to provide targeted technical assistance and training as needed. The quality of performance reporting was high, with the team reporting regularly, candidly, and comprehensively through Aide Memoires and biannual implementation reports. The team altered and substantiated Bank ratings as required to respond to issues during implementation, in communication and cooperation with both the Borrower and the Bank management. Progress and challenges were transparently recorded to ensure all parties were on the same page and to focus dialogue on the pertinent issues and reaching solutions to achieve development outcomes. As a result, restructurings were sought and executed to amend the Project as needed. Finally, the Bank worked through the appropriate country channels at all times and provided capacity building as needed to ensure adequate transition of supported activities at the time of Project closing. 76. One minor shortcoming pertains to the turnover of Bank staff. Although strong support was offered by the Bank teams, there were shortfalls in the transition periods. The changes among the Bank team, including the Task Team Leader, as well as experts in the social, environmental, infrastructure, and procurement units, led the Borrower to feel a lack of continuity of the advisory services offered, which in some cases led to delays (see annex 5, Borrower, Co-Financier and Other Partner/Stakeholder Comments). As noted, the quality of supervision was increased when the new team was in place by mid- 2015. Page 26 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 77. Another minor shortcoming pertained to the closing date extensions, which were processed four separate times. Although the extensions themselves were justified, extending the Project at one time would have avoided the repeated costs of the administrative work involved. The piecemeal approach was due to information availability at the time, as well as donor processes. In particular, in December 2015, the GoN asked for an 18-month extension for the whole Project. The EU complied, but the GPE was unable to extend more than 12 months, due to internal regulations. Nonetheless, the Bank, GPE, and EU showed flexibility and innovation and agreed on a cross-financing mechanism where GPE funds would be used to pay for schools completed before the GPE closing date and the EU financed activities compensating GPE works later. In addition, the team continued to work with the GPE on the extension itself, and several months later, the GPE agreed to extend the closing date of the GPE trust fund in line with the EU fund, to allow the client to conclude the execution of infrastructure works. Justification of Overall Rating of Bank Performance 78. The services provided by the Bank ensured both quality at entry and quality of supervision. Given the minor shortcomings detailed above, the Bank’s overall performance is considered Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 79. The risk that development outcomes achieved under this Project might not be sustained is considered Low. Project priorities are fully owned by the GoN and are aligned with the 2017–2021 ESS and upcoming National Development Plan. Moreover, to ensure Project continuity, the GoN agreed to finance future recurrent costs including salaries, material replacement, and infrastructure maintenance. 80. The Project supported sustainability by ensuring activities were implemented through MINED’s existing organizational structure and line departments, to build capacity for sustained efforts and avoid transition shocks at the end of the Project. Improvements in learning outcomes are likely to be sustained given lasting effects of key activities, such as the use of the preschool curriculum and development of improved learning materials, as well as repeated activities such as teacher training and distribution of learning materials, which will be sustained by the GoN. 81. In addition, the Project bolstered sustainability of infrastructure investment by (a) ensuring MINED’s legal ownership of schools targeted under the Project; (b) using long lasting school furniture and equipment; and (c) carrying out training courses and communication campaigns focused on the use and maintenance of school infrastructure to MINED regional delegates, school staff, students, local communities, and parent organizations. The Project financed the creation of a school maintenance guide, a booklet for school communities with a list of recommendations and awareness materials, which is used to provide training on school maintenance nationwide, as part of a GoN school maintenance program. This guide was distributed to all schools financed by the Project, along with a training on the guide during 2017. In addition, all civil works were built to withstand natural disasters to avoid risks of loss of infrastructure investments due to weather events. Furthermore, the social safeguard guidelines used in this Project, such as the TRP, have been included as part of MINED’s processes for working with communities on school infrastructure. As an example, MINED hired social safeguards staff for their infrastructure team with national funds. Finally, the Project will have positive effects through ongoing activities, such as the ECD M&E system, which will help inform MINED efforts in the sector going forward. Page 27 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 82. In addition, a follow-up operation, the ACE Project, is already being implemented and includes interventions to improve teaching practices, enhance physical learning environment, and strengthen the technical and fiduciary capacity of MINED, in particular, by supporting the development of an infrastructure management system. These activities build directly on the Project’s activities, were developed in coordination with the current Project, and will ultimately help sustain gains made under the Project. V. LESSONS AND RECOMMENDATIONS 83. Multiple donors add complexity that can delay implementation. Combining different financing sources can reduce transaction costs in terms of preparing and managing only one project. However, there are inherent complexities in having more than one donor, because this multiplies the operational requirements (i.e. having to adhere to multiple Legal Agreements, prepare multiple Procurement Plans, conduct multiple audits, and report to multiple entities for one project). This is increased when there are disparate processes for each donor. Extended negotiations of the WB-EU Administrative Agreement caused delays in the start of implementation. Lengthy processes to obtain a project extension from the GPE caused delays during implementation. This adds extra work to both the Borrower and the Bank, increases cost, and delays work on the ground. Simplifying and harmonizing donor processes can increase efficiency of projects and put the donors, and the Bank, in a better place to assist clients. Further, engaging early with the donors during the design stage of Project preparation can smooth implementation. 84. Aligning Project objectives with government priorities and using institutional mechanisms for implementation can ensure ownership and support sustainability. The Project was aligned with the main objectives of the 2011–2015 ESS. The ESS prioritizes education quality and places teacher quality at the center, with the support of improved teaching practices within the classroom. Project activities were aligned with this priority, ensuring strong buy-in from all actors on the ground during the Project and after its closing. Furthermore, activities were managed and implemented by MINED through existing organizational structures (rather than a separate implementation unit). Relying on its own institutional arrangements creates stronger ownership and internal coordination and supports sustainability after Project close, as procedures are internalized and capacity is built within the relevant agencies. 85. Focused scope, in line with available capacity and supported by appropriate staffing, is critical to Project implementation. The multitude of activities included in the Project generated implementation challenges and difficulties to adequately supervise the work of many line directorates. A narrower scope could ease implementation by involving fewer technical teams with more integration. Furthermore, the Borrower needs to ensure that key Project management positions, i.e. procurement, FM, and safeguards, are continuously filled with appropriate staff. The Bank can support this through appropriate covenants, prompt supervision, and technical advice as needed. The Bank must also ensure smooth transitions during changes in team composition to avoid disruption in implementation. The new ACE Project has already taken this into consideration and focuses more narrowly on one item, specifically, strengthening teaching practices, and has ensured appropriate staff and continuous support are available. 86. Adequate attention to design can prevent delays in implementation. Weaknesses in design of safeguards instruments such as IPPs and Resettlement Plans can seem peripheral to main activities, but have a significant effect on implementation. Inaccurate preparation work, such as assessment of Page 28 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) infrastructure cost, has the same effect. Investing greater time and resources during preparation can prevent delays down the road. 87. Quality measures in the Results Framework need careful consideration. Using national student assessments to measure education quality can be problematic, because countries are often changing/improving the assessments, leading to data that may not be comparable to previous years. Instead, or in addition, international measures could be used where possible, and backup indicators identified preemptively to demonstrate efficacy if needed. . Page 29 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increase Access to Preschool Education in Selected Municipalities Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Gross enrollment rate for Percentage 40.00 48.00 44.00 50.00 preschool in selected municipalities. 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 Comments (achievements against targets): Target surpassed (250%). Original Baseline value: 41.7% Original Target: 48% Achievement against original baseline/target: 132% Revised Baseline value: 40% Revised Target: 44% Achievement against revised baseline/target: 250% Targets and baselines were revised in February 2016 to reflect updated school year data (2013) and updated population projections (2015). However, notably, both revised target and original target were surpassed significantly (250% and 132%), suggesting high success of the project in attracting and accommodating a higher percentage of preschool students than anticipated. The target was not adjusted upwards throughout implementation as the increase came in final year of the project. Objective/Outcome: Increase Access to Lower Secondary Education in Selected Municipalities. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 30 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Gross enrollment rate for Percentage 47.00 54.00 57.20 lower secondary (grades 7-9) in selected municipalities 15-May-2013 30-Apr-2016 15-Nov-2017 Comments (achievements against targets): Target surpassed (146%). Original Baseline value: 48.6% Original Target: 54% Achievement using original baseline: 146% Revised Baseline value: 47% Achievement against revised baseline: 159% Baselines were revised in February 2016 to reflect updated school year data based on 2013 figures. Given steady progress, the target remained the same, and was significantly surpassed considering both original and revised baselines (146% and 159%), suggesting high success of the project in attracting and accommodating a higher percentage of lower secondary students than anticipated. The target was not adjusted upwards throughout implementation as the largest increase came in final year of the project. Objective/Outcome: Improve Preschool Education Learning Conditions Nationwide Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Learning conditions improved Text No Yes Yes in preschool. 15-May-2013 30-Apr-2016 15-Nov-2017 Comments (achievements against targets): Target Achieved. Learning conditions would be considered effectively improved if 50% of the preschools nationwide have at least 3 out of the 4 conditions described below: i) Preschool teachers trained in the use of the unified curriculum ii) Revised classroom learning instruments are available in preschools, iii) Preschool teachers are certified iv) Preschools have received a package of learning materials. At project end, 62% of preschools met the criteria. Objective/Outcome: Improve Lower Secondary Education Completion Rates Nationwide Page 31 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Grade 9 completion rate. Percentage 64.00 57.00 71.00 71.90 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 Comments (achievements against targets): Target surpassed (113%) The target for this indicator was surpassed, suggesting high success of the project in retaining students. The definition of this indicator was updated in February 2016 to reflect conventional WB/UNESCO methodology, and baselines and targets were updated accordingly (to 64% to 71% respectively). As the methodology changed, achievement using the original baseline (52%) and target (57%) are not calculated. Objective/Outcome: Improve Lower Secondary Education Quality Nationwide Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of Grade 9 Percentage 75.00 66.00 77.00 0.00 students achieving intermediate, advanced or 31-Dec-2010 30-Apr-2016 24-Feb-2016 15-Nov-2017 above proficiency levels standardized evaluations for Spanish Language. Comments (achievements against targets): The baseline was measured through the National Student Learning Assessment carried out in 2010, with the expectation that the figures would be updated when the test was carried out in 2015. However, the results of the assessment could not be used due to concerns regarding the quality of questions on the test overall and the reliability of data in the math section. As such, this indicator cannot be measured. The indicator was not updated as this information was only available at the end of the project. Page 32 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of Grade 9 Percentage 51.00 53.00 0.00 students achieving intermediate, advanced or 31-Dec-2010 30-Apr-2016 15-Nov-2017 above proficiency levels standardized evaluations for Mathematics Comments (achievements against targets): The baseline was measured through the National Student Learning Assessment carried out in 2010, with the expectation that the figures would be updated when the test was carried out in 2015. However, the results of the assessment could not be used due to concerns regarding the quality of questions on the test overall and the reliability of data in the math section. As such, this indicator cannot be measured. The indicator was not updated as this information was only available at the end of the project. A.2 Intermediate Results Indicators Component: Increasing Access and Improving Learning Conditions of Preschool Education Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of preschool Number 0.00 200.00 106.00 110.00 classrooms built/rehabilitated and 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 equipped in selected municipalities Comments (achievements against targets): Target surpassed (104%). Page 33 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) The target was surpassed as 110 classrooms were built/rehabilitated against a revised target of 106. The target was revised downwards in February 2016 (from 200 classrooms to 106) to reflect a change in Government strategy that included increased complementary investments per school (in terms of water & sanitation, security, recreational facilities, and electricity) as well as an increase in the price of materials compared to original estimates and a loss in the exchange rate of Euros to Dollars. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Preschool unified curriculum Text No Yes Yes developed 15-May-2013 30-Apr-2016 15-Nov-2017 Comments (achievements against targets): Target achieved. One curricular framework and three preschool classroom instruction guides were integrated in one unified preschool curriculum approved by the Executive Directorate of MINED and is available on the MINED website. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of certified Number 0.00 2700.00 2300.00 2321.00 preschool teachers (Nationwide). 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 Comments (achievements against targets): Target surpassed (101%). The target was revised downwards in February 2016 from 2700 to 2300 to reflect the increased investment per teacher due to higher cost of participation than expected, in particular with regards to transportation for teachers from remote areas such as the Caribbean coast. The original estimate at design stage was made using an average participation cost for the whole country. Page 34 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of preschools that Number 0.00 14000.00 8500.00 8903.00 receive learning material packages 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 Comments (achievements against targets): Target surpassed (105%). The target was revised in February 2016 to align with updated data on number of preschools, and the new target exceeded, with 8,903 preschools receiving materials against a target of 8,500, which accounts for all the preschools in the country. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion System to evaluate early Number 0.00 2.00 2.00 2.00 childhood development outcomes and preeschool- 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 age learning outcomes in place and operational Evaluation system designed Number 0.00 1.00 1.00 1.00 and piloted 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 System validation Number 0.00 1.00 1.00 1.00 conducted in a sample of the selected municipalities, 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 and results published Page 35 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Comments (achievements against targets): Target achieved. A system to evaluate early childhood development outcomes in preschool and preschool-age learning outcomes is designed and piloted, and system validation was conducted in a sample that included a group from the 43 selected municipalities, and covered a wide variety of contexts and regions, both in rural as well as urban areas, and the results were published. Component: Improving Access, Quality and Completion of Lower Secondary Education Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of core schools (K-9) Number 0.00 200.00 75.00 74.00 with facilities improved for lower secondary education. 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 Comments (achievements against targets): Target substantially achieved (99%). The achievement is slightly below target as 74 schools were built/rehabilitated against a revised target of 75, due to the cancelation of building one school that received no eligible offers after multiple bidding processes. The target was revised downwards in February 2016 (from 200 schools to 75) to reflect a change in Government strategy that included increased complementary investments per school (in terms of water & sanitation, security, recreational facilities, and electricity) as well as an increase in the price of materials compared to original estimates and a loss in the exchange rate of Euros to Dollars. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion A university-based Text No 900 Student Teachers 750 Student Teachers 969 Student Teachers certification program for certified Certified Certified Page 36 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) lower-secondary teachers 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 implemented Comments (achievements against targets): Target surpassed (129%). Baseline value: 0 Original Target: 900 Achievement against original target: 108% Baseline value: 0 Revised Target: 750 Achievement against revised target: 129.2% Targets were revised in February 2016 to reflect an expectation of higher dropouts in the university-based certification program based on trends. However, notably, both revised target and original target were surpassed (129% and 108%), possibly as a result of the Government providing its own funding to increase the training’s reach. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of secondary school Number 0.00 1640.00 1600.00 1529.00 teachers who participate in the 22-day in-service teacher 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 upgrading training based on the secondary curriculum. Comments (achievements against targets): Target substantially achieved (96%). Original Target: 1640 = 93% Revised Target: 1600 = 96% Achievement is slightly below target due to a higher dropout rate than was expected for the in-service training. The target was revised in February 2016 to align with updated data on number of teachers in the applicable subjects in the 43 targeted municipalities. Page 37 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of secondary Number 0.00 1100000.00 1500000.00 1703860.00 textbooks distributed. 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 Comments (achievements against targets): Target surpassed (114%). Original target: 1.1 million Achievement against original target:155% Revised target: 1.5 million Achievement against revised target: 114% The target was revised in February 2016 to align with updated data on number of students in public lower secondary schools (1.2 million) is higher than estimated (1.1 million). In addition the Project also provided: (i) about 200,000 textbooks in 5 subjects to all students in grades 10 and 11 in public upper secondary schools; (ii) 386,7000 textbooks to distance education students enrolled in secondary school living in rural areas; and (iii) about 10,500 textbooks in 5 subjects to all teachers in grades 7, 8, 9, 10 and 11 in public secondary schools. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of teaching guides Number 0.00 16300.00 22800.00 distributed to secondary school teachers 15-May-2013 30-Apr-2016 15-Nov-2017 Comments (achievements against targets): Target surpassed (140%). Target was surpassed with 22,800 teachers receiving guides against a target of 16,300, as the guides were distributed to all secondary teachers of public and semi-private schools nationwide. Page 38 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Component: Strengthening of the Ministry of Education’s Institutional Capacity for Preschool Education Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Directorate of Preschool Yes/No N Y Y Education is equipped for Project implementation and 15-May-2013 30-Apr-2016 15-Nov-2017 supervision activities Comments (achievements against targets): Target Achieved. Directorate of Preschool Education received equipment, including vehicle for supervision of Project activities and computers to develop curriculum and classroom guides. Component: Strengthening of the Ministry of Education’s Institutional Capacity for Lower Secondary Education Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion General Directorate of Number 0.00 3.00 3.00 School Infrastructure (GDSI) offices set up in two 15-May-2013 30-Apr-2016 15-Nov-2017 departmental education delegations and one territorial division of RAAS. Comments (achievements against targets): Target Achieved. Three GDSI offices were created in Boaco, Leon and Zelaya Central. Each office includes basic facilities for the GDSI to conduct pre-feasibility studies, as well as the monitoring and supervision of civil works contracted to individual consultants. Page 39 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Development of Yes/No N Y Y infrastructure maintenance guides/norms 15-May-2013 30-Apr-2016 15-Nov-2017 Comments (achievements against targets): Target Achieved. Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 569000.00 585000.00 562685.00 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 Female beneficiaries Percentage 0.00 50.00 50.00 51.00 15-May-2013 30-Apr-2016 24-Feb-2016 15-Nov-2017 Comments (achievements against targets): Target substantially achieved (96%). The target was revised in February 2016 to reflect a broader definition of project beneficiary, which included nationwide figures (rather then limited to 43 municipalities) and represented cumulative data. The final number is slightly lower than the target as it was based on estimates of enrollment in Feb 2016, which were slightly higher than actuals in 2017. Page 40 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) B. KEY OUTPUTS BY COMPONENT Objective 1: Increase Access to Preschool Education in Selected Municipalities Outcome Indicators Gross enrollment rate for preschool in selected municipalities Number of preschool classrooms built/rehabilitated and equipped in Intermediate Results Indicators selected municipalities Key Outputs by Component 110 preschool classrooms built/rehabilitated Objective 2: Increase Access to Lower Secondary Education in Selected Municipalities Gross enrollment rate for lower secondary (grades 7–9) in selected Outcome Indicators municipalities Number of core schools (K-9) with facilities improved for lower Intermediate Results Indicators secondary education Key Outputs by Component 74 schools with improved facilities for lower secondary education Objective 3: Improve Preschool Education Learning Conditions Nationwide Outcome Indicators Learning conditions improved in preschool • Preschool unified curriculum developed • Number of certified preschool teachers (Nationwide) • Number of preschools that receive learning material packages Intermediate Results Indicators • System to evaluate early childhood development outcomes and preschool-age learning outcomes in place and operational • Directorate of Preschool Education is equipped for Project implementation and supervision activities Page 41 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) • One curricular framework and three preschool classroom instruction guides integrated into unified preschool curriculum. • 2,321 certified preschool teachers. • 8,903 preschools received learning material packages, achieving 100% of revised target. Key Outputs by Component • System to evaluate ECD outcomes successfully designed and piloted, and results published from system validation sample. • Directorate of Preschool Education received equipment, including vehicle for supervision of Project activities and computers to develop curriculum and classroom guides. Objective 4: Improve Lower Secondary Education Completion Rates Nationwide Outcome Indicators Grade 9 completion rate. • A university-based certification program for lower-secondary teachers implemented • Number of secondary school teachers who participate in the 22- day in-service teacher upgrading training based on the secondary curriculum • Number of secondary textbooks distributed Intermediate Results Indicators • Number of teaching guides distributed to secondary school teachers • General Directorate of School Infrastructure (GDSI) offices set up in two departmental education delegations and one territorial division of RAAS • Development of infrastructure maintenance guides/norms • 969 Student teachers certified • 1,529 school teachers participated in 22-day in-service teacher Key Outputs by Component upgrading training • 1,703,860 secondary textbooks distributed Page 42 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) • 16,300 teaching guides distributed to secondary teachers • 3 GDSI offices created, matching original and revised target • Infrastructure maintenance guides/norms developed Objective 5: Improve Lower Secondary Education Quality Nationwide • Percentage of Grade 9 students achieving intermediate, advanced or above proficiency levels standardized evaluations for Spanish Language Outcome Indicators • Percentage of Grade 9 students achieving intermediate, advanced or above proficiency levels standardized evaluations for Mathematics Intermediate Results Indicators (Same as Objective 4) Key Outputs by Component (Same as Objective 4) Page 43 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Michael Drabble Task Team Leader(s) Miriam Montenegro Operation Specialist Patricia de la Fuente Hoyes Disbursement Enrique Román Financial Management Francisco Rodríguez Procurement Specialist Ignacio Jauregi Counsel Mary Lisbeth Gonzalez Social Safeguards Specialist Abdelazis Lagnaoui Environmental Safeguards Specialist Mary Dowling Team Assistant Supervision/ICR Enrique O. Alasino Massetti Task Team Leader(s) Juan Diego Alonso Task Team Leader(s) Patrick Ramanantoanina Task Team Leader(s) Carlos Lago Bouza Procurement Specialist(s) Enrique Antonio Roman Financial Management Specialist Dianna M. Pizarro Social Safeguards Specialist Erika Piber Social Safeguards Specialist Marco Antonio Zambrano Chavez Environmental Safeguards Specialist Pilar Elisa Gonzalez Rodriguez Counsel Norma Vida Malespin Ramirez Local Infrastructure Specialist Fernanda Balduino de Oliveira Finance Analyst Julio Daniel Martinez Education Consultant Tatiana Cristina O. de Abreu Souza Finance Officer Sandra Monica Tambucho Senior Finance Officer Linda Castillo Program Assistant Gabriel Barrientos Program Assistant Adriana Cecilia Espinal Saballos Program Assistant Page 44 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY13 17.022 139,037.96 FY14 2.888 51,832.58 FY18 0 705.13 Total 19.91 191,575.67 Supervision/ICR FY13 0 0.00 FY14 30.543 170,208.48 FY15 59.432 288,336.68 FY16 58.772 336,733.77 FY17 .847 16,976.02 FY18 0 14,711.87 Total 149.59 826,966.82 Page 45 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Closing Percentage of Approval Components (US$, millions) (US$, millions) (US$, millions) Increasing Access and Improving Learning 15.9 15.85 99.69 Conditions of Preschool Education Improving Access, Quality and Completion of Lower 33.5 30.69 91.61 Secondary Education Strengthening of the Ministry of Education’s Institutional 0.8 0.85 106.25 Capacity for Preschool Education Strengthening of the Ministry of Education’s Institutional 1.5 2.00 133.33 Capacity for Lower Secondary Education Total 51.7 49.39 95.53 Page 46 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ANNEX 4. EFFICIENCY ANALYSIS 1. Nicaragua has shown progress in preschool and lower secondary gross enrollment among the 43 selected municipalities covered and improvements in learning conditions in preschool and grade 9 completion rates nationwide. The project focused on preschool (ages 3–5) and lower secondary (ages 12– 14) in the rural areas of Nicaragua, where the poverty levels are the highest. In four components, the program offered infrastructure investment, development and alignment of a unified curriculum, various teacher trainings and certifications with supplemental learning materials for teachers and students, and institutional strengthening. The 43 municipalities were strategically selected due to their high poverty levels, illiteracy, and repetition and dropout rates. Of these municipalities, 25 were also targeted under PASEN II, helping with the continuity and vertical integration from preschool to primary to secondary education. Although the implementation experienced some delays causing an 18.5-month extension, almost all indicators exceeded the revised targets of the project. Preschool gross enrollment rate increased from 40 percent in 2013 to 50 percent in 2017, and lower secondary gross enrollment increased from 47 percent to 57.2 percent. Grade 9 completion rates showed an increase from 64 percent to 71.2 percent in the four years. 2. Component 1 targeted supply-side challenges by investing in the rehabilitation/construction of 110 classrooms, various teacher trainings and certifications, provision of learning materials, and development and alignment of a unified preschool curriculum. The infrastructure investment allowed access to over 65,000 rural preschool students with better teaching quality that results from the enhanced learning environments (Berhman, Ross, and Sabot 2008). In addition, a key approach to improving teaching quality is to develop and strengthen the skills of the existing teacher workforce (Kraft, Blazar, and Hogan 2016). Subcomponent 1.3 precisely aimed to increase teaching quality in this way by offering certifications to preschool teachers, as well as various trainings, workshops, demonstration classrooms, and intraschool teacher exchanges. To complement these teacher training programs, the Project also provided materials and training for their use in the classroom. Students who benefited from Component 1, therefore, will see benefits on their productivity observed as increased wages once in the labor market, as a result of the additional years of education and improved teaching quality (Card 1999). Moreover, Berlinski, Galiani, and Gertler (2009) estimate an 8 percent increase in student performance on grade 3 tests per additional year of pre-primary education gained by investing in infrastructure. Chetty et al. (2014) also show that a 1 standard deviation (SD) improvement in teacher value added is associated to a 1 percent increase in earnings at age 28. Assuming the ratio of gains between student achievement and earnings in instructional quality is the same as the one observed for value added, then a 1 SD change in instructional quality has an effect of 2 percent increase on earnings. 3. Component 2 focused on the improvement and construction of 512 classrooms in 74 multigrade schools, as well as teacher trainings and certifications and provision of learning materials. The construction, rehabilitation, and furnishing of the ICT laboratories and learning and complementary facilities in addition to the classrooms will allow access to education to over 66,000 lower secondary students in areas where this expansion of classrooms may be the only schooling option for the teenagers. Teachers play a crucial role on student achievement, retention, and repetition rates. In fact, Chetty, Friedman, and Rockoff (2014) found that teachers who have an impact on the learning of their students also have an impact on important long-term outcomes such as college attendance, earnings, and the probability of having children as teenagers. Component 2 certified 1,529 in-service educadoras and Page 47 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) trained them in the use of ICTs in the classroom, in all schools that received ICT labs under the Project. For secondary education, Araujo et al. (2016) found that student performance increases by 18 percent per 1 SD change in instructional quality. Assuming the same ratio gains between student achievement and earning as in Component 1, a 1 SD change in instructional quality translates into a 1.5 percent increase on earnings. 4. Components 3 and 4 had the goal of strengthening the preschool and lower secondary education institutional capacity of MINED. The main activities of these components were the technical development, assistance, and training for MINED teams and the provision of basic office equipment. These two components, as well as Subcomponent 1.5 on the development of an ECD M&E system, are harder to quantify in monetary terms and, consequently, were not part of the estimations of this analysis. The institutional measures improve long-term costs and time efficiency for the education system, and therefore, the overall results of the analysis are an underestimation of the benefits achieved and the reach of the Project. There are many beneficiaries of this type of intervention including MINED staff at different levels of the education systems as well as, in the long term, students nationwide. 5. The economic analysis estimates the cost-benefit ratio of monetary benefits deriving from the increase in education quality and additional years of education expected from the Project to be 5.63 with an IRR of 7.21 percent. There are important individual gains from investing in education; nonetheless, many of them were not captured in this analysis, and therefore, this estimation is to be considered a conservative estimation of the benefits of the Project due to data restrictions. The model works with the following assumptions: (a) 2016 GDP per capita (US$2,151) is used as a proxy for average earnings given the high levels of informality in the labor market in rural Nicaragua. (b) Students who benefited from the project enter the workforce at age 22 and retire at age 60 (after 38 years of work). (c) The employment rate used is 62.7 percent. (d) The discount rate used is 8 percent. (e) The duration of school infrastructure is 20 years, which is below the 25–30 years that is normally assumed. (f) The years of impact from the unified curriculum is 15 years, again below the 20 years suggested in the literature. (g) The teacher training is assumed to have 10 years of impact. (h) The learning materials delivered have a five-year span. 6. The analysis uses yearly administrative data from MINED on enrollment and completion rates; it estimated an average classroom size of 20 students for preschool and 21 for lower secondary classrooms. For consistency, it uses the same costs of education per pupil from the efficiency analysis in the Project Appraisal Document (PAD) (US$80 preschool, US$182 primary, US$109 lower secondary, and US$101 upper secondary). All labor statistics are obtained from the World Bank data bank and trading economics, and the projections in population growth and student enrollments are obtained from official MINED reports. The estimated IRR of 7.21 percent is below the original calculations of the economic analysis performed at the baseline of 10 percent, which considering the delays and Project adjustments seems reasonable. Similar to the efficiency analysis in the PAD, this analysis also applies a gradual effect on the benefits only reaching the full potential of the benefits at year 5. Page 48 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 7. Through a simple sensitivity test, it can be safely concluded that these estimations hold generally, and the IRR does not present major changes. To test for overestimation of benefits, a sensitivity test was performed by increasing the discount rate and decreasing the assumed effect of each of the components. By increasing the discount rate to 9 percent and 10 percent, benefit-cost ratios of 4.20 and 3.17 were obtained, respectively. These results are not as high as the calculated 5.63; however, they are not low enough to cast doubt on the benefits of the Project. Moreover, the assumptions are already on the conservative side, and by decreasing them even further, the changes in IRR and benefit-cost ratios were only minimal. Page 49 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS This is a summary of the Borrower’s Final Report of the Education Sector Strategy Support Project (Informe Final Proyecto de Apoyo a la Estrategia del Sector Educativo, PAESE) translated into English and redacted by the Bank. Any mistakes are the sole responsibility of the Bank. 1. Assessment of Project Objectives, Design, and Implementation 1.1. Background and Assessment of Objectives Background 1. In 2010, MINED, on instructions from the Office of the President, prepared an institutional diagnostic that identified strengths; weaknesses; and the need to coordinate administrative, financial, and technical processes. Subsequently, the 2011–2015 ESS was prepared to define the pillars and strategic guidelines for the sector for a five-year period. The ESS established three strategic pillars: Coverage/Equity, Quality, and Institutional Strengthening, each responding to the Government’s Education Policies, which emphasize the universalization of education, increasing enrollment and retention, reducing illiteracy, strengthening teacher capacity, ensuring adequate environments, adequate educational materials, comprehensive support to students, and strengthening human and institutional resources to promote the leadership of families and the educational community for the transformation and continuous improvement of the education process. 2. In this context, the ESS was presented to the Local Partners in Education to channel financial resources to the strategic pillars, with MINED assuming leadership and coordination of the various sources of financing. This was the first instance where external resources were channeled into institutional plans and objectives, replacing the previous strategy of obtaining resources in which donor organizations decided the areas to finance. With donor organizations available and willing to collaborate in the education system—within the framework of the ESS—and with employability identified as one of the EU’s main strategies for cooperation, employability was introduced in the National Education Policy, mainly in primary and secondary education.19 At the time, primary education received a higher percentage of the national budget, complemented with financing from PASEN II, presenting an opportunity for the EU to invest in secondary education with resources from the Nicaragua Education Sector Strategy Support Project (PROSEN). It is along these lines that the GPE joined the donors’ table, focusing on strengthening early preschool education through teacher training, provision of pedagogical materials, and curriculum updates to improve retention of preschool children in the first year. 3. Given that PASEN II focused on primary education, resources were required to strengthen basic and lower secondary education. In line with the objectives established in the ESS, PAESE was created to focus on primary and lower secondary, supported by PROSEN/EU priorities and complemented by GPE funds to respond to the need and demand in preschool education. 19Primary education comprises first to sixth grade; secondary education comprises seventh to eleventh grade (high school graduates or bachilleres). Page 50 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 4. On August 3, 2012, an agreement (denominated PROSEN) was signed between the GoN and the EU, to be implemented through the EU–World Bank Decentralized Administration Agreement and the Administration Agreement between the World Bank and the Government of Reconciliation and National Unity. These agreements constitute three legal instruments consistent with the 2011–2015 ESS. In April 2013, when this new program financed by PROSEN/EU/GPE was consolidated, two projects were defined and the identification of components and activities began. MINED considered it necessary to estimate costs to organize and distribute these activities, optimizing human, material, and financial resources, which is why it prepared a technical justification to integrate funds from the GPE, PROSEN, and PASEN II Projects. As part of this process, schools were selected for infrastructure rehabilitation, based on the deficit of primary schools to supply schooling in areas that did offer the next educational level. The methodology to calculate these schools contained technical criteria and indices to determine the enrollment rate, linked to population projections from the 2005 Census. 5. Given that school infrastructure in the three projects (GPE, PROSEN, and PASEN II) represented considerable costs requiring centralized management, a management strategy was created in 2013, in which PROSEN and GPE financed the execution of works and PASEN II assumed pre-investment costs, including social and environmental studies, among others. Similarly, training of preschool teachers was complemented with training for community educators and secondary teachers, along with the evaluation of childhood learning and development, which covers preschool, primary, and secondary education. Assessment of Objectives (a) Increase access to preschool education in the selected municipalities, and improve preschool education learning conditions nationwide (b) Increase access to lower secondary education in selected municipalities, and improve lower secondary education quality and completion rates nationwide. 6. The Project contributed to increased coverage and retention in both urban and rural areas, improved school climate, and improved quality of education, while promoting comprehensive development. This allowed the acquisition of skills, habits, values, autonomy, creativity, and attitudes necessary for students’ personal and social lives. The objectives for lower secondary were met, because students and teachers now have the tools needed to facilitate learning. Teachers have been equipped with new teaching strategies and have been trained in the use of textbooks and didactic guides. Scientific and methodological knowledge and skills of 2,498 secondary teachers were strengthened through training. Their competencies in the preschool curriculum were also strengthened to support preschool learning with greater quality and warmth. A total of 4,262 community educators were trained, strengthening the different training spaces, such as the Pedagogical Sessions for Shared Learning, pedagogical circles, and reference classrooms, where they shared their learning with another 4,177 community educators who converge in these spaces in their municipalities. 1.2. Design and Implementation 7. The Project aimed to assist the preschool and secondary educational population in the most vulnerable municipalities with unmet basic needs. It also supported teacher training in these levels, Page 51 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) equipping teachers with didactic material and training for the integration of the ESS. The leaders by implementation unit are as follows: • The GDPE provides services to children ages 3–5 in regular preschools, community centers, or multigrade schools in rural and urban areas. The two modalities, Regular and Community, offer three levels with differentiated schedules based on age, from Monday to Friday, in morning and afternoon shifts, implementing the same educational program. The Regular modality attends primary schools in urban areas, mainly with graduate teachers. The Community modality attends rural areas and reaches the most hard-to-reach areas of the country, taking place in family homes, community centers, churches, and primary schools, and is led by volunteer community educators characterized by a great spirit of service and disposition. • The GDSE attends students enrolled in Regular Education (morning and afternoon shifts) and Distance Secondary Education in rural areas (Saturday shifts). • The GDTT attends students from the eight teacher training colleges in the country, as well as in-service training at the municipal level for teachers in the different modalities. To ensure compliance with and support to the substantive areas, the Project also supported management units that provided technical assistance throughout the implementation period to achieve the objectives. Component 1: Increasing Access and Improving Learning Conditions of Preschool Education 8. This component focused on increasing access and improving learning conditions through improvements in school infrastructure and provision of furniture, didactic material, fungible material, and teacher training to improve performance. The main implementation units were the GDPE, GDSI, and the Department of Evaluation of Learning and Child Development. The Project supported activities beyond those laid out in the original design, such as training to parents and production of work, registry, and planning books for teachers and community educators. Subcomponent 1.5 was restructured to add activities 1.5.2 Preparation of a conceptual and methodological framework for the Integrated ECD M&E System (SEIDI); 1.5.3 Design of instruments to measure child development; 1.5.4 Validation of SEIDI implementation in a representative sample of preschools; and 1.5.5 Dissemination of the SEIDI baseline. Activities related to pre-investment studies were removed, because they were covered under PASEN II. This component also included the construction of 200 preschool classrooms; the need to carry out improved works with integrity of design, however, resulted in cost increases and required more time for the adjustments, resulting in a new target of 110 preschool classrooms. Component 2: Improving Access, Quality and Completion of Lower Secondary Education 9. This component aimed to improve access, quality, and completion of the first cycle of secondary education (grades 7–9) through the distribution of textbooks, works to improve school infrastructure and school furniture, and in-service teacher training on improving education. The main implementation units were the GDTT, the GDSE, the GDSI and the Autonomous Regions Educational System (SEAR). The three subcomponents were not modified. A modification was required, however, in the Operational Manual (OM), in activity 2.2.1 Training for 7th-9th grade teachers during 2014 and 2015, where an amendment Page 52 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) and new strategy were included in the payment of scholarship to teachers (annexed in the OM). Activity 2.2.5 Training for 7th-9th grade teachers on the use of ICT during 2013, 2014, and 2015 was modified to include a team to support the preparation of multimedia material for trainings, which was not part of the original design. The scope of the original objectives for the Caribbean Coast were broadened for activities 2.3.2 Cultural adaptation of Secondary textbooks in the Caribbean Coast and 2.3.4 Printing of guides and materials for Secondary teachers during 2013’. Pre-investment studies foreseen for infrastructure were not carried out, because they were funded by PASEN II. Component 3: Strengthening of the Ministry of Education’s Institutional Capacity for Preschool Education 10. The main implementing units were the GDPE, the GDTT, the Financial Division, the Procurement Division, and the GDSI. MINED’s technical departments were supported and strengthened through provision of equipment and furniture to the GDPE, field supervision, and development of Project activities. This component also ensured field supervision and carrying out of Project activities. The two subcomponents were not modified. However, there were changes to the description in the OM for activity 3.2.8 Support to Project operational management to adjust the description to what was detailed in the original budget. All planned activities were implemented, except for activity 3.2.6 Audits, carried out using PASEN II funds. Component 4: Strengthening of the Ministry of Education’s Institutional Capacity for Lower Secondary Education 11. This component comprised four subcomponents and 15 activities to strengthen the GDSE, the Directorate of Planning and Monitoring, the GDTT, the GDSI, the Project Directorate, and the Financial Division and ensure supervision of works in all prioritized municipalities. Activity 4.4.5 Strengthening MINED’s planning, monitoring and statistical capacity was added to strengthen MINED’s planning and monitoring system; however, the system was not established and funds were redirected to school rehabilitation. Activity 4.4.6 Technical assistance for Project implementation was also added to include activities required for Project management as part of the cooperation strategies with other sources of funding. All planned activities were implemented, except for activity 4.4.3 Audits, carried out using PASEN II funds. 12. Early implementation faced delays in the formalization of the following legal documents: Nicaragua–EU Agreement (signed in August 2012); GPE–World Bank Agreement (signed in May 2013); and World Bank–EU Agreement (signed on September 17, 2013). This 13-month delay affected implementation, resulting in the need to restructure the Project. Delays in launching activities and pre- investment studies, a reduction in resources due to varying euro-dollar exchange rates, increased construction costs, and the introduction of the concept of comprehensive works resulted in an extension of the implementation period to August 31, 2016. A second extension of the implementation period was required to achieve the Project’s objectives. Page 53 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 2. Assessment of Results Against Objectives 2.1. General results 13. The main project objectives were achieved, with some exceeding original targets. Indicator 1: Gross enrollment rate for preschool in selected municipalities 14. This indicator increased in the last three years: 41 percent in 2015, 45 percent in 2016, and 50 percent in 2017. This was possible due to the Project’s contribution to coverage, retention, and quality of preschool education through an updated and unified Program for both modalities. The Program was characterized by simplicity, ease of management, and adequacy for the varying academic and experience levels of the preschool community educators and teachers. Training in the use of curricular documents allowed for their efficient implementation in the classroom, helping children play an active role in their learning. This also contributed to educational quality, because the availability of didactic and fungible materials in preschool classrooms promoted increased participation, and adequate school climate, improved spaces, and school furniture contributed to improved learning/recreational experiences and attendance in early preschool. The provision of fungible material to each of the key players further contributed to this. Providing the basic tools to develop activities to awaken children’s interest, as well as equipping teachers and community educators with instruments to control activities in adequate spaces, motivated parents to take children to school with greater confidence, resulting in increased enrollment. Indicator 2: Gross enrollment rate for lower secondary (grades 7-9) in selected municipalities 15. The target of 47 percent was surpassed, reaching 57.2 percent by 2017. The Project contributed to the professionalization of 969 empírico teachers at the different schools in the prioritized municipalities. This activity originally intended only to ensure the professionalization of lower secondary education, but during the process, many completed their university degrees. This helped strengthen their teaching methodologies, which in turn increased school retention rates and students’ interest in learning. The 10 university majors in which secondary school teachers were professionalized were Spanish Language and Literature; Natural Sciences; Social Sciences; Physics; Mathematics; Foreign Language; Physical Education and Sports; Vocational and Technological Orientation; Culture and Arts; and Pedagogy, with a minor in education for diversity. 1,529 teachers participated in refresher courses in teacher training colleges, focusing on specific content to strengthen acquired knowledge. Through this professionalization, the scientific and methodological knowledge, skills, and capacities of secondary teachers were strengthened. These efforts sought to improve their pedagogical practices and, consequently, the quality of learning in the classroom. 16. Rehabilitation of 250 core schools was originally planned, but due to a need to consider comprehensiveness20 criteria for school infrastructure; increased cost of construction materials, labor, and transportation in the national market; and fluctuation in the euro-dollar exchange rate, the scope was reduced to 75 core schools. A more comprehensive vision of school works was integrated into the pre- investment studies, incorporating complementary spaces for education and recreation. 74 schools were 20Comprehensiveness refers to carrying out an integral intervention in school infrastructure works, updated to the real costs derived from pre-investment studies and addressing a minimum of complementary needs. Page 54 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) rehabilitated; the pending rehabilitation will be financed by alternate funds. A percentage per year was planned for this activity, but a total of 6 schools were rehabilitated in 2014, 27 in 2015, 6 in 2016, and 35 in 2017, resulting in a total of 74. Each school received furniture, consisting of desks, a teacher set (desk and chair), and cabinets. Due to a lack of school furniture nationwide, distribution was broadened to additional core schools through other sources of funding already operating in the prioritized municipalities. A total of 22,610 desks, 646 sets for teachers, 646 cabinets, and 646 blackboards were distributed. This all contributed to improved quality of school climates for teachers and students in the prioritized municipalities, thus complying with the need to adequately equip spaces to foster learning. 17. All of these complementary actions ensured physical spaces with adequate learning environment, as well as teachers scientifically and methodologically prepared to face the challenges in fostering learning and motivate student attendance. This is evidenced in the rising percentages over the last three years. Indicator 3: Learning conditions improved in preschool 18. The GDTT structured an in-service teacher training course consisting of seven modules on Communications Skills; Development of Logical Mathematical Thought; Neuroscience; Assessment of Learning; Transition to First Grade; Special Education; and Cultural and Artistic Expression. These courses strengthened the knowledge and skills of community educators to improve the development of skills in preschool children. Three factors were used to measure this indicator: (a) 2,321 certified community educators from the prioritized municipalities were trained, (b) 1,941 preschool teachers were trained in the use of the Unified Studies Plan and duly modified instruments, and (c) 8,903 schools nationwide received didactic and fungible material. Preschools nationwide benefited from the implementation of a new preschool Unified Studies Plan; 6,790 pedagogical advisers, teachers, and community educators were trained in the successful transition from the third year of preschool to the first grade of primary. Preschool teachers and community educators received didactic and pedagogical material to carry out their activities and facilitate the learning processes. About 12,000 student attendance and evaluation books, 150,000 workbooks for the third year of preschool, and planning books for teachers and community educators were distributed, in line with the yearly plans throughout implementation. 19. There were further achievements in preschool beyond those foreseen at appraisal. Community educators were strengthened in curriculum improvement. The Project also contributed to the creation of an interdisciplinary teacher training team that reviewed and supervised the sequencing and importance of the transition to first grade, as well as the competencies that children must acquire. Schools and quality were improved, and coverage was extended to the entire country, rather than just the prioritized municipalities. The training of community educators contributed to strengthening different training spaces, such as the pedagogical sessions for shared learning, pedagogical circles, and reference rooms, where they shared what they learned with other community educators in their municipalities. 20. After carrying out a diagnosis for the design of a conceptual framework for SEIDI, a child development measurement instrument was implemented nationwide, with the participation of the Ministry of Health (MINSA), MIFAN, and MINED. The sample selected for the diagnosis included 400 MINED schools, 86 MIFAN daycare centers, and 359 MINSA health centers. Interviews were conducted with key actors in all three institutions. The results of the diagnosis were used as inputs for the SEIDI Conceptual and Methodological Framework. Technically appropriate instruments were prepared and redesigned to meet the desired technical characteristics and to collect objective evidence on the impact Page 55 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) on children’s development and learning in two key areas of preschool evaluation: (a) quality of learning environment (class observation, teacher survey, parent survey, and school principal survey) and (b) development and learning (direct assessment of preschool children, teacher survey, and parent survey). A pilot was carried out in a representative sample of 149 schools in 77 municipalities in the country’s 15 departments and 2 Autonomous Caribbean Coast Regions. Psychometric and statistical analyses of the information collected were carried out to identify the functioning of the various items that made up the piloted instruments. The results were used to prepare a report, and the instruments will be modified to yield definitive versions that will be used to create the baseline for the evaluation of development and learning in preschool. 21. These achievements are complemented by the refurbishing of 110 classrooms by 2017 and their equipment with 1,020 preschool sets, 170 bookcases, 340 shelves, 170 teacher sets, and 170 cabinets. These spaces are now adequate to carry out a quality learning process. Indicator 4: Grade 9 completion rate 22. The target of 71 percent, was surpassed, reaching 74.6 percent, according to the 2016 statistical data, indicating that 75 out of 100 adolescents that begin grade 7 conclude grade 9. This achievement strengthened the provision of tools to develop learning, with the first nationwide distribution of 1,512,385 textbooks in 2014. By 2016, an additional 191,475 textbooks were distributed, for a total of 1,703,860 grade 7–11 Mathematics, Language and Literature, Social Sciences, Natural Sciences, English, Physics, and Biology textbooks benefiting approximately 284,990 public school students and teachers nationwide, 51 percent of which are women. It is important to highlight that 386,700 self-taught modules were produced and distributed to students enrolled in distance secondary education, covering Mathematics, Language and Literature, Natural Sciences, Social Sciences, Coexistence, and English. 23. The Project also helped facilitate dynamic and participatory learning processes, student experience exchanges, and a higher level of ownership of class contents, contributing to independent work and study practices. School retention and academic performance improved, with a higher index of completion and graduation. In-service training was provided to the departmental and municipal Technical Advisors on the use and care of textbooks in the school and was then replicated in each school. The GDTT carried out a total of 170 classroom visits to secondary teachers and provided assistance and pedagogical advice according to observed strengths and needs, contributing to the improved quality of teaching. 24. School rehabilitation in prioritized municipalities contributed to improved learning conditions for students and teachers and, subsequently, improved school retention and completion. Textbooks were culturally adapted for the Caribbean Coast, integrating Bilingual Intercultural Education through periodical training in both the northern and southern autonomous regions. This contributed to an increase in the number of students completing lower secondary, because they had teachers who attended refresher courses and/or joined the MINED and SEAR curricular commissions. Technical assistance was provided in the formulation and design of programs, methodological guides, and textbooks for bilingual intercultural education in five indigenous languages, which will be reproduced by MINED with alternate funding. Of the 74 rehabilitated schools, 62 received computer equipment, didactic and fungible material, and science and literature books to support the learning process. This was not contemplated in the original Project design, but was implemented as a complement to improvements in school infrastructure. Page 56 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Indicator 5: Percentage of Grade 9 students achieving intermediate, advanced or above proficiency levels standardized evaluations for Spanish Language and Mathematics 25. MINED focuses on improving the quality of learning for students in the different education levels. In this regard, and continuing the efforts made in 2010 to evaluate student learning in Language, Literature, and Mathematics in grades 4, 6, and 9, another National Learning Evaluation was carried out in October 2015. This evaluation provided information to strengthen pedagogical practices in the classroom, improve student learning, and inform decision making on existing education policies. The 2015 evaluation was administered to 5,665 grade 9 students. Questionnaires were also distributed to students, teachers, and principals in the selected schools to collect information on factors related to the learning processes of the students being evaluated. Based on the results of the evaluation, reports were prepared on related factors and learning outcomes, including information on average scores and performance levels among grade 9 students. The results indicated that 41 percent of students were at an advanced level, 31.1 percent were at an intermediate level, and 27.8 percent were at a basic level. Estimation of Mathematics performance levels was not possible because the reliability parameters of the psychometric analysis of the evaluation were very low. In light of this, a revision and update of the item bank is foreseen, considering curricular adjustments and the new pedagogical focus being implemented by MINED. 2.2. Other Achievements Related to Institutional Strengthening 26. Throughout the Project cycle, activities focused mainly on improving physical spaces and equipping schools for institutional strengthening of the GDPE. The supervision of works was strengthened by hiring the required staff of supervisors at the central and departmental levels, who then monitored the execution of works. Hiring these personnel faced delays because the bidding processes were declared void on several occasions, and all the personnel were only hired in 2016. Project management was strengthened through the hiring of fiduciary and administrative staff to carry out administrative processes. Over the course of the Project, five annual audits took place, all with moderately satisfactory results and offering suggestions that were addressed in a timely manner. 27. Departmental and municipal Technical Advisors received training on the use, care, and handling of textbooks in schools, which they replicated in each school. The GDSE received the equipment necessary to design textbooks. The GDTT was strengthened with technical and administrative staff that supported the implementation of planned activities, thus ensuring that a considerable number of processes took place in each municipality; the acquisition of computer equipment and furniture; and the hiring of experts for the preparation, design, and editing of self-taught modules. These helped guarantee close support to teachers in the prioritized municipalities, as well as the completion of the planned educational research workshops. 28. A School Maintenance Manual was prepared and distributed to every school benefiting from rehabilitation works, which will serve as a tool to ensure their maintenance and sustainability, complemented by training to school principals. Page 57 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) 3. MINED Performance during Project Preparation and Implementation 3.1. Monitoring 29. MINED, through the Administrative and Financial Vice Ministry, ensured the effective monitoring of the various Project processes and activities. This was possible due to teamwork among its different support units: Financial Division, Administrative Division, Financial Management Division, Legal Advisory, Procurement, External Resource Coordination, and Project Management, as well as the different implementing agencies within MINED and governing bodies like the Ministry of Finance and Public Credit (MHCP). MHCP also facilitated space for communication between MINED and the Bank to streamline Project implementation. Interdisciplinary staff was hired to build the team supervising the implementation of the Project; providing technical assistance and advisory services to supporting units within MINED on Bank guidelines and procedures; monitoring systems; and following up with the Project management, governing bodies, and other entities involved in implementation. This experience has enriched the work between supporting units and implementing units due to the level of understanding and ownership of Bank procedures, their harmonization with MINED’s internal control procedures, and development of its own methodology and internal control mechanisms, which builds capacity in MINED for institutionalized implementation of projects financed by the Bank and other sources of funding. 3.2. Financial Control and Auditing 30. PAESE is MINED’s third recent experience implementing Bank projects. From a financial perspective, its performance has been consolidated in the past few years through updated accounting, fluidity in reporting expenses, capacity to implement and manage financial and accounting controls, and annual auditing. This led to the execution of Bank and other donor funds in compliance with the organization’s and governing bodies’ procedures. Despite this progress, the use of manual procedures prevails, making the automatization of flows and procedures, along with the implementation of information systems that are well articulated among the different units, imperative to yield updated and uniform information for timely decision making. 31. Financial control and auditing were carried out by the Financial Division, in coordination with the Project Monitoring Office, by a team hired under the Project that worked with MINED line staff to prepare the different FM reports and participate in meetings, technical missions, and financial supervision. This team presented the Project’s achievements and implementation experiences, proposing alternatives and measures for the Dirección Superior to make decisions in coordination with World Bank management units. 3.3. Indicators 32. Project monitoring included ensuring compliance with the indicators established for each Project. This activity, however, was not initially identified and was thus organized as different situations arose, resulting in the development of strategies to collect results by each implementing unit responsible for PDO indicators, including: • Meeting with the different implementing units to inform them of the indicators and the required support for their achievement, Page 58 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) • Establishing ownership of the Results Framework (the basic instrument for monitoring), • Establishing a monitoring plan with deadlines for data collection for each indicator, • Supporting and advising as needed, as often it was unclear what supporting documentation was required for each indicator, • Continuously monitoring to ensure indicator achievement and define pertinent actions as needed, • Coordinating with implementing units through institutional mail (INFOMINED) when the responsibility for indicators was established in different municipalities to ensure achievement in each Project, with support from the Dirección Superior, and • Validating information sent by each implementing unit with different instruments provided by the supporting units (statistics, procurement, and human resources) to ensure consistency in information. 33. Monitoring included constant communication among the various units, field delegates, and even school principals. 3.4. Performance of Other Units 34. This Project was implemented under the institutionalized implementation scheme, with active participation from the various substantive and supporting units in MINED, generating experience and acquiring knowledge regarding World Bank guidelines and procedures. While coordination and management were complex due to the number of actors involved, there is evidence of professional growth and knowledge gained by personnel in the different units, divisions, directorates, and general directorates, as well as by the team of public servants involved. Moreover, under the leadership of the Dirección Superior, domain, institutional ownership, technical comprehension, and clear identification of variables in the implementation of Projects with different sources of financing and procedures has been observed. This allowed the institution to respond to situations such as turnover of public servants without substantially affecting the processes and activities of this and other Projects. 4. World Bank and Donor Performance 35. The World Bank supported the Project design by hiring consultants to work with the Project Directorate, under the leadership of the Dirección Superior. During implementation, the Bank supported preparation, processing, and execution of the Project, resolving different situations that arose in substantive areas, in the development of actions, and in field visits to document quantitative and qualitative progress. Changes in Bank specialists (including the Task Team Leader and Social, Environmental, Infrastructure, and Procurement Specialists) at times interrupted the continuity of advisory services, causing delays in the presentation of final products to Bank management. This was the case with the Pre-Investment Studies, whose structure was not agreed until late 2015, delaying the start of bidding processes for 51 schools. Recommendations for both the Bank and MINED teams are: • Ensure stability in the staff providing technical support, as turnover negatively affects implementation, • Maintain uniform criteria among supporting consultants, as they often concentrate only on their specialization, overlooking key aspects to improving results, Page 59 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) • Establish cluster procurement processes only if the implementation dates of the different areas coincide to avoid delays in the processes of one area or another (no objection to supporting documents), and • Update the OM in a comprehensive manner. 5. Lessons Learned 5.1. Implementation • Creation of interunit commissions in MINED, under the Administrative and Financial Vice Ministry, for specific topics allowed the creation of strategies for interdisciplinary solution of specific technical situations, fostering fluid communication and effective coordination in MINED’s different areas. • Decentralization of processes in implementing units and timely decision making by the management team allowed for group and individual commitments. • Effective empowerment of MINED technical and administrative staff involved in the different stages of implementation. • Unify processes to reduce direct and transactional costs due to the number of goods and services procured. A weakness was identified, however, in the practice of synchronizing dates for the implementing units to present technical specifications and in the ability of providers to comply with long-term contracts. • Each implementing unit should prepare market studies before starting a process to avoid delays in bidding or void processes due to a lack of bids. • Ongoing support by Bank advisors in different processes contributes effectively to implementation. • Timely responses to requests for No Objection from the Bank team streamlines processes. 5.2. Results • Effectively empower public servants on the management instruments, as well as the methodology and technical activities of the Project, in terms of accounting and procurement processes. • Implement diverse strategies to involve responsible actors in each process in the Dirección Superior’s decision making. • Hold focus groups before the end of key activities to gather qualitative evidence from the field, to gather information from the direct stakeholders. • Clarity in Project objectives facilitates the achievement of desired results. • Monitoring of the trainings implemented in the field makes it possible to see results in teachers. 5.3. Project Performance • Open communication between MINED and the Bank is crucial to achieving the proposed objectives. Page 60 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) • The experience in implementing PAESE contributed to institutional capacity building, both at the central level and in the field. • Through the Project, technical capacity was transferred from headquarters to the field delegations. • The Project has fostered an exchange of experience in relation to the implementation of other MINED Projects and other realities in the Latin American context. 6. Arrangements to Ensure Sustainability 36. MINED intends to continue the achievements of the Project by taking actions that ensure its sustainability, including the following: • The preschool quality model continues to be replicated; materials prepared for evaluation and planning have been a basic tool for community educators and preschool teachers, and these documents will continue to be used in the ACE Project for the next four years. • As an incentive for preschool children, teachers and mobile trainers were trained in folkloric dance, which will be continued in other Projects that will provide audio equipment for children to showcase their artistic ability. • Design improvements in preschool learning environments were satisfactory and contributed to the development of good supervision practices in teachers and community educators, allowing for greater supervision of children at all times, including when using bathrooms, which are now located in the classrooms. • The preparation of a School Maintenance Manual will serve as a guideline for principals, teachers, and the education community to ensure proper maintenance of each rehabilitated school. Further, as part of the strategy of the Projects managed by the Bank (PASEN II and PAESE), the GDSI was strengthened by social and environmental safeguards and the achievements in maintenance and inventory of schools. This resulted in the incorporation of these practices in their functions, adapted to their context and availability of human, material, and financial resources. • Bilingual Intercultural Secondary Education programs, methodological guides, and textbooks were designed and prepared in five indigenous languages (Miskitu, Kriol, Tuahka, Panamahka, and Ulwa) and will be reproduced under the ACE Project. Page 61 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ANNEX 6. SUPPORTING DOCUMENTS Abadzi, Helen. 2006. Efficient Learning for the Poor: Insights from the Frontier of Cognitive Neuroscience. Washington, DC. https://openknowledge.worldbank.org/handle/10986/7023 Araujo, M. C., P. Carneiro, Y. Cruz-Aguayo, and N. Schady. 2016. “Teacher Quality and Learning Outcomes in Kindergarten.� Quarterly Journal of Economics 1415–1453. doi:10.1093/qje/qjw016 Bandiera, O., N. Buehren, R. Burgess, M. Goldstein, S. Gulesci, I. Rasul, and Munshi Sulaiman. 2012. Empowering Adolescent Girls: Evidence from a Randomized Control Trial in Uganda. Gender Impact: The World Bank's Gender Impact Evaluation Database. Washington, DC: World Bank. Behrman, J. R., D. Ross, & R. Sabot. 2008. Improving quality versus increasing the quantity of schooling: Estimates of rates of return from Rural Pakistan. Journal of Development Economics, 94-104. Berlinski, S., S. Galiani, & P. Gertler. 2009. The effect of pre-primary education on primary school performance. Journal of Public Economics, 219-234. Card, D. 1999. The Causal effect of Education on Earnings. In O. Ashenfelter, & D. Card, Handbook of Labor Economics (pp. 1802-1959). Elsevier Science. Chetty, R., J. N. Friedman, and J. E. Rockoff. 2014. “Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates.� American Economic Review 104 (9): 2593−2632. doi: 10.1257/aer.104.9.2593. Chetty, R., J. N. Friedman, and J. E. Rockof. 2014. Measuring the Impacts of Teachers II: Teacher Value- Added and Student Outcomes in Adulthood. American Economic Review, 2633-2679. Duarte, Jesús, Florencia Jaureguiberry, Mariana Racimo. 2017. “Suficiencia, equidad y efectividad de la Infraestructura escolar en América Latina según el TERCE� UNESCO/IBD Education for People and Planet: Creating Sustainable Futures for All. Global Education Monitoring Report. Paris: UNESCO, 2016. http://unesdoc.unesco.org/images/0024/002457/245752e.pdf. Evans, D., and A. Popova. 2015. “What Really Works to Improve Learning in Developing Countries? An Analysis of Divergent Findings in Systematic Reviews.� Policy Research Working Paper, WPS 7203, World Bank Gimenez, L., S. Van Wie, M. Muller, R. Schutte, M. Z. Rounseville, and Martha Viveros Mendoza. 2015. “Enhancing Youth Skills and Economic Opportunities to Reduce Teenage Pregnancy in Colombia.� World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/22344. Cunningham, W., and Paula Villaseñor. 2014. “Employer Voices, Employer Demands, and Implications for Public Skills Development Policy.� World Bank Group Policy Research Working Paper 6853, World Bank, Washington, DC. World Bank 2013. Page 62 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Glewwe, Paul (Ed.) (2016). Education policy in developing countries, University of Chicago Press. Chicago. Glewwe, P. W., E. A. Hanushek, S. D. Humpage, and R. Ravina. 2013. “School Resources and Educational Outcomes in Developing Countries: A Review of the Literature from 1990 to 2010.� In Education Policies in Developing Countries, edited by P. W. Glewwe, 13–26. Chicago: University of Chicago Press. Glewwe, P. W., and K. Muralidharan. 2016. “Improving Education Outcomes in Developing Countries: Evidence, Knowledge Gaps, and Policy Implications.� In Handbook of the Economics of Education, edited by E. A. Hanushek, S. Machin, and L. Woessmann, Vol. 5, 657–743. Netherlands: Elsevier. doi: http://dx.doi.org/10.1016/B978-0-444-63459-7.00010-5. Kraft, M., Blazar, D., & Hogan, D. (2016). The effect of Teacher Coaching on Instruction and Achievement: A Meta-Anlayis of the Causal Evidence. Review of Educational Research, Forthcoming. Otter, T., and C. Villalobos Barría. 2009. “Determinants of Student Achievements in the Primary Education of Paraguay.� Ibero-America Institute for Economic Research Discussion Paper No. 198, Georg- August-Universität Göttingen, Goettingen, Germany. Popova, Anna, David Evans, and Violeta Arancibia. 2016. “Training Teachers on the Job: What Works and How to Measure It.� Policy Research Working Paper WPS 7834, WDR 2018 Background Paper. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/122031474915232609/Training-teachers-on-the- job-what-works-and-how-to-measure-it. Schneider, Mark. 2002. “Do School Facilities Affect Academic Outcomes? National Clearinghouse for Educational Facilities.� http://www.edfacilities.org/pubs/outcomes.pdf. Serra V., Luis, and Marcia Castillo S. 2003. “Informe de Investigación: Situación de la Niñez y Perspectivas de Desarrollo Humano en Nicaragua,� as quoted in Por qué hay niñas en Nicaragua que temen ir al baño en la escuela, BBC Mundo (2016). World Bank. 2012. Nicaragua - Country partnership strategy for the period FY13-FY17 (English). Washington, DC: World Bank. http://documents.worldbank.org/curated/en/828301468290161276/Nicaragua-Country- partnership-strategy-for-the-period-FY13-FY17 World Bank. 2013. Nicaragua - Education Sector Strategy Support Project. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/304271468122358474/Nicaragua-Education- Sector-Strategy-Support-Project Page 63 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) World Bank. 2015. Nicaragua - Country partnership strategy for the period FY2013-2017: performance and learning review. Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/760311468188340065/Nicaragua-Country- partnership-strategy-for-the-period-FY2013-2017-performance-and-learning-review World Bank. 2013. Life Skills: What Are They, Why Do They Matter, and How Are They Taught? Adolescent Girls’ Initiative (AGI) Learning from Practice Series. Washington, DC: World Bank. World Bank. 2016. Nicaragua - Education Sector Strategy Support Project: restructuring. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/983411468123249474/Nicaragua-Education- Sector-Strategy-Support-Project-restructuring World Bank. 2017. Nicaragua - Alliance for Education Quality Project. Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/225281492308020968/Nicaragua-Alliance-for- Education-Quality-Project World Bank. 2018. Nicaragua - Country partnership framework for the period FY18-FY22. Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/419731521338434751/Nicaragua-Country- partnership-framework-for-the-period-FY18-FY22 World Bank. 2018. Nicaragua - Completion and learning review for country partnership framework for the period FY13-FY17: IEG review (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/620931522202439021/Nicaragua-Completion- and-learning-review-for-country-partnership-framework-for-the-period-FY13-FY17-IEG-review Page 64 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ANNEX 7. THEORY OF CHANGE WITH SUPPORTING LITERATURE Figure 7. 1. Overview of the Project’s Theory of Change Activities/Outputs Intermediate PDOs/Outcomes Long- Outcomes Term Outcomes Increased number of Serra V. et al. • Increase access to classrooms and safety (2003) preschool education in of environment Improving/expanding selected municipalities education infrastructure Schneider • Increase access to lower Improved physical secondary education in environment for (2002) selected municipalities learning Development/ Updated curriculum alignment of Abadzi (2006) Increase and instruments for preschool curricular individual preschool nationwide instruments and social benefits Training and Improved teaching • Improve preschool of Chetty (2014) education learning certification of practices and teaching education teachers effectiveness conditions nationwide • Improve lower secondary Development and Increased supply and Glewwe et al. education quality and distribution of improved materials for (2013) completion rates learning materials learning nationwide Improved management Institutional of preschool and lower strengthening of secondary education MINED sectors 1. The logic of the results chain of the Project is well-supported by evidence in the literature: • Number of classrooms and safety of environment. Evidence supports that distance to school and the state of school water and sanitation infrastructure matter for participation (Serra V., Luis, and Marcia Castillo S. 2003). The project financed building of additional classrooms to provide additional access and reduce distances to school. Creation of classrooms at schools where there were none increases access by reducing the distance to school for young children (and increasing number of classrooms improves student/teacher ratios). In Nicaragua, only 26 percent of basic education schools have basic sanitation (UNESCO 2010). Unhygienic latrines or latrines without doors/locks can be a disincentive to stay in school, particularly for girls. The Project financed interventions in sanitation to ensure that this did not discourage students from attending school. The Project also financed access to clean water, which can improve health outcomes and positively affect attendance and avoid dropouts. To increase safety at school, the Project included perimeter walls to prevent access to strangers, to both avoid disruption to learning and ensure parents feel safe to send Page 65 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) their children to school. Finally, infrastructure investments included ramps and accessible bathrooms for students with disabilities. • Environment for learning. The available research suggests that improving physical environment is also associated with better learning outcomes. Schneider (2002) reviewed 137 papers and concluded that spatial configuration, noise, heat, cold, light, and air quality are strongly associated with learning outcomes. A 2017 UNESCO/Inter-American Development Bank study concludes that infrastructure is positively and significantly associated with student learning, even after controlling for student socioeconomic level (Duarte, Jaureguiberry, and Racimo 2017). The conclusion supports other work (i.e. Glewwe 2016) stating that attending a school with sufficient physical learning spaces, that is, a school with drinking water, bathrooms, electricity, telephone, library, space for sport or student meetings and classrooms equipped with the basic materials, is generally associated with better school learning. In addition to the investments mentioned above, the Project also invested in sports facilities. Sport supports the development of socioemotional skills, which in turn can have positive impacts on education, health, and labor market outcomes in the short and long term (Gimenez et al. 2015; see also: Bandiera et al. 2012; Cunningham and Villaseñor, 2014; World Bank 2013). Furniture can increase comfort for students to aid learning as well as better safeguard learning materials. There is also evidence from Paraguay that schools with better infrastructure perform better (Otter, T., and C. Villalobos Barría. 2009). • Updated Curriculum. The updated curriculum and associated teaching instruments improve learning outcomes through pedagogical theory using principles of neuroscience (i.e. Abadzi 2006), and was built jointly with international experts on the topic. The alignment of the curriculum helps ensure the successful transition of children through the three grades of preschool, as well as into primary school. • Teaching practices. There is widespread evidence that teacher quality is the most important input to affect learning in developed countries (Chetty, Friedman, and Rockoff 2014). Araujo et al. (2016) also showed that teacher quality, measured by teachers’ classroom competency, has a significant impact on learning of primary students. Bruns and Luque (2015) highlighted that grooming more effective teachers requires investing in classroom observation and supportive feedback to teachers to ensure a better assessment of their competencies. Popova, Evans, and Arancibia (2016) find that programs that provide complementary materials, focus on a specific subject, and include follow-up visits tend to show higher learning gains. The Project supported teacher training, both for certification and in-service, and included reference classrooms, mentoring, workshops and support groups, and training on specific subjects with provided materials, which are all supported in literature as effective methods to improve teacher quality. • Learning materials. There is also evidence that inputs such as appropriate textbooks and other pedagogical materials have positive effects on learning outcomes (Glewwe, P. W., E. A. Hanushek, S. D. Humpage, and R. Ravina 2013). Textbooks promote learning by enabling students to better follow lessons and allowing students to work independently of the teacher. In Kenya, the distribution of official textbooks and materials increased the Page 66 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) probability that students who stayed in school through grade 8 would complete primary school and enroll in secondary school (Glewwe, P. W., and K. Muralidharan 2016). • Development of ECD M&E system. The Project supported development of a system to evaluate ECD and learning outcomes in preschool. The system will gather objective evidence on (a) quality of settings (class observation, teacher survey, parent survey, and school principal survey); and (b) development and learning (direct assessment of preschool children, teacher survey, and parent survey). The results will inform further initiatives to improve learning outcomes for children. Page 67 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) ANNEX 8. REVISION HISTORY OF PROJECT INDICATORS PDO Indicators PDO Indicator 1 - Revised Original Indicator Original Description Original Original Reason Baseline Target Increase in the Number of students 41.7% 48% The wording was revised gross enrollment enrolled in grades 3–5 of for clarity and to take into rate for preschool preschool divided by the account that ‘enrollment’ (ages 3–5 years) in total population of children refers to every child selected ages 3–5 years based on enrolled in preschool, not municipalities the 2005 Population only those who are 3 to 5 Census Data. The indicator years old. In addition, the will be calculated for all baseline value was revised preschools in the 43 based on 2013 school year selected municipalities. data, while the target value was revised Revised Indicator Revised Description Revised Revised downward given the Baseline Target increase in the number of Gross enrollment Number of students 40% 44% children ages 3–5 years in rate for preschool enrolled in the three levels selected municipalities in selected of preschool in the (according to updated municipalities selected municipalities 2015 projections from the divided by the total 2005 Population Census population of children of Data).21 The description ages 3–5 years based on was also revised to reflect projections from the 2005 these changes. Population Census Data. The indicator will be calculated for all preschools in the 43 selected municipalities. PDO Indicator 2 - Revised Original Indicator Original Description Original Original Reason Baseline Target Increase in the Number of students 48.6% 54% The wording was revised gross enrollment enrolled in grades 7–9 of for clarity. The baseline rate for lower preschool divided by the value was revised based 21 Demographic projections are updated on an annual basis. Page 68 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) secondary (grades total population of children on revised 2013 school 7–9) in selected ages 12–14 years based on year data. The target municipalities the 2005 Population (54%) was not changed, in Census Data. The indicator consideration of the will be calculated for all important increase preschools in the 43 between 2013 (47%) and selected municipalities. 2014 (51%). Revised Indicator Revised Description Revised Revised Baseline Target Gross enrollment Unchanged 47% 54% rate for lower secondary (grades 7–9) in selected municipalities PDO Indicator 3 - Unchanged Learning conditions improved in preschool. PDO Indicator 4 - Revised Original Indicator Original Description Original Original Reason Baseline Target Increase in the Number of students 52% 57% The wording of the grade 9 enrolled in grade 9 at the indicator was revised for completion rate end of the school year clarity and the description divided by the total was replaced by the population of children age conventional UNESCO 14 years based on the 2005 definition of completion Population Census Data. rate. Baseline, annual, and target values were revised Revised Indicator Revised Description Revised Revised consequently. Baseline Target Grade 9 The number of new 64% 71% Considering the new completion rate entrants (enrollments description and the revised minus repeaters) in grade baseline value, a new target 9, regardless of age, value was set in line with divided by the population past years’ trends. at the entrance age for grade 9. This indicator is calculated at the national level. PDO Indicator 5 - Revised Original Indicator Original Description Original Original Reason Baseline Target Page 69 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Increase in the The next assessment Mathematics Mathematics The wording was percentage of grade will be carried out in 51.2% 53% revised for clarity and 9 students achieving 2015. Baseline is the description was intermediate, 2010. The baseline is Spanish Spanish revised to clarify that advanced or above the sum of the two 64.9% 66% the indicator would proficiency levels levels (advanced and measure three levels standardized excellent). Tests of achievements evaluations for conducted on a (intermediate, Spanish and representative sample advanced, and Mathematics. at national level. excellent) instead of Revised Indicator Revised Description Revised Revised two (advanced and Baseline Target excellent); the fourth Percentage of grade The next assessment Mathematics Mathematics level of achievement 9 students achieving will be carried out in 51% 53% (low) would not be intermediate, 2015. The baseline measured. Based on advanced and above year is 2010, which is Spanish Spanish available information proficiency when the last 75% 77% on national (excellence) level in assessment was assessment in grade 9 standardized carried out. The in 2010, the following evaluations for baseline is the sum of changes were made: Spanish and three of the four (a) for Spanish, the Mathematics. levels established in baseline value was the scale of the 2010 corrected to integrate assessment: the revised figures intermediate, from the GoN; and (b) advanced, and for mathematics, the excellent. Tests are baseline was rounded conducted on a and the target value representative sample was confirmed. With at the national level. the delays in Project implementation, impact on quality was only measurable after Project completion. Therefore, conservative target values were established considering that in- service teacher training and textbook distribution are expected to contribute positively to better student outcomes. Page 70 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) However, the increase in completion rate (indicating a greater number of low student performers in the system), would lower student outcomes. Intermediate Results Indicators IRI Indicator 1 - Revised Original Indicator Original Description Original Original Reason Baseline Target Number of preschool Interventions to improve a 0 200 The wording was revised classrooms and preschool classroom to better measure Project complementary would comprise: (a) the activity. Target values facilities improved construction and/or were revised downward and equipped in rehabilitation of because of the increase in selected classrooms; (b) the price of materials and municipalities complementary facilities the additional costs for including, but not limited complementary facilities to toilets, recreational and equipment for all areas, water/sanitation, preschools under the and security; and (c) the Project. provision of furniture and equipment. Revised Indicator Revised Description Revised Revised Baseline Target Number of preschool Unchanged 0 106 classrooms built/rehabilitated and equipped in selected municipalities IRI Indicator 2 - Revised Original Indicator Original Description Original Original Reason Baseline Target Preschool unified One curricular framework No Yes The description was revised. curriculum and three preschool Instead of “published in a developed classroom instruction short document,� the one guides would be integrated unified preschool in one unified preschool curriculum integrating one curriculum approved by the curriculum framework and Page 71 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) Executive Directorate of the three preschool MINED and published in a classroom instruction short document. guides would be “available Revised Indicator Revised Description Revised Revised on the MINED website.� Baseline Target Those documents had Preschool unified One curricular framework No Yes already been distributed to curriculum and three preschool all preschools, but they developed classroom instruction needed to be available for guides would be integrated the public to make in one unified preschool informed decisions on curriculum approved by the sending children to Executive Directorate of preschool. MINED and available on the MINED website IRI Indicator 3 - Revised Original Indicator Original Description Original Original Reason Baseline Target Increase in the Total number of 44% 73% The indicator was modified to percentage of preschool teachers is (2700) register the amount of certified preschool 9,296. Total number community educators and teachers (Nation- of certified preschool preschool teachers that are wide) teachers in 2011 is certified under the Project. This 4,096 (44%). Total is due to the weaknesses in the number of preschool availability of information in teachers to be MINED with regard to the total certified by the number of community educators Project is 2,700 and preschool teachers currently (equivalent to 29% teaching in the classroom around of the 9,296 the country. For example, there preschool teachers were four separate figures in 2011). regarding the total number of Revised Indicator Revised Description Revised Revised community educators and/or Baseline Target preschool teachers teaching in Number of certified The number of 0 2,300 the classroom, as reported by preschool teachers preschool teachers the Human Resources (Nationwide) that have been Directorate, the Education certified under the Statistics Directorate, the Project. Preschool Directorate or the Teacher Training General Directorate. Therefore, it was impossible to rigorously monitor the total number of community educators and/or teachers and whether they are certified or not Page 72 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) to provide preschool education. Further, the original goal of the project was to certify 2,700 community educators and preschool teachers. However, due to the increase in the cost of scholarships for those educators and teachers from remote areas of the country (especially in the Caribbean Coast) it was necessary to increase investment per teacher and reduce the target to 2,300. IRI Indicator 4 - Revised Original Original Description Original Original Reason Indicator Baseline Target Number of Learning materials included in 0 14,000 The target value was preschools that each package will include revised according to receive learning small school supplies and toys official updated data on material to be distributed in all the number of preschools. packages preschools nationwide. The description was Revised Revised Description Revised Revised revised for better clarity of Indicator Baseline Target the indicator. Unchanged Two types of learning 0 8,500 material packages will be delivered: (a) educational materials, including dominos, puzzles, musical instruments, etc.; and (b) basic material packages, including crayons, watercolors, scissors etc. Preschools includes public and community preschools. IRI Indicator 5 - Revised Original Original Description Original Original Target Reason Indicator Baseline System to A new system will 0 Evaluation system This indicator was evaluate early be developed and designed and modified to better childhood piloted. The tools piloted; system reflect the reality of development developed to validation project activities. The outcomes in measure child conducted in a project had successfully place and development sample of the carried out a full Page 73 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) operational outcomes will be selected diagnostic assessment validated on a municipalities, and on the tools currently representative results published. used to measure ECD sample of the 43 from MINSA, MIFAN, municipalities. and MINED, as well a Revised Revised Description Revised Revised Target Conceptual and Indicator Baseline Methodological System to A new system will 0 (a) System to Framework for the evaluate early be developed and evaluate ECD system. However, the childhood piloted for outcomes in complexity of the development validation. The tools preschool and political and institutional outcomes and developed to preschool-age situation made it preschool-age measure ECD learning outcomes difficult to move to the learning outcomes and is designed and implementation stage of outcomes in preschool-age piloted; (b) system a single system managed place and learning outcomes validation jointly by the three operational will be applied to a conducted in a institutions, which sample that includes sample that assesses all of the a group from the 43 includes a group dimensions of ECD, and selected from the 43 to do so during the municipalities and selected Project cycle. As a result, which covers a wide municipalities and MINED opted to begin variety of contexts which covers a wide the implementation and regions, both in variety of contexts stage for only the rural as well as and regions, both in aspects related to the urban areas. rural as well as assessment of ECD and urban areas, and learning from the results published. perspective of preschool education. This process would allow for the implementation of the system at the national level shortly after the end of the project. The description of the indicator and the target was modified to reflect this change to the indicator. IRI Indicator 6 - Revised Original Indicator Original Description Original Original Reason Baseline Target Number of core Interventions to improve a core 0 200 The target value was schools (K-9) with school facility would comprise: revised downward Page 74 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) facilities improved (a) two new and/or because of (a) higher for lower rehabilitated classrooms on costs per school with secondary average per school; (b) the change to a more education improved learning facilities holistic approach; (b) (ICT, libraries, and vocational an increase in the workshops); (c) classroom price of materials; furniture and blackboards; and and (c) a loss in the (d) complementary facilities, exchange rate of such as improved sanitation, euros to U.S. dollars. water supply, etc. Revised Indicator Revised Description Revised Revised The description was Baseline Target revised to clarify that, under the revised strategy, the number Unchanged Interventions to improve a core 0 75 of classrooms to school facility would comprise: rehabilitate was (i) eight new and/or increased to 8 instead rehabilitated classrooms on of 2, as originally average per school; (ii) designed. improved learning facilities (ICT, libraries, vocational workshops); (iii) classroom furniture and blackboards; and (iv) complementary facilities, such as improved sanitation, water supply, etc. IRI Indicator 7 - Revised Original Indicator Original Description Original Original Reason Baseline Target A university-based Universities will sign a contract No 900 The target value certification with MINED for the was revised program for lower- implementation of the university- downward to secondary teachers based certification program and reflect a more implemented the enrollment of secondary realistic expected school teachers in the three-year number of lower program to end in 2014. secondary teachers Revised Indicator Revised Description Revised Revised that have Baseline Target graduated from Unchanged Universities will carry out, in No 750 university. Dropouts coordination with MINED, a were higher than certification program for originally projected. secondary school teachers that The description of lasts three years and which will the indicator was Page 75 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) grant the certification for modified to better ‘secondary education teacher reflect the content with superior technical level’. of the course. IRI Indicator 8 - Revised Original Indicator Original Description Original Original Reason Baseline Target Number of 10 days of in-service 0 1,640 The indicator was revised to secondary school courses would be offered align with the final MINED teachers who by the eight Teacher strategy for in-service participate in the Training Institutes training, which is based on 22 20-days in-service (Escuelas Normales) in days of training with one day teacher upgrading 2014 and 2015. per week instead of 20 days training based on as originally designed. The the secondary description was revised curriculum because all in-service courses Revised Indicator Revised Description Revised Revised would be implemented in Baseline Target 2016 instead of 2014 and Number of Number of secondary 0 1,600 2015, as originally scheduled. secondary school school teachers Finally, the target value was teachers who participating in 22 days revised downward to 1,600 participate in the of in-service courses. secondary teachers (instead 22-day in-service of 1,640). According to teacher upgrading updated school data, there training based on are 1,600 secondary teachers the secondary in the four subjects curriculum. (mathematics, natural sciences, social sciences, and languages) in the 43 targeted municipalities. IRI Indicator 9 - Revised Original Indicator Original Original Original Reason Description Baseline Target Number of Textbooks would 0 1.1 million The target value was revised secondary be distributed for because the updated textbooks grades 7 to 11 in number of students in public distributed five subject lower secondary schools (1.2 matters according million) was higher than to table 2.5 of the estimated (1.1 million). In PAD. addition, the Project would Revised Indicator Revised Revised Revised provide (a) about 200,000 Description Baseline Target textbooks in 5 subjects to all Unchanged Unchanged 0 1.5 million students in grades 10 and 11 Page 76 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) in public upper secondary schools; (b) about 100,000 textbooks to students under the Rural Secondary Strategy; and (c) about 10,000 textbooks in 5 subjects to all teachers in grades 7, 8, 9, 10, and 11 in public secondary schools. IRI Indicator 10 – Unchanged Number of teaching guides distributed to secondary school teachers IRI Indicator 11 - Revised Original Indicator Original Description Original Original Reason Baseline Target Personnel from the No Yes The indicator was revised Directorate of because of the elimination of the Preschool training of personnel from the Education are GDPE in basic management trained in basic techniques. Project activities management have been focused on techniques strengthening the capacity of the Revised Indicator Revised Description Revised Revised Directorate of Preschool Baseline Target Education to (a) elaborate and Directorate of Directorate of No Yes develop a unified curriculum and Preschool Preschool Education classroom instruction guides; Education is receives equipment, and (b) implement/supervise equipped for including vehicle for Project activities on ECD Project supervision of (preschool teacher training, use implementation Project activities and of the new tools, etc.). The and supervision computers to wording and description were activities. develop curriculum changed consequently. and classroom guides. IRI Indicator 12 - Revised Original Indicator Original Description Original Original Reason Baseline Target General Three GDSI offices are to be 0 3 The description was Directorate of created in Boaco, Leon, and revised to reflect School Zelaya Central. Each office will MINED’s strategy for Infrastructure include basic facilities for the works control and (GDSI) offices set GDSI to conduct pre-feasibility supervision. Instead of Page 77 of 78 The World Bank Education Sector Strategy Support Project ( P133557 ) up in two studies, as well as the contracting ‘firms’, as departmental monitoring and supervision of stipulated in the education civil works contracted to firms. original description, delegations and works monitoring and one territorial supervision would be division of RAAS. contracted to Revised Indicator Revised Description Revised Revised ‘individual Baseline Target consultants’. Unchanged Three GDSI offices are to be 0 3 created in Boaco, Leon, and Zelaya Central. Each office would include basic facilities for the GDSI to conduct pre- feasibility studies, as well as the monitoring and supervision of civil works contracted to individual consultants. IRI Indicator 13 – Unchanged Development of infrastructure maintenance guides/norms IRI Indicator 14 - Revised Original Indicator Original Description Original Original Reason Baseline Target Direct Project Total number of pre-primary 551,000 569,000 The description was Beneficiaries, of and secondary school children revised because (a) the which percent enrolled in 2011 in the 43 indicator should not be female. selected municipalities as limited to 2011, as reported by MINED. stipulated in the original Revised Indicator Revised Description Revised Revised description, but it Baseline Target should be a cumulative Unchanged Total number of pre-primary 0 585,000 number; and (b) Project and secondary school children beneficiaries are not enrolled at the time of limited to the 43 distribution of textbooks and selected municipalities, total number of teachers in but are nationwide. pre-primary and lower Baseline, annual, and secondary schools at the time target values were of distribution of teaching revised and updated materials, as reported by with recent education MINED. data. Page 78 of 78