Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) Report Number: ICRR0024211 1. Project Data Project ID Project Name P163218 Cote d'Ivoire: ESDEP Country Practice Area(Lead) Cote d'Ivoire Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-A7376,TF-B2775 31-Dec-2023 48,252,237.89 Bank Approval Date Closing Date (Actual) 04-Jun-2018 31-Dec-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 50,696,000.00 50,696,000.00 Revised Commitment 50,696,000.00 48,252,237.89 Actual 48,264,373.62 48,252,237.89 Prepared by Reviewed by ICR Review Coordinator Group Diana Goldemberg Salim J. Habayeb Susan Ann Caceres IEGHC (Unit 2) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Grant Agreement (page 6) and the Project Appraisal Document (PAD, page 11), the project objectives were as follows: • To increase access to preschool; and • To improve learning outcomes in beneficiary primary schools in participating regions. Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) The PDO was not revised; however, the PDO and intermediate indicator targets were revised upward to reflect the Additional Financing (AF) for the project. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 30-Jun-2020 c. Will a split evaluation be undertaken? No d. Components Component 1. Improving Conditions for Improved Learning Outcomes (appraisal: US$13.2 million; AF: US$10.3 million; planned: US$23.5 million; actual: US$24.2 million): This component supports the Education Sector Plan (ESP) by (i) increasing pre-schooling opportunities, particularly in rural areas, through piloting a community based preschool model; and (ii) improving in-service early grades teacher training in reading and mathematics, along with providing increased pedagogical support and administering learning assessments – specifically, an Early Grade Reading Assessment (EGRA) and an Early Grade Mathematics Assessment (EGMA) each year. Component 2. Strengthening Sector Management and Accountability (appraisal: US$2.9 million; AF: US$0.5 million; planned: US$3.4 million; actual: US$4.4 million): This component aims to strengthen accountability within the education system at school level. This would be achieved by (i) encouraging citizen participation in school management; (ii) piloting a results-based financing approach through the provision of grants to schools for improved performance; and (iii) building capacity in education policies and service delivery by supporting the existing National Education Task Force. Component 3. Improving Teaching and Learning Environment for Beneficiary Schools (appraisal: US$2.7 million; AF: US$13.9 million; planned: US$16.6 million; actual: US$11.8 million): This component seeks to improve the learning environment by (i) financing the rehabilitation and construction of primary classrooms, including water and sanitation works; (ii) supporting school health activities that included deworming and handwashing campaigns; (iii) providing complementary school kits to girls in Grades 1-3. Component 4. Project Implementation and Monitoring and Evaluation (appraisal: US$4.6 million; AF: US$2.5 million; planned: US$7.1; actual: US$7.9 million): This component focuses on capacity building for the ministry and the Project Implementation Unit (PIU) working on the project. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. At appraisal in June 2018, the total project costs were estimated at US$ 23.35 million. In June 2020, an Additional Financing (AF) was processed to include an additional grant, revising the estimated Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) costs to US$ 50.696 million. The actual disbursements at project closing were US$ 48.264 million, with the undisbursed amount attributable to a Performance Based Condition (PBC) that was not met. Financing: The project was financed by two grants from the Global Partnership for Education (GPE), the first committed at appraisal and the second as AF, totaling US$ 50.696 million. Borrower Contribution: There was no planned Borrower contribution. Dates: The project was approved on June 04, 2018, and became effective on July 31, 2018. An AF was processed on June 30, 2020. Its mid-term review was held on May 17, 2021. The project closed on December 31, 2023, which was its originally scheduled closing date as of the AF approval. 3. Relevance of Objectives Rationale At appraisal in 2018, Cote d’Ivoire’s basic education system faced several challenges. It had not yet achieved universal primary education completion, with a rate of 63% for primary education, compared to an average of 73% in Sub-Saharan Africa. Learning outcomes were also very low. To improve primary education outcomes, access to early childhood services was deemed crucial; however, preschool coverage stood at only 8.8%. The Government had developed an Education Sector Plan (ESP 2017-2025) aimed at increasing equitable access to various levels of the education system. This project was designed to support the implementation of the ESP. Its objectives were highly relevant and aligned with the Country Partnership Framework (CPF) for FY2016-2019, specifically supporting the focus area of building human capital for economic development and cohesion. The project remained aligned with the FY2023-2027 CPF, which reiterated the focus on improving human capital and reducing spatial disparities. The project and its objectives remained relevant amidst changing circumstances during implementation, particularly due to the COVID-19 pandemic. The additional finance approved in 2020 allowed for the scale- up of original project components and support to additional activities to mitigate the pandemic’s impact, while the PDO remained unchanged. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) OBJECTIVE 1 Objective To increase access to preschool. Rationale The theory of change for this objective was clear. The project aimed to increase preschool opportunities, particularly in rural areas, by piloting a community-based preschool model. These activities were likely to lead to the intended outcome of increased access to preschool. Outputs: • 117 community preschools were built by the project, meeting the target of 117 preschools. These schools are located in rural areas and serve children ages 4 to 5. The preschools are equipped with teaching materials and early childhood development kits, and educators have received training sessions through a three-month pre-service and additional in-service sessions. • 240 preprimary classrooms were constructed, exceeding the target of 225 classrooms. Since 2019, the Ministry of National Education and Literacy (MENA) has required the construction of a preprimary class in all primary schools benefiting from new construction or rehabilitation programs. The project also financed a two- week induction training course for 225 preprimary teachers assigned to the beneficiary localities. • 38,271 students benefitted from direct interventions to enhance learning in Early Childhood Education, which is below the revised target of 47,970 students (80% achieved). The original target was increased when the AF was approved, but it was not met due to implementation delays. The preschool pupils could only start in the 2020/21 school year, rather than in the 2019/20 school years as initially expected. Outcomes: • 349 preschools met quality standards, exceeding the revised target of 342 preschools. The original target was 90 schools, but it increased to 342 schools when the AF was approved. Quality standards have been developed with UNICEF support and take into account construction, equipment, tools and teaching practices. Rating Substantial OBJECTIVE 2 Objective To improve learning outcomes in beneficiary primary schools in participating regions. Rationale At approval, the project envisioned several activities: the development and distribution of teaching materials in math and reading, in-service training for teachers, improvements to infrastructure in beneficiary primary schools, and strengthening management and accountability in the education sector. These efforts were expected to plausibly contribute to improved learning outcomes. The AF scaled up several activities, maintaining the focus on the original six participating regions while increasing the number of beneficiary primary schools, which correspondingly raised the project targets (no target was revised downward). Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) Outputs: • 356,986 students benefitted from direct interventions to enhance learning, exceeding the target of 345,000 students. • 7,582 teachers were recruited or trained, exceeding the target of 2,250 teachers. Although the project initially planned to train only the teachers assigned to the first three grades of primary education, it ultimately trained all the teachers in the beneficiary schools. • 100 teachers received at least one supervision visit per quarter from inspectors or pedagogic advisers in beneficiary schools, exceeding the target of 90 teachers. • 100 beneficiary public schools reported school grants and students results through community briefings, exceeding the target of 90 schools. • 730 classrooms were rehabilitated or built by the project, exceeding the target of 675 classrooms. • 10,508 girls received complementary school kits, exceeding the target of 4,500 girls. Although the kits were initially reserved for girls in the first three grades of primary education, they were ultimately distributed to all girls in the beneficiary schools, to avoid a sense of inequity within the community. • 198,406 pupils benefited from deworming, below the target of 240,000 pupils. According to the ICR (p.19), the deworming target was missed due to a misunderstanding between the health department and the Ministry of Education regarding the targeted beneficiaries. The project was to provide deworming kits for project- supported schools. However, when the health teams arrived in the six project regions, they used the deworming kits for all schools in the area, not just those targeted by the project. As a result, some non- beneficiary schools received the deworming kits, while some of the project-supported schools did not receive them because the health teams ran out of the kits purchased by the project. • 3 large-scale learning assessments were completed, meeting the target of 3 assessments (EGMA, EGRA, PASEC). • 6 policy notes were published by the MENETFP, falling short of the target of 9 policy notes. • An impact evaluation report was published by the MENETFP, achieving the target. The evaluation found positive impacts of both the preschool and primary school components of the project, suggesting that the main channel for these impacts was an improvement in teachers’ pedagogical performance. Outcomes: • 64% of second grade pupils met minimum competency standards for mathematics in beneficiary primary schools, exceeding the target of 32% and the baseline of 24% of pupils. • 21% of second grade pupils met minimum competency standards for reading in beneficiary primary schools, falling short of the target of 25% but above the baseline of 6% of pupils. The late delivery of textbooks for students may partly explain this lower result. • 94% of teachers in beneficiary primary schools implemented at least three of the new teaching practices developed for mathematics, exceeding the target of 80% of teachers. This was monitored through a classroom observation tool, developed by the MENETFP, and applied by pedagogical advisors from the local level education units, who visited the beneficiary schools about once per quarter. • 92% of teachers in beneficiary primary schools implemented at least three of the new teaching practices developed for reading, exceeding the target of 80% of teachers. Monitoring was the same as the above point. Rating Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Both objectives (to increase access to preschool, and to improve learning outcomes in beneficiary primary schools in participating regions) were almost fully achieved and rated Substantial. Thus, the overall efficacy is rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Design: The economic analysis prepared at appraisal and completion followed the same methodology: a Cost- Benefit Analysis conducted for the original project and revised at the AF. Expected benefits arise from increased schooling for project beneficiaries, which translates into higher labor productivity, while costs reflect the actual project expenses. According to the ICR, the resulting Economic Rate of Return (ERR) was 12.5 percent for the original project and 12.9 for the AF. The ICR revised some of the economic analysis assumptions based on an impact evaluation report funded by the project, which used a randomized control trial method and produced results above the original estimates. Implementation: The project was planned for four years, with an eighteen-month extension of the closing date due to the AF, which more than doubled its original commitment amount. The project closed on time, with 95% of committed resources disbursed (the undisbursed balance is due to an unmet PBC). Implementation progress was mostly on track during the project, with all Implementation Status & Results reports (ISRs) rating it as moderately satisfactory or satisfactory. There were aspects of implementation that reduced project efficiency, such as the initial delay in the delivery of teaching and learning materials to the beneficiary schools in 2018/2019, changes in construction prices, and the effects of the COVID19 pandemic, which impacted several activities in the project and led to school closures. Political tensions linked to the 2020 elections also resulted in temporary school closures. The development of distance learning courses alleviated the later issues. Despite these challenges, project-supported activities were completed without any major issues. According to the ICR (p. 21), the PIU had managed a previous World Bank-financed operation, and the experience and stability of its staff positively impacted the project’s implementation efficiency. The turnover of Bank staff was also low, with two Task Team Leaders (TTLs) during the project’s 5.5 years of implementation. The efficiency of the project was rated Substantial, considering the value for money and cost-effectiveness of interventions, but with some negative aspects of implementation that moderately reduced overall project efficiency. This level of efficiency is what would be expected in the operation’s sector. Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  12.50  Not Applicable 100.00 ICR Estimate  19.40  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives is rated High due to full alignment with the country's context, its development plans, and the World Bank's CPF. Efficacy is rated Substantial, as development objectives were almost fully achieved. Efficiency is also rated Substantial, based on the value for money and cost-effectiveness of interventions, with some aspects of implementation that moderately reduced overall project efficiency. The aggregation of these findings supports a Satisfactory outcome rating. This is indicative of only minor shortcomings in the project’s overall preparation, implementation, and achievement. a. Outcome Rating Satisfactory 7. Risk to Development Outcome No significant risks to maintaining development outcomes were identified. The project covered six regions of Côte d’Ivoire, and a national scale-up of its activities is the object of a Bank-supported PforR operation (PRSEP 2023-2028, P177800). Additionally, education represented approximately 12 percent of the government’s 2023 budget. With the continued support of the government and donors for the education sector, Côte d’Ivoire should be able to sustain and expand upon education improvements. 8. Assessment of Bank Performance a. Quality-at-Entry Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) The project was closely aligned with the Government’s education sector plan and the GPE’s strategic plan, and it received endorsement from the donors group supporting education. The project design considered national and international evidence of effective practices, and preparation included broad consultations with stakeholders. The major challenges not identified nor mitigated during the design phase – specifically, the political tensions related to the 2020 elections and the COVID-19 pandemic – were factors outside the control of both the government and the Bank team. One important design weakness was highlighted by the ICR (p. 27): “the inclusion of the PBCs with national targets that had no connection with the project design which focused on six regions was a problematic design feature. This design feature proved to be challenging for implementation (...).” This requirement was mandated by GPE policy at the time of project preparation, which necessitated the use of national-level targets for PBCs. The ICR (p. 28) notes that GPE has subsequently modified its policy to address this issue. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision According to the ICR (p. 24), the project benefitted from a consistent Bank team that supported implementation, along with continued political support and full commitment from technical units. Bank implementation support missions regularly met with the PIU during periodic visits, reporting that the technical directors responsible for the various subcomponents showed a strong interest in the project’s progress and provided their full support. The project’s mid-term review allowed for a thorough assessment of project activities, which demonstrated strong implementation progress and results leading to the AF. The external factors that disrupted the 2019-2020 school year (political tensions linked to the 2020 elections and the COVID-19 pandemic) were effectively managed and did not fundamentally threaten the achievement of the project objectives. The Bank’s task team collaborated with the PIU to ensure that the project was closed on time and in an orderly manner. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The original Results Framework was simple and straightforward, soundly linked to the PDO through the Theory of Change prepared at appraisal. The indicators were relevant, clearly defined, and measurable. Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) The M&E design included the application of learning assessments (with related learning outcome indicators) and the use of classroom observation (with related teaching practices indicators). The project also included six PBCs, which were national targets per GPE requirements (see section 8), accompanied by clear verification protocols. According to the ICR (p.23), this was the first education project in Côte d’Ivoire designed as an IPF with PBCs. b. M&E Implementation The PIU implemented the M&E plan to collect and report key data. According to the ICR (p.25), the collected data were integrated into the ministry’s broader, well-functioning M&E and information system. Approximately 60 monthly progress reports were prepared and disseminated within the government, to the Bank, the donor community, and the public. Additionally, an impact evaluation was conducted at the end of the project, using a randomized design, by researchers from Université de Nantes. The evaluation found positive impacts of both the preschool and primary school components of the project, suggesting that the main channel for these impacts was an improvement in teachers’ pedagogical performance. c. M&E Utilization The monthly progress reports were utilized by stakeholders to monitor and evaluate project impact and inform decision making. Notably, this M&E data was used to inform the AF and scale up project activities. Finally, the lessons learned from the project were used by both the government and the Bank in designing the PforR operation (PRSEP 2023-2028, P177800) which capitalized on the project’s achievements. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified under Environmental Assessment Category B, as it triggered Safeguard Policy OP/BP 4.01 in view of school construction and civil works. It also triggered OP/BP 4.11 on Physical Cultural Resources and OP/BP 4.12 on Involuntary Resettlement. Consequently, an Environmental and Social Management Framework (ESMF) and a Resettlement Policy Framework (RPF) were prepared and publicly disclosed. The project’s grievance redress mechanism (GRM) registered and resolved 35 complaints. A gender-based violence (GBV) sensitization module was delivered to 3,912 stakeholders, and an evaluation of the capacity of the child protection committees in primary schools was conducted alongside capacity development activities. Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) Safeguards performance was rated Moderately Satisfactory or Satisfactory throughout project implementation, as reported in the Implementation Status Reports (ISRs). b. Fiduciary Compliance Financial Management: The project’s financial management aspects such as budgeting, planning, accounting, financial reporting, flow of funds, external audits, and PBC verification were adequate and remained unchanged throughout project implementation. FM performance was rated Moderately Satisfactory or Satisfactory in all ISRs. Procurement: The PIU procurement staff used the WB’s Systematic Tracking of Exchanges in Procurement (STEP) for managing procurement contracts, with occasional delays in archiving information and documents in STEP. The Bank’s team worked closely with the PIU to resolve any challenges that arose and provided capacity development, particularly related to STEP. Procurement performance was rated Satisfactory in all ISRs. c. Unintended impacts (Positive or Negative) The ICR reports (p.22) the scale up of the project, from a regional pilot to nationwide implementation supported by a PforR operation as a positive impact of the project. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Both ICR and ICRR rated Quality at Entry as Moderately Satisfactory and the Quality of Supervision as Satisfactory. Since both ratings are in the Bank Performance Satisfactory Moderately Satisfactory same range, the lower of the two ratings determines the rating of overall Bank Performance, thus resulting in a Moderately Satisfactory overall rating for Bank Performance. Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) Quality of M&E Substantial Substantial Quality of ICR --- High 12. Lessons The ICR (pages 28-29) offers insightful lessons, including (adapted by IEG): • Continuity in staffing both at the PIU and the Bank teams can foster cooperation to ensure effective project implementation. The PIU team had experience implementing a previous Bank- supported project, thus, it was familiar with Bank policies and procedures, and with ministerial staff. • Formulation of performance-based conditions (PBCs) that are project-specific, rather than national-level, can more adequately foster accountability. Because the project PBCs were formulated to monitor national level achievement, disbursements related to the PBCs relied partly on achievements outside the project scope. Subsequently, GPE has modified its policy on the use of national-level indicators to project-specific indicators. • Ensuring that teachers are well equipped is essential to improve learning. The project supported training of teachers, principals, and pedagogical advisors on the use of scripted lessons. This approach was successful in the project supported schools and is now being scaled up nationally. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a thorough overview of project preparation and implementation. The ICR was results oriented, internally consistent, reasonably concise, and compliant with guidelines. The quality of evidence and ICR analysis were adequate and aligned with the PDOs. The ICR was candid, and offered several useful lessons derived from the specific experience and findings of the project. a. Quality of ICR Rating High Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Cote d'Ivoire: ESDEP (P163218) Page 12 of 12