Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00006423 IMPLEMENTATION COMPLETION AND RESULTS REPORT Grant Numbers TF0A7376 and TF0B2775 ON THE GRANT OF US$ 23.5 MILLION AND AN ADDITIONAL GRANT OF US$ 27.35 MILLION FROM THE GLOBAL PARTNERSHIP FOR EDUCATION TO THE REPUBLIC OF CÔTE D’IVOIRE FOR THE EDUCATION SERVICE DELIVERY ENHANCEMENT PROJECT June 27, 2024 Education Global Practice Western and Central Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 8, 2024) Currency Unit = CFA Francs (CFAF) US$1= CFAF 606.38 FISCAL YEAR January 1 – December 1 Regional Vice President: Ousmane Diagana Country Director: Marie-Chantal Uwanyiligira Regional Director: Trina S. Haque Practice Manager: Waly Wane Task Team Leader(s): Yves Jantzem, Ali Coulibaly ICR Main Contributor: Sophie Cerbelle, Ezechiel Djallo ABBREVIATIONS AND ACRONYMS AF Additional Financing BCR Benefit-Cost ratio CAFOP Animation and Pedagogical Training Center (Centre d’Animation et de Formation Pédagogique) CBA Cost Benefit Analysis CP1 First Grade (Cours préparatoire 1) CP2 Second Grade (Cours préparatoire 2) CE1 Third Grade (Cours élementaire 1) CE2 Fourth Grade (Cours élementaire 2) CM1 Fifth Grade (Cours moyen 1) CM2 Seventh Grade (Cours moyen 2) COGES School Management Committees (Comités de Gestion des Etablissements Scolaires) CPF Country Partnership Framework DPFC Directorate of Pedagogy and In-service Training (Direction de la Pédagogie et de la Formation Continue) DSPS Directorate of Primary, Lower and Upper Secondary Schools (Direction des Strategies, dela Planification et des Statistiques) DVSP Directorate of Program Monitoring and Evaluation (Direction de la Veille et du Suivi des Programme) E&S Environmental and Safeguards ECD Early Childhood Development ECE Early Childhood Education EEP Eligible Expenditure Program EGRA Early Grade Reading Assessment EGMA Early Grade Mathematics Assessment ERR Economic Rate of Return ESDEP Education Service Delivery Enhancement Project ESMF Environmental and Safeguards Management Framework ESMP Environmental and Safeguards Management Plan ESP Education Sector Plan FM Financial Management GA Grant Agreement GBV/SEA Gender-Based Violence/Sexual Exploitation and Abuse GDP Gross Development Project GPE Global Partnership for Education GRM Grievance Redress Mechanism HDI Human Development Index HLO High-Level Outcome ICR Implementation Completion and Results Reports IRI Intermediate Results Indicators IRR Internal Rate of Return LEPG Local Education Partners Group MENA Ministry of National Education and Literacy (Ministère De l’Education Nationale Et De l’Alphabétisation) MENETFP Ministry of National Education, Technical Education and Vocational Training (Ministère De l’Education Nationale, de l’Enseignement Technique et de la Formation Professionnelle) M&E Monitoring and Evaluation MICS Multiple Indicator Cluster Survey NETF National Education Task Force NFM New Funding Model NGO Non-Governmental Organizations NPV Net Present Value PASEC Educational Systems Analysis Program (Programme d'Analyse des Systèmes Educatifs de la Conférence des Ministres Francophones) PAD Project Appraisal Document PBC Performance-Based Conditions PIU Project Implementation Unit PDO Project Development Objective PNAPAS National Early Learning Program RF Results Framework RPF Resettlement Policy Framework SC Steering Committee SDG Sustainable Development Goals SNAPAS National Early Learning Strategy SSA Sub-Saharan Africa STEP Systematic Tracking of Exchanges in Procurement TA Technical Assistance TOC Theory of Change UNDP United Nations Development Program UNESCO United Nations Educational, Scientific and Cultural Organization UNICEF United Nations Children’s Fund WCAES Western and Central Africa Education Strategy TABLE OF CONTENTS DATA SHEET ........................................................................ ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ........................................................ 5 A. CONTEXT AT APPRAISAL ........................................................................................................... 5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ............................................................... 12 II. OUTCOME ...................................................................................................................... 15 A. RELEVANCE OF PDOs .............................................................................................................. 15 B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................ 16 C. EFFICIENCY ............................................................................................................................. 20 D. JUSTIFICATION OF OVERALL OUTCOME RATING ..................................................................... 21 E. OTHER OUTCOMES AND IMPACTS .......................................................................................... 21 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 23 A. KEY FACTORS DURING PREPARATION..................................................................................... 23 B. KEY FACTORS DURING IMPLEMENTATION .............................................................................. 24 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 25 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................. 25 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 26 C. BANK PERFORMANCE ............................................................................................................. 27 D. RISK TO DEVELOPMENT OUTCOME ........................................................................................ 28 V. LESSONS AND RECOMMENDATIONS .............................................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ............................................................ 30 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 42 ANNEX 3. PROJECT COST BY COMPONENT............................................................................. 45 ANNEX 4. EFFICIENCY ANALYSIS ............................................................................................ 46 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 52 ANNEX 6. COMPARISON OF ORIGINAL AND AF COMPONENT ACTIVITIES .............................. 53 ANNEX 7. ADDITIONAL FINANCING PBC VARIABLE PART MATRIX.......................................... 55 ANNEX 8. MODIFICATIONS MADE TO INTERMEDIATE RESULTS INDICATORS ......................... 58 ANNEX 9. SUPPORTING DOCUMENTS .................................................................................... 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Cote d'Ivoire: Education Service Delivery Enhancement P163218 Project Country Financing Instrument Cote d'Ivoire Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Ministry of Economy and Finance Ministry of National Education and Literacy Project Development Objective (PDO) Original PDO The Project Development Objective is to increase access to pre-school and improve learning outcomes in beneficiary primary schools in participating regions Page 1 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 23,350,000 23,350,000 23,349,333 TF-A7376 27,346,000 27,346,000 24,915,041 TF-B2775 Total 50,696,000 50,696,000 48,264,374 Non-World Bank Financing 0 0 0 Total 0 0 0 Total Project Cost 50,696,000 50,696,000 48,264,374 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 04-Jun-2018 31-Jul-2018 17-May-2021 31-Dec-2023 31-Dec-2023 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 01-Nov-2018 Satisfactory Satisfactory 2.11 02 02-Jun-2019 Satisfactory Moderately Satisfactory 4.71 03 17-Dec-2019 Moderately Satisfactory Moderately Satisfactory 10.30 Page 2 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) 04 30-Jun-2020 Moderately Satisfactory Moderately Satisfactory 13.48 05 14-Dec-2020 Satisfactory Satisfactory 26.41 06 30-Jun-2021 Satisfactory Satisfactory 31.24 07 16-Dec-2021 Satisfactory Satisfactory 31.60 08 25-Jun-2022 Satisfactory Satisfactory 36.63 09 23-Dec-2022 Satisfactory Satisfactory 39.84 10 16-May-2023 Satisfactory Satisfactory 43.37 11 20-Dec-2023 Satisfactory Satisfactory 48.24 SECTORS AND THEMES Sectors Major Sector/Sector (%) Education 100 Early Childhood Education 34 Public Administration - Education 32 Primary Education 34 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100 Gender 76 Education 100 Access to Education 31 Education Financing 76 Teachers 88 Education Governance, School-Based 44 Management Education Facilities 76 Page 3 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Ousmane Diagana Country Director: Pierre Frank Laporte Marie-Chantal Uwanyiligira Director: Dena Ringold Trina S. Haque Practice Manager: Meskerem Mulatu Waly Wane Task Team Leader(s): Patrick Philippe Ramanantoanina Yves Jantzem, Ali Coulibaly Sophie Jackie Danielle Cerbelle ICR Contributing Author: Discour Page 4 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Country Context. At the time of appraisal in February 2018, Côte d'Ivoire had become more stable after a decade of socio-political instability marked by low economic growth real Gross Domestic Product (GDP) growth of 1.1 percent per year between 2000 and 2010, compared to 5 percent per year in Sub-Saharan Africa (SSA) over the same period. At the same time, the poverty rate remained high, with 46.3 percent of the population living below the national poverty line in 2015. Located in West Africa, Côte d'Ivoire was a country with a population estimated at 24,295,000 inhabitants in 2017. The GDP per capita (US$1,495 in 2016) was close to the average for SSA (US$1,636). 2. Important macro-economic achievements had been observed, but the situation in the social sectors was still below that of countries with a comparable level of income. Human development outcomes (as measured by key indicators) were low compared to countries with a similar per capita income. Côte d’Ivoire was ranked 171 out of 187 countries in the 2015 United Nations Development Program (UNDP) Human Development Index 1. The under-five mortality rate was 92.6 deaths per 1,000 births, against 83.2 per 1,000 SSA and 52.8 in lower middle-income countries. Life expectancy at birth was 51.2 years in Côte d’Ivoire, while it was 58.1 years on average in SSA and 67 in lower middle- income countries. In education, the average years of schooling was equal to 7.68 years, compared to an average of 8.2 in the region 2. 3. The demographic pressure on the education system was (and is still) a challenge. The annual population growth rate (2.6 percent), coupled with a compulsory education policy for 6 – to 16-year-olds adopted in 2015 3, implied an increased need for teachers, teaching materials, schools, classrooms, sanitation and water facilities. These inputs would be crucial to meet the prerequisite in terms of improving overall access rates, while guaranteeing the quality and strong governance of educational services, 4 all of which being essential elements to the strengthening of human capital in the country. Sector and Institutional Context 4. Sector context. Côte d’Ivoire had a favorable demographic context for education, but it had not yet achieved universal primary education completion and there were important social and regional disparities in the education system. Social demand for education services in Côte d’Ivoire was slightly low in comparison to other SSA countries, as the share of the school age population (6 to 18 years), stood at 22.3 percent of the total population, against 25.6 percent on average in the SSA region. Nonetheless, education completion rates remained low—63.1 percent for primary education and 35.5 1 As measured by key indicators available here: https://hdr.undp.org/data-center/human-development-index#/indicies/HDI 2 As measured by key indicators available here: https://hdr.undp.org/data-center/human-development-index#/indicies/HDI 3 With a phased implementation over 10 years, the compulsory education will be fully effective by 2025. 4 The Ivorian education system is organized around preschool level (two years in community preschool or one year in pre- primary); primary level of six years (CP1, CP2, CE1, CE2, CM1, CM2), lower secondary school collège for four years (6ème, 5ème, 4ème, 3ème), upper secondary lycée (seconde, première, terminale), technical and vocational education and training, and higher education. Primary and lower secondary (collège) are compulsory. Page 5 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) percent for lower secondary education, compared to 72.6 percent and 48.6 percent on average in SSA (2015)1. Gender disparities also existed as the primary education completion rate stood at 78.4 percent for boys, compared to 71.6 percent for girls (2017). Moreover, 55.5 percent of boys reached the end of lower secondary education against only 42.7 percent of girls. Regional disparities were also significant, with the access rate to the last grade of primary school varying by region from 2 percent to 74 percent according to the Living Standard Survey in 2015 [Enquête sur le Niveau de Vie (ENV)]. The low rate of completion rates at the primary education level was the combined result of inadequate access to education and low retention. The access rate to first grade of primary education was around 80 percent (ENV), while the retention rate was 89 percent. In some regions, access to education was limited due to lack of schools, with a high proportion of children walking more than 30 minutes to reach the nearest primary school. In other regions, empirical evidence from the 2015 household survey showed that, due to family financial constraints, only 85 percent of primary school-age children attended school even if the school was in reasonable walking distance (15 min). With regards to retention, many children who had easy access to school still dropped out before the end of primary education. This was a combined result of low quality, high opportunity cost, and schools with incomplete grades. 5. Lack of school readiness. Readiness to learn for primary-age children was affected by challenges facing Early Childhood Education (ECE) in both coverage and quality. Only 1 out of 10 children benefited from ECE. This coverage was far below rates in other countries (the average in Sub-Saharan African countries is 28 percent) but has been slowly increasing 5. At the time of appraisal, girls and boys had equal access to preschool, but inequalities existed between urban and rural areas. Around 62 percent of preschools were in urban areas, and they enrolled 74 percent of preschoolers. The enrollment rate for children from the poorest quintile was 1.5 percent and 51.4 percent from the richest quintile. 6 6. The primary sector exhibited poor learning outcomes. The results of the 2014 Educational Systems Analysis Program (Programme d'Analyse des Systèmes Educatifs de la Conférence des Ministres Francophones, PASEC) assessment, which compares 12 Francophone countries, showed students from Côte d’Ivoire scoring less than average in French, and among the lowest in mathematics. Moreover, PASEC results showed that boys scored 26 points higher than girls in mathematics. 7. Outdated teaching practices remained prominent, and learning in classrooms was not effective. The teaching and learning process was undermined by an inadequate pedagogical approach and lack of tools for early learning, weak teacher training, and limited professional development support. In-service teacher development programs were available, but they were not part of a coherent regional or national in-service training plan. 7 Despite the high proportion of qualified teachers (84 percent of teachers have a pedagogical diploma either because they had undergone initial training at teacher training center or because they have passed a diploma such as the "certificat d'aptitude pédagogique”), 8 strengthening the quality of pre- and in-service teachers training remained a challenge. Teachers were insufficiently supervised, mainly because of lack of human and financial resources. When it was provided, supervision focused little on improving classroom practices to foster learning. Regarding pedagogical materials, less than half of students had textbooks in reading/writing and mathematics. 8. The learning environment at primary level was inadequate in most of the country. There were strong disparities between regions and between rural and urban areas. For schools constructed with low-quality materials, incomplete 5 The current gross preschool enrollment rate is 11.19 percent based on the Ministry of National Education and Literacy, statistical yearbook 2022/2023. 6 UNICEF, Multiple Indicator Cluster Survey (MICS), 2016-2017 7 UNESCO, Rapport d’étude sur la formation initiale et continue des enseignants (2017). 8 UNESCO, Rapport d’étude sur la formation initiale et continue des enseignants (2017). Page 6 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) infrastructure, and lack of facilities (such as latrines), there was a high risk of school interruptions because of problems with students’ safety during the rainy seasons or lack of properly fenced school grounds. 9. Accountability was insufficient, leading to limited improvement in the quality of education. As underscored in the Project Appraisal Document (PAD), accountability along the services delivery chain was limited (between the central, regional, and school level) and a weak feedback monitoring system hampered the capacity of the system at all levels to gather and use data for systematic improvement of education provided in the country. The country was committed to monitoring quality with participation to international and national standardized student assessments. However, there had not been tangible progress on education quality because of lack of accountability at different levels and a poorly developed feedback monitoring system. School management committees (Comités de Gestion des Etablissements Scolaires, COGES) had been created but their capacity to contribute effectively to better school management and performance was weak. Education inspectorates and pedagogical counselors were decentralized at local levels, but they lacked resources because of the centralized financial management. The sector lacked institutional capacity to assess, develop and monitor policies to address the challenge education system was facing (low education quality, high dropout rates, social disparities in school). 10. Government Strategy. The Education Delivery Enhancement Project (ESDEP) was designed to support the implementation of the Government’s Education Sector Plan (ESP 2017-2025) that was endorsed by the donor community. Specifically, ESDEP contributed to the achievement of: (i) Outcome 1—children’s (3-5 years) access to quality preschool education; and (ii) Outcome 2—children’s (6-11 years) access to, and completion of, quality primary education. The project’s design built on planned activities in the ESP and the intermediate indicators were aligned with those of the ESP. Theory of Change (Results Chain) 11. The PAD presented the project’s Theory of Change (TOC) broken down by objective and component. The following consolidated TOC (Table 1) has been prepared based on the appraisal inputs and the project’s Results Framework (RF). Table 1. Theory of Change Challenges Activities Outputs Outcomes High level Outcomes Lack of school  Preschool construction  Community preschool established. Objective 1: readiness  Pre-service and in service training for  Quality standards implemented. Increased access preschool educators to preschool  Communities and parents training  Quality standards set up  Preschool and preprimary school kits Greater and supplies performance of Poor learning  Classrooms observation  Teachers trained on scripted Objective 2: human capital outcomes and  Student learning assessment pedagogy. Improved and education out of school  Teacher in service training  New in-service teachers training learning service delivery children  Teacher pedagogical support programs implemented. outcomes  Development of new pedagogical  Regular school visits in place. package for students and teachers  New pedagogical package available (decodable, decipherable, teachers in classrooms. “guide)  Teachers evaluated.  Training and resources for supervisors Page 7 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Lack of school  School grants  School grants and top ups delivered. management  Community training  School score cards in place. and system  Top up school grant on a performance-  Deworming delivered. accountability based mechanism.  School kits for girls delivered.  School score card  Classrooms successfully built.  Deworming Environment not  Complementary school kits for girls conducive to  Primary classroom rehabilitation / learning construction Lack of sector  Capacity building, equipment, studies  Capacity building and studies knowledge and  Impact evaluation delivered; equipment purchased. sector  Key analysis and policy notes coordination published.  Impact evaluation delivered. Project Development Objectives (PDOs) 12. The PDO was to increase access to pre-school and improve learning outcomes in beneficiary primary schools in participating regions 9. The PDO statement in the PAD and Grant Agreement (GA) are consistent. However, there was a discrepancy between the PAD and the GA related to the use of performance-based conditions (PBCs). The original PAD and project design include the use of PBCs, which included indicator targets, disbursement rules, and verification processes. However, the 2018 GA did not include the use of PBCs nor the related requirements. The GA did include categories for eligible expenditures related to the use of preschool and school grants as described below under the components section of this Implementation Completion and Results Report (ICR). This oversight was rectified in the new GA signed at the time of the 2020 Additional Financing (AF). 9The Project was implemented across six regions. These were the regions where primary education enrollment and efficiency rates were the lowest, and poverty levels were the highest: Bagoué, Béré, Bounkani, Kabadougou, Poro, and Tchologo. Page 8 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Key Expected Outcomes and Outcome Indicators 13. The achievement of the PDO was to be measured through six PDO indicators, directly linked to the overarching objectives and supplemented by eleven intermediate results indicators (IRIs). • Outcome 1: Increased access to preschool education in participating regions with the following indicator: (i) number of community preschools meeting quality standards 10. • Outcome 2: improve learning outcomes in beneficiary primary schools in participating regions with the following indicators: (i) percentage of pupils in second grade of primary education with intermediate or high-level score in reading, in beneficiary schools in poor and lagging communities (percentage girls); (ii) percentage of pupils in second grade of primary education with intermediate or high-level score in mathematics, in beneficiary schools in poor and lagging communities (percentage girls); (iii) number of teachers implementing at least three of the new teaching practices developed for mathematics in the project, in beneficiary schools; (iv) number of teachers recruited or trained (number of females); and (v) students benefiting from direct interventions to enhance learning 11. 14. The original project’s design also included three “stretch indicators”. The stretch indicators were based on the Global Partnership for Education (GPE) requirement that 30 percent of the project grant be used for PBCs that were directly linked to ‘stretch indicators.’ The stretch indicators needed to be related to the ESP and were to focus on the GPE three required domains of: (i) equity; (ii) efficiency; and (iii) learning outcomes. It should be noted that the ESP indicators were national in focus and to be measured based on national data. As indicated, the project was to be implemented in six of the most vulnerable regions of the country. Therefore, direct project attribution for achievement of the stretch indicators would not be possible. Achievement for the stretch indicators was to be monitored through the annual updating of the results framework associated with the ESP rather than the project specific results framework. The three PBCs related to the three domains at the time of appraisal were: (i) equity—percentage of children in preschool enrolled in community-based schools; (ii) efficiency—number of weekly hours of teachers in lower secondary education; and (iii) learning outcomes—score in reading and mathematics for grade 3 and grade 4. It was recognized that the efficiency indicator was not related to the project’s PDO or activities. However, it was the only ESP efficiency indicator and therefore, GPE wanted to include the indicator to comply with their efficiency domain requirement. Components 15. Component 1. Improving Conditions for Improved Learning Outcomes (Initial US$13.19 million; AF US$10.33 million; Total cost US$23.52 million; Actual cost: US$24.25 million). The objective of Component 1 was to support the ESP and to align with the government’s strategy to increase pre-schooling opportunities, particularly in rural areas, 10 The quality standards were developed with technical assistance support and include five domains with associated categories and indicators. The five domains are: (i) Domain 1: Access—including geographical and the conditions for admission; (ii) Domain 2: Infrastructure and equipment—includes standards for space and the conditions to be met for infrastructure and equipment; (iii) Domain 3: Teaching/Learning—includes the elements likely to facilitate the didactic and psycho-pedagogical management of a preschool class; (iv) Domain 4: School environment—related to natural and socio-cultural conditions that constitutes the child’s learning environment that are like to influence the child including links between school and family; and (v) Domain 5: School governance—related to the elements that promote good governance of a pre-school institution that highlights the standards of administrative and pedagogical skills required for preschool school leadership and the level of community involvement. 11 The definition for this indicator indicates that the measure is for number of students in primary schools in the six areas supported by the project. Page 9 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) through: (i) piloting a community based preschool model; and (ii) in-service early grades teacher training in reading and mathematics along with increased pedagogical support and learning assessment. • Sub-component 1.1: Improving community preschool in beneficiary regions (Initial US$ 7.92 million; AF US$ 6.25 million; Total US$ 14.14 million). This sub-component was to pilot implementation of a two-year play-based community based preschool program that targeted children 4 to 5 years old in 117 communities. This was to be done through the provision of community grants for community construction of preschool spaces that would include handwashing facilities, child friendly latrines, and sets of furniture and teaching materials. Additionally, the sub-component was to support three months of pre-service training for the community preschool educators as well as develop minimum standards and monitoring tools for the community-based preschools. Under this sub- component, UNICEF was to help to implement activities related to: (i) teacher training; (ii) development of monitoring tools and minimum standards, and (iii) strengthen capacity of government agencies and communities (COGESs) • Sub-Component 1.2: Improving teaching and learning of reading and mathematics in beneficiary primary schools (Initial US$ 5.27 million; AF US$ 4.08 million, Total US$ 9.35 million). This sub-component aimed to improve in-service teacher training for teachers in 150 primary schools through a new methodological approach in reading and mathematics (scripted pedagogy) and tools (decodable, decipherable). It also aimed to improve pedagogical support for teachers through provision of training for pedagogical counselors in how best to support teachers as well as teacher classroom observations. Pedagogical support also included the provision of educational materials, and transportation costs for the counselors. Additionally, feedback from the classroom observations was to be used to improve training content materials. Finally, there was support for the administration of an Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA) each year to students. 16. Component 2: Strengthening Sector Management and Accountability (Initial US$ 2.85 million; AF US$ 0.53 million; Total coast US$ 3.38 million; Actual cost: US$ 4.39 million). The main objective of Component 2 was to support the government to strengthen accountability within the education system, in particular, at school level. This was done by focusing on: (i) encouraging citizen participation in school management; (ii) piloting the using a results-based financing approach through the provision of grants to schools for improved performance; and (iii) building capacity in public education policies and service delivery. • Sub-Component 2.1: Citizen participation in school management (Initial US$ 0.64 million; AF US$ 0.30 million; Total US$ 0.94 million). This sub-component aimed to build capacity for 150 communities to: (i) implement of school activities, monitor service performance and report on the quality-of-service performance; and (ii) strengthen accountability mechanism in the education system, through community (COGESs) and parents’ participation in school management. The project was to support training, development of training materials, and development and publication of school score cards. • Sub-component 2.2: Results-based financing (Initial US$1.34 million; AF US$0 million; Total US$1.34 million). This sub-component was to pilot a results-oriented financing mechanism for schools through the provision of grants 12 to 150 communities. The grants would be provided to COGESs based on an index calculated in the score card. 13 12 The school grants were to consist of a top up amount with two main goals that included: (i) consideration of the need to increase the resources for most of the schools and (ii) rewarding the better performing schools to enhance the value for money and their ability to strengthen the linkages between resources and outcomes. The grants were to be transferred to COGES bank accounts. 13 The index was to consider the actual performance of the schools to encourage the schools already doing well to stay on track as well as progress to reward improvements made by schools with low outcomes at the initial stage. Page 10 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) • Sub-Component 2.3: Building capacity in public education policies and service delivery (Initial US$0.87 million; AF US$0.23 million; Total US$1.10 million). This sub-component aimed to support institutional capacity building in sector policy and management by contributing to specific studies, production of policy notes and key sectoral documents. Support the existing National Education Task Force, which covers three sub-sectors (basic and secondary education, technical and vocational training, and higher education) as well as the Directorate of Primary, Lower and Upper Secondary Schools (Direction des Strategies, dela Planification et des Statistiques— DSPS). The project was to support technical assistance (TA), training, surveys as well as materials, equipment and operating costs. 17. Component 3: Improving Teaching and Learning Environment in Beneficiary Schools (Initial US$2.69 million; AF US$13.95 million; Total cost US$16.64 million; Actual cost: US$11.75 million). The main objective of Component 3 was to improve the learning environment to address issues related to overcrowded classrooms, student’s absenteeism, and girls’ education. Component 3 was to finance rehabilitation and construction for 144 primary classrooms as well as furniture, latrines, and when needed and accessible, water supply. Civil works was to be implements by the COGESs and carried out based on the community approach. 14 This component was to also support school health activities that included deworming campaigns and handwashing campaigns. Finally, the enhancement of girls’ education, through the provision of complementary school kits to girls in Grades 1, 2 and 3, from poor households to ease financial burden for their families. 18. Component 4: Project Implementation and Evaluation (Initial US$4.62 million; AF US$2.53 million; Total cost US$7.15 million; Actual cost: US$7.87 million). Component 4 aimed to support capacity building for ministry department working on the project, including TA, study tours, training, and equipment but also to ensure robust impact evaluation. • Sub-Component 4.1: Implementation (Initial US$2.66 million; AF US$ 2.08 million; Total US$ 4.74 million). This sub-component was to provide support to the Ministry of National Education, Technical Education and Vocational Training (MENETFP) Project Implementation Unit (PIU) to implement the project It also supported sensitization activities with all stakeholders, local authorities, communities, and parents (including workshops, community forums, mass media campaigns, and other relevant channels). • Sub-Component 4.2: Evaluation (Initial US$1.96 million; AF US$0.45 million; Total US$2.41 million). This sub- component aimed to support several evaluations, including: (i) monitoring and evaluation activities to be continuously carried out to provide updates on progress towards the targeted project outcomes; and (ii) randomized impact evaluations to assess the impact of project activities, in a two-step approach. As a first step, the project activities would assess to what extent classroom observations and related in-services teacher training activities could lead to improved teaching and learning. Secondly, an impact evaluation would assess whether improved teaching and learning and the incentives mechanism contribute to increasing the learning achievements of students. 14Under the community based approach, the resources were transferred to the community (school management committee) to select and hire (mostly) local firms. Support and technical assistance was provided by dedicated staff (civil works engineers) under the responsibility of the PIU. Page 11 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets 19. The project’s PDOs were not revised during the course of project implementation. However, the PDO and intermediate indicator targets were revised upward to reflect the increased financing for the project. The two PDO indicators related to learning achievement were restated to ensure that they were in line with the Sustainable Development Goal 4 (SDG4) 15. (Table 2) Additionally, the AF updated the PBCs based on the GPE requirements, and the AF Grant agreement ensured that the PBCs were included in Schedule 3 of the agreement along with the appropriate indicator targets and disbursement rules. (See Annex 7 for PBC details.) Revised PDO Indicators 20. At the time of 2020 AF modifications to the RF was modified as indicated in Table 2. Table 2: 2020 Additional Financing PDO Indicator Modifications Indicator Baseline Original Revised Comments 2018 Target Target July 2022 December 2023 Objective 1: Increased access to preschool in participating regions 1Community preschools meeting quality 0 90 342 Indicator revised: Number of preschools standards (number) meeting quality standards. The indicator has been slightly modified to remove the word “community”, to include both preschool models (i.e., the community preschools under the original project and the new preprimary preschools under AF). The target was increased to account for the additional preschools to be supported under the AF. Objective 2: Improved learning outcomes in beneficiary primary schools in participating regions 2. Pupils in second grade of primary 24 32 32 Indicator revised: Pupils in second grade of education with intermediate or high-level (50%) (50%) primary education meeting minimum scores in mathematics in beneficiary (unchanged) competency1 threshold for mathematics in schools in poor and lagging communities beneficiary schools in poor and lagging (Score) (percentage girls) communities. The indicator is revised to be in line with the SDG4 requirement of measuring minimum competency. 3. Pupils in second grade of primary 6 15 25 Indicator restated to “Pupils in second grade education with intermediate or high-level (Revised (50%) (50%) of primary education meeting minimum scores in reading in beneficiary schools in downward (unchanged) competency3 threshold for reading in poor and lagging communities (scores) to 1 in 2020 beneficiary schools in poor and lagging (percentage girls) AF2. communities” to be in line with the SDG4 requirement of measuring minimum competency. 15 SDG 4 aims to ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. Page 12 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) 4.Teachers implementing at least three of 0 80 80% Indicator revised and split into 2 different the new teaching practices developed in (math) indicators: the project in beneficiary schools 80% (i) % of teachers implementing at least three (percentage) (reading) of the new teaching practices developed for mathematics in the project in beneficiary schools; and (ii) % of teachers implementing at least three of the new teaching practices developed for reading in the project in beneficiary schools. 5. Teachers recruited or trained (Corporate 0 1.350 This indicator was moved to the Indicator) intermediate level. 6. Students benefiting from direct 0 144.0004 This indicator was moved to the intervention intermediate level. 1 The minimum competency in mathematics is 40 out of a score of 100 on the national standardized assessment using the EGMA assessment. 2 The original baseline was based on the 2016 PASEC. However, the baseline was modified based on a 2019 EGRA assessment that revealed that only 1 percent of pupils in second grade of primary schools were meeting the minimum competency threshold for reading at beneficiary schools. 3 The minimum competency level for reading is 40 out of a score of 100 on the national standardized assessment the using EGRA assessment. 4 The text of the 2020 Project Paper inaccurately indicated that the target was increased from 15 to 33 percent. The RF in the 2020 Project Paper and subsequent Implementation Status and Results Reports (ISRs) reported against the target of 25 percent. Revised Components 21. At the time of the 2020 AF some project activities were scaled up based on the successful implementation under the original project and some new activities were added. See Annex 6 for a comparison of original project components and AF components. The details of the AF modifications were as follows: Component 1: Improving Conditions for Improved Learning Outcomes. • Sub-component 1.1: Improving community preschool in beneficiary regions. This sub-component added construction and provision of equipment for an additional 225 preprimary schools along with provision of in- service training for teachers in these additional preprimary schools. The construction of the preprimary schools was to continue the community-based approach to construction and be financed by grants for the construction. MENA had enacted a new directive that required the construction of a preprimary classroom in all primary public schools and was included in the ESP. To assess each approach the AF was to finance and impact assessment of both models to inform MENA’s policy discussions related to the two different modalities. • Sub-Component 1.2: Improving teaching and learning of reading and mathematics in beneficiary primary schools. The AF added: (i) in-service teacher training for teachers in an additional 225 public primary school; (ii) classroom observations in the 225 new public schools; (iii) training for 1,350 teachers in early learning and teaching practices related to reading and mathematics; (iv) on-the-job training for an additional 100 pedagogical support staff in classroom observation tools; and (v) support for distance learning and training to address school closures due to COVID-19. The AF added a new activity related to the development of bridging classes for 4,500 out-of-school children in the six regions. The AF was to support workshops to define needs of out-of-school children and develop guidelines for the Directorate of Literacy and Non-Formal Education. The AF would also support the development of bridging classes and school kits for approximately 4,500 children. Component 2: Strengthening Sector Management and Accountability • Sub-Component 2.1: Citizen participation in school management. The AF scaled up support to capacity building activities for the 225 additional communities for implementation of school activities, monitoring service performance and reporting on the quality of expected results. • Subcomponent 2.2: Results-based financing. The AF did not provide additional resources for this sub-component. Page 13 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) • Sub-Component 2.3: Building capacity in public education policies and service delivery. The AF provided support for MENETFP’s institutional capacity building and inter-ministerial task force for one additional year. Component 3: Improving Teaching and Learning Environment in Beneficiary Schools. The AF provided support for rehabilitation and/or new construction of 675 additional classroom in the 225 public primary schools added under the AF. Construction was to be based on the community approach through the provision of grants to communities. approach. Component 4: Project Implementation and Evaluation. The AF continued support for the PIU for an additional 18 months and added a impact assessment that included a comparative analysis of the two preschool models. Other Changes 22. The changes made at the time of the June 30, 2020 AF included: (i) additional GPE funds for the project in the amount of US$27.34 million and component allocations; (ii) extension of the project’s closing date from July 28, 2022 to December 31, 2023 (17 months); (iii) modifications to the PDO indicators (Table 2) to reflect the scaled up operations; (iv) modifications to the intermediate indicators (Annex 8) to reflect the scaled up/modified activities; (v) modifications to project components as indicated; (vi) expansion and modification of the PBCs from three to six to reflect the additional resources under the AF—it should be noted that the PBC targets continued to be national targets that were directly aligned with the ESP as required by GPE; (vii) formalizing the PBCs in the GA in order to correct the omission made in the original GA (Annex 7); (viii) modifications to financial management to include the eligible expenditures against the PBCs; (ix) updating of the Environmental and Social Management Framework (ESMF) safeguards which was re-disclosed on February 12, 2020—no new safeguards policies were triggered with the AF; (x) revision of funds allocation to project’s components as indicated; and (xi) reallocation between disbursement categories and disbursement forecasts. Rationale for Changes and Their Implication on the Original Theory of Change 23. The modifications did not affect the original theory of change, but rather expanded and enhanced the project’s ability to further measure progress toward achievement of the PDO. The 2020 AF was important for scaling up the scope of the project in support of increasing access to preschool and primary schools in additional communities in the six project- supported regions. The AF provided coverage for an additional 225 preschools and 675 primary school classrooms in the same 225 communities. Through the AF, the project was able to reach approximately 240,000 additional students along with training of preschool teachers and primary school teachers in reading and mathematics to continue to improve learning outcomes in the supported schools. Additionally, the 2020 restructuring added support for distance learning and training programs to address challenges of face-to-face training that occurred due to COVID-19 and included activities to help address the challenges of out-of-school children in the project supported regions. These modifications fit directly into the TOC and supported the long-term goal of more children completing quality and equitable primary education thereby reducing learning poverty. 24. The AF PBCs using national targets directly related to the ESP was a GPE requirement. While achievement of these indicators could not be directly linked or show direct attribution to project interventions, they did allow, through project implementation, to monitor achievements of the ESP indicators which were designed to measure the system as a whole. The Local Education Partner Group (LEPG) agreed on this approach and the GPE approved this design feature for their grant resources. As indicated below, the targets were largely met with all but one PBC being fully disbursed. However, this design feature was unrelated to the TOC. Finally, it should be noted that GPE did recognize this challenge for the project. However, during implementation, they did not allow modifications to the PBCs because of the prevailing Page 14 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) GPE guidelines at the time. Subsequently, GPE has revised its policies and no longer requires national level indicators but rather indicators that are directly related to the project design and TOC. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 25. The PDOs were and remained Highly relevant to the country’s development priorities and sectoral needs at the time of the project was approved, throughout the course of project implementation, and at completion. 26. At the time of appraisal, the project was an integral part of the response to the World Bank’s Country Partnership Framework (CPF) 16, FY2016-2019. Specifically, the project supported Focus Areas 2 and 3: (i) building human capital for economic development and cohesion; and (ii) strengthening public financial management and accountability. The project remains in line with the FY2023-2027 CPF which supports the Government of Côte d’Ivoire in its goal of halving poverty levels and achieving middle-income country status by 2030. It was aligned with the High-Level Outcome (HLO) 1 and 2 which focused on improved human capital (HLO1) and reduced spatial disparities and strengthened resilience. Finally, it remains consistent with the Western and Central Africa Education Strategy (WCAES) 17 which envision “a region where all girls and boys come to school ready to learn, acquire real learning and are ready to enter the workforce with the right skills to become productive and fulfilled citizens”. Moreover, the project was directly aligned with achieving the goals, stated in GPE’s Strategic Plan (2016-2020) of: (i) improved and more equitable learning outcomes; (ii) increased equity, gender equality and inclusion; and (iii) effective and efficient management of the education system. The project remains relevant to the GPE 2025 Strategic Plan (GPE 2025) which focuses on strengthening national capacity and systems to address equity and the learning crisis in education through a more coordinated action and financing plan meant to enable transformative change. The project remains relevant and consistent with the WB’s Human Capital Project, in which education reforms and educational attainment are a means of increasing human capital accumulation, while addressing wide regional disparities (including in gender)—particularly for the rural and deprived areas—as well as the WB’s twin goals of ending extreme poverty and boosting shared prosperity. 27. The project remains fully aligned with governments education national strategies. The project was designed to support the implementation of the Government’s Education Sector Plan (ESP 2017-2025) endorsed by the government in May 2017. The project specifically contributed to the achievement of Outcome 1 which indicates that children ages 3-5 years have access to quality preschool education and Outcome 2 which indicates that children ages 6-11 years have access to and complete quality primary education. The project’s design was built on ESP planned activities and included PDO, intermediate and PBC indicators that were aligned with those of the ESP. The 2017-2025 ESP has been extended by one year (until end of 2026). Under the extended period of the ESP, the government has confirmed that the early learning reforms and related governance reforms piloted by the project will continue to be supported and scaled up by the ESP. In addition, the SDGs - 4 (quality), 5 (gender equality) and 10 (reduced inequalities) remained highly relevant. The project aimed at contributing to the achievement of the three SDGs. By focusing on quality of education and learning outcomes 16Report No. 96515-CI – August 17, 2015. 17Western and Central Africa Education Strategy, From School to Jobs: A Journey for the Young People of Western and Central Africa. (Washington, DC: World Bank, 2022). https://documents.banquemondiale.org/fr/publication/documents- reports/documentdetail/099317206202273482/p17614902262190750bc3405fc3ae20d77b. Page 15 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) in underserved areas with high poverty rates and low education achievements, the project sought to support the government’s efforts to ensure inclusive and quality education. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 28. The overall efficacy of the PDOs is rated as Substantial. This is based on the available evidence and the fact that both the project's two overarching objectives were substantially met. The rating is also based on the fact that four of the five PDO indicator targets were exceeded, one PDO was 84 percent achieved, and 10 of the 13 IRIs were met or exceeded. Additionally, five of the six PBC targets were met and fully disbursed. The one PBC that was not achieved was related to the efficiency indicator which, again, was a national level indicator which could not be revised. The AF did not downsize the project or the indicators and therefore, the ICR evaluation does not warrant a split evaluation. The following assessment of the PDOs is organized around two key objectives: (i) increased access to pre-school and (ii) improved learning outcomes. 29. The efficacy of Objective 1 - increase access to pre-school – is rated Substantial. The outcome indicator target was met and the target for 2 (out of 3) intermediate indicators were met as indicated in Table 3. Table 3: PDO 1: Increase access to pre-school Indicator Baseline Target Actual Status Outcome indicator PDO 1: Number of preschool meeting quality standards 0 342 349 Exceeded Intermediate Results indicators IRI 1.1 Number of community preschool built by the project 0 117 117 Achieved IRI 1.2 Number of preprimary classrooms built in beneficiary communities by the 0 225 240 Exceeded project IRI 1.3 Students benefitting from direct interventions to enhance learning in Early 0 47,970 38,271 Not met Childhood Education (ECE) (80% achieved) 30. PDO indicator 1 – number of preschool meeting quality standards – was achieved and exceeded expectations (expected target 342; achieved target 349). Quality standards have been developed with UNICEF support and apply to both Côte d’Ivoire community preschool centers and public preprimary. To date, 342 evaluations have been carried out confirming compliance with quality standards (regarding construction, equipment, tools, teaching practices) for community preschools and for preprimary. 31. IRI 1.1 related to the number of community preschool built by the project was achieved. Community preschools are aimed at children aged 4 to 5 (2 years of preschool) in rural areas. This type of preschool offers services of comparable quality to those offered in the regular public system; they are supervised by the ministry which also provides public funding for community educators. Using a community-based approach, civil works were completed for the expected 117 community preschools. Equipment such as teaching materials and early childhood development (ECD) kits are available in each of the centers. Several guides/manuals and module have been developed and made available to stakeholders i.e., guides for making toys and fun games from local materials, guides for community mobilization, the manual for community training, guides for preschool center creation and management and the parental education module. Educators and assistants have benefited from training sessions (pre-service and in-service training). For educators teaching in community preschool, the project supported a new three-month preservice training (two months training before the school year and Page 16 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) the one month during the long vacation at the end of the school year). The training was specific to the skills required to teach young children through a play-based approach, highly practical and encouraged the promotion of more holistic early childhood development. In addition, the preschool community assistants received two weeks of preservice training, focusing on the needs of young children and how to interact effectively with them. 32. IRI 1.2 related to the number of pre-primary classrooms built by the project was achieved. Since 2019, MENA guidelines require the construction of a pre-primary class in all schools benefiting from a new construction or rehabilitation program. Communities were trained in a community-based approach for civil works and oversaw the construction of 240 public pre-primary classrooms. Additionally, MENA recruited qualified teachers (as civil servants) for the new pre-primary classes. The project financed a two-week induction training course for 225 teachers assigned to pre- primary classes in beneficiary localities to ensure that these teachers assigned to beneficiary localities master the skills and knowledge specific to early childhood development. With the increased focus on ECE the training course has now been scaled up in the Animation and Pedagogical Training Center - Centre d’Animation et de Formation Pédagogique 33. IRI 1.3 related to the number of students benefitting from direct interventions to enhance learning in Early Childhood Education was 80 percent achieved. This was due to a slight delay in the implementation of the additional financing, which meant that the cohort of young pre-school pupils could not be taken on as early as the start of the 2019/2020 school year as first expected but could only start as early as the start of the 2020/2021 school year. In the meantime, an impact evaluation of these interventions (see later in the document) showed that preschool models under the project contributed to the continuation of primary schooling for children who attended preschools and the early acquisition of basic reading and mathematical skills. Based on the above, achievement of improved access to pre-primary education is rated Substantial. 34. The efficacy of Objective 2 - improve learning outcomes – is rated Substantial. Three out of four PDO indicators linked to this objective were exceeded and one was 84 percent achieved. Additionally, the target for eight (out of ten) intermediate indicators were met as indicated in Table 4. Table 4: PDO 2: Improve learning outcomes. Indicator Baseline Target Actual End Status (2018) (2023) Value Outcome indicators PDO 2: Pupils in second grade of primary education meeting minimum 24 32 64 Exceeded competency threshold for mathematics in beneficiary primary schools (47%) (50%) (51%) Exceeded in poor and lagging communities (percentage) (% girls) PDO 3: Pupils in second grade of primary education meeting minimum 6 25 21 Target missed competency threshold for reading in beneficiary primary schools in (84% achieved) poor and lagging communities (percentage) (% girls) (47%) (50%) (52%) Exceeded PDO 4: Teachers implementing at least three of the new teaching 0 80 94.1 Exceeded practices developed for mathematics in the project, in beneficiary schools (percentage) PDO 5: Teachers implementing at least three of the new teaching 0 80 91.7 Exceeded practices developed for reading in the project, in beneficiary schools (percentage) Intermediate indicator Students benefiting from direct interventions to enhance learning 0 345,000 356,986 Exceeded Teachers receiving at least one supervision visit by inspectors or 0 90 100 Exceeded pedagogic advisers in beneficiary schools, per quarter Teachers recruited or trained 0 2,250 7,582 Exceeded Page 17 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Large-scale primary/secondary learning assessments completed 0 3 3 Achieved Public schools reporting school grants and student results through 0 90 100 Exceeded community briefings meetings, at least once a year, in beneficiary schools Number of classrooms rehabilitated/built by the project 0 675 730 Exceeded Number of girls receiving complementary school kits 0 4,500 10,508 Exceeded Number of beneficiaries benefiting from deworming 0 240,000 198,406 Target missed (83% achieved) Number of policy notes published by the MENETFP 0 9 6 Target missed (66% achieved) Impact Evaluation report published by the MENETFP No Yes Yes Achieved 35. Progress towards achievement of PDO indicator 2 (pupils in second grade of primary education meeting minimum competency threshold for mathematics, in beneficiary schools in poor and lagging communities) exceeded expectations (expected target 32 percent; achieved target 64 percent). The project supported the production of early learning mathematics tools for the first three grades of primary education (CP1, CP2 and CE1). The concepts used were based on the most recent research in neurosciences which shows that when teaching methodologies are explicit and entail structured, scripted, and progressive lessons learning can be improved. The curriculum package encompasses decipherable, student’s notebook, lessons plans and teachers’ guides. All these supports were made available for beneficiary students, teachers, and supervisors – “conseillers pédagogiques”. The project strongly emphasized practicing specific classroom skills. This “practice-based” professional development supports “lesson fidelity”, which is the ability of teachers to implement the pedagogical plan they are given (e.g., lesson plans with tightly structured and effective pedagogy, and offers detailed guidance to aid teachers in low-capacity settings. 36. Progress achieved towards achievement of PDO indicator 3 (pupils in second grade of primary education meeting minimum competency threshold for reading in beneficiary schools in poor and lagging communities) was 84 percent achieved. Despite not meeting the end targets, the indicators show improvement in reading outcomes over project implementation, confirming that initial changes in pedagogical approach, tool and practices inside the classroom were occurring, although not at the expected speed. Indeed, the baseline for this PDO was 1 percent showing a very low level of skills of students in beneficiary school. The trend is clearly towards improving students' reading skills, since 8 percent of them met minimum competency in 2021; 25 percent in 2022 and 21 percent in 2023. The drop in results for CP2 pupils in 2023 may be explained by the late delivery of textbooks for CP1 students the previous year (2022). In fact, the pupils assessed in CP2 in 2023 were CP1 pupils in 2022. In 2022, the delivery of teaching inputs (decodable, decipherable and student’s workbook) arrived in the middle of the year. As a result, CP1 pupils were unable to complete the full CP1 curriculum. This delay in the completion of the program had an impact on the following year. These same students, then enrolled in CP2, were unable to complete the entire CP2 program before the assessment, which could be a factor explaining this lower result. 37. Progress achieved towards achievement of PDO indicator 4 (teachers implementing at least three of the new teaching practices developed for mathematics in the project, in beneficiary schools) exceeded expectations (expected target 80 percent; achieved target 94.1 percent). The focus of classroom practice training helped teachers apply what they learned in their classrooms immediately. This in turn allowed the consolidation of the practiced skills, as teachers observed how students responded to this more focused instruction. The new classroom practices for mathematics were: (i) manipulation; (ii) schematization; and (iii) questioning. The implementation of the classroom observation tool was intended to measure improved teaching practices. The classroom observation instrument was successfully implemented in Côte d’Ivoire throughout project duration. The implementation of this tool benefited teaching practices by: (i) building Page 18 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) national capacity on classroom observation, (ii) opening the dialogue with the government on the need to align the contents and demands of teacher training; (iii) providing reliable data on teaching practices and quality in the country; and (iv) generating global knowledge on classroom observation tools. Collected data were also used to inform the future National Early learning Strategy to be rolled out in 2023/2024 and future teacher trainings accordingly. 38. Progress achieved towards achievement of PDO indicator 5 (teachers implementing at least three of the new teaching practices developed for reading in the project, in beneficiary schools) exceeded expectations (expected target 80 percent; achieved target 91.7 percent). For PDO5, the focus of classroom practice training helped teachers apply what they learned in their classrooms immediately. The new classroom practices for reading and writing were: (i) individual students reading aloud while pointing; (ii) corrective feedback from teachers; and (iii) explanation by teacher of unknown vocabulary. 39. Eight out of ten IRIs were either achieved or exceeded. Regarding the number of girls receiving complementary school kits, the kits were initially reserved for girls in CP1, CP2 and CE1, but were ultimately distributed to all girls in the beneficiary schools to avoid the sense of inequity felt by the community. Regarding teachers, it was planned to train only the teachers in the classes concerned by the project (CP1, CP2 and CE1), but in the end the project trained all the teachers in all the beneficiary schools. The deworming target was missed due to a misunderstanding between the health department and the ministry of education regarding the targeted beneficiaries of the deworming program. The project was to provide deworming kits for project-supported schools. However, when the health teams arrived in the six project regions, they used the deworming kits for all schools in the area, not just those targeted by the project. As a result, some non-beneficiary schools received the deworming kits, while some of the project supported schools did not get the deworming kits because the health teams ran out of the kits purchased by the project. 40. The impact evaluation was published by the MENETFP thereby meeting the established target. During project implementation two evaluations were conducted: one in 2021 at mid-term of the project and another four years later, in 2023. Two groups were considered: (i) sites where the project was operating and (ii) sites without project intervention (control group). The impact of the pre-school interventions showed that children who attended community preschools supported by the project have, at the age of 6 years old, a level of knowledge in reading that is higher (+5 percentage points, or +36 percent) than the control group and in mathematics (+4 percentage points or +15 percent) compared to the control group. Additionally, findings from the evaluation found that the impact of primary school intervention had a positive impact on learning conditions. It found that the primary intervention had improved the reading and mathematics performance of children ages 6-8 years. In reading there was an increase of 5 percentage points, or +28 percent) and mathematics there was an increase of 7 percentage points, or +21 percent) compared to the control group. The analysis suggested that these improvements were based on improvement of teachers’ pedagogical performance. 41. In addition to the indicators, the bridging classes were able to bring 4,758 children (compared to the original target of 4,500) back into the formal system. The bridging system used facilitators, established an operating method to connect with the formal sector, and provided some financial support through provision of school kits. The bridging classes approach/model that was successful to the point that the government decided to continue this approach and included it in the framework of the Strengthening Primary Education System Project (P177800) with support from non-governmental organizations to assist with the program. Justification of Overall Efficacy Rating 42. Based on the analysis presented above, overall efficacy is rated Substantial. Page 19 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) C. EFFICIENCY Assessment of Efficiency and Rating 43. Economic analysis at appraisal. The project’s impact was analyzed according to short, medium- and long-term prospects. In the short term, the project was expected to: (i) benefit young Ivorian boys and girls who would have a better access to primary education; (ii) impact positively the retention rate; and (iii) the completion of primary education. In the medium term the project was expected to contribute to improvement of the learning achievements of the education system through strengthening of learning outcomes evaluation and remediation mechanism. Social and demographic equity in access was also expected. In the long term, the project was expected to impact on economic growth through the human development of the country. Considering the assumptions used the Net Present Value (NPV) was US$98.0 million; the Internal rate of Return (IRR) was 18.5 percent, and the Economic Rate of Return (ERR) was 12.5 percent. 44. The economic analysis conducted during the AF showed that the project is economically cost-effective, with a Net Present Value (NPV) of US$432.7 million (discounted at 10 percent) and an Economic Rate of Return (ERR) of 12.9 percent. The current economic analysis, like the one above mentioned, is based on a Cost-Benefit Analysis (CBA) approach, relying solely on pecuniary benefits of education, because several benefits are not measurable or straightforward to monetize (e.g., teachers career development thanks to the training they’ve received, personal growth, social integration, healthier lifestyle choices, increased civic engagement, enhanced parenting skills, improved quality of life, cultural enrichment, resilience and adaptability). 45. We calculated the NPV and Benefit-Cost Ratio (BCR) using the 6.25 percent Central Bank (BCEAO) discount rate in force in Côte d'Ivoire since 2009 (BCEAO, 2021), as well as the IRR. Since the AF CBA used the discount rate of 10 percent, we have also estimated the NPV, BCR and IRR using this rate. We make the key assumption that the impact induced by the project remains constant over time, and the average yearly future earning of the beneficiaries will be at least the average yearly income estimated from the EHCVM 2021. The discounted investment total cost (project cost, plus operating costs, plus opportunity costs) amounts to approximately US$62.75 million. Discounted benefits are estimated at US$216.58 million amassed over 42 years (from the legal age of 16 years for entering the workforce until the expected 58 years of life). The corresponding NPV of the operation is US$144.79 million, and the associated BCR is 3.5. The Internal Rate of Return (IRR) of this investment is 19.4 percent. Each of these results (positive NPV, BCR greater than 1 and IRR greater than discount rate) confirms the economic cost-effectiveness of the project, although some benefits, such as the non-pecuniary benefits of education, are not easily quantifiable and were not included in the analysis. 46. Operational efficiency. The project was planned for four years, became effective in July 2018 for an amount of US$23.35 million. Based on its successful implementation, 24 months later an additional US$27.35 million was added and 18 months extension of closing date was granted. The project closed without delays in December 2023 with 95.21 percent (or US$48,272,474.40) of available resources disbursed (the undisbursed amount is due to the PBC that could not be reached). Based on its initial design the project was supposed to disburse an average US$5.8 million per year; with the AF the project disbursed an average of US$9.2 million per year in 5.5 years, or 56 percent more per year than initially designed. Speed of implementation was high, and it is worth noting that the project reached all its objectives while disbursing only 95.21 percent of its financing. Page 20 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) 47. Readiness for implementation was adequate. The first year of implementation reached 20 percent disbursement which improved to 40 percent after 18 months well above the disbursement curve, confirming that the project was ready to be implemented at effectiveness. 48. The project implementation unit managed a previous Bank-financed project. It was experienced and its staff turnover was relatively low. The Bank staff turnover was also low with 2 task managers in 5.5 years. 49. Some objectives have been exceeded. The project managed to substantially exceed certain indicators with the same amount of money. By way of illustration, while it was planned to introduce tools up to CE1 (grade 3), the project managed to reach CE2 (grade 4). Similarly, the project built more classrooms than originally planned. A total of 4,758 children from the bridging classes were able to enter the formal system, compared with the original target of 4,500. 50. Based on the above, efficiency is rated Substantial. D. JUSTIFICATION OF OVERALL OUTCOME RATING 51. The overall project outcome performance is rated as Satisfactory. The project piloted a new approach to foundational learning, based on the latest neuroscience research, which the government would not have been able to introduce without this financing. This new approach of structured pedagogy was accompanied by an impact evaluation which measured the improvement in performance between a control group (who did not benefit from the project) and a group who benefited from the structured pedagogy approach. It was through this pilot, and the rigorous impact assessment that was carried out, that the country decided to scale up this approach, relying on a PforR financing instrument. Finally, the approach developed through the project has now become an integral part of the national strategy on foundational learning, and all partners wishing to support the government must align themselves with this strategy. Sub-Ratings Overall Outcome Rating Relevance Efficacy Efficiency High Substantial Substantial Satisfactory E. OTHER OUTCOMES AND IMPACTS Gender 52. Although not specifically mentioned as an objective in the PDO, the project had a focus on gender, as reflected by its monitoring of several indicators focused on gender. Under the project, 10,000 complementary school kits were distributed to girls from poor households to help ease the financial burden on their families and improve learning conditions for girls. Girls' enrollment is effective in beneficiary communities, with a proportion of 64 percent girls compared to the 50 percent initially envisaged. During implementation, MENA has set up a Directorate of Gender Equality and Equity (Direction de l’Egalité et de l’Equité du Genre). The project supported its workplan specifically in the areas of awareness campaign on girls' schooling and gender-based violence. Institutional Strengthening 53. The project helped strengthened the national core team of trainers specializing in integrated early childhood development (6 supervisors and 13 master trainers) as well as early learning (reading/writing and mathematics) in Page 21 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) primary (5 supervisors and 27 master trainers). This central team (in place during the project’s lifetime and still in place today) were involved in: (i) classrooms observations; (ii) tools development; (iii) teachers’ training; (iv) teachers’ support (coaching); and (v) oversee the overall process of improving teaching/learning practices at school level. The same core team is now involved in the scaled-up WB-supported Côte d'Ivoire Strengthening Primary Education System Operation (PRSEP—P177800). 54. Under the project, the evaluation capacity of the education system was strengthened. This was done through: (i) the consolidation of national level learning assessments both reading/writing and mathematics, with four census-based standardized student assessments carried out; and (ii) the use of data and information to improve outcomes. The project supported the DVSP in the design and implementation of EGRA and EGMA. An independent auditor (enrolled to confirm PBC achievement) has confirmed compliance with procedures and quality standards in the implementation and analysis of the results of these standardized national assessments. Poverty Reduction and Shared Prosperity 55. As explained in the PAD, the project targeted six regions where primary education enrollment and efficiency rates were the worst and poverty levels were the highest—117 communities were targeted. The AF continued supporting the same six regions with an additional 225 communities targeted. The project was designed to contribute to: (i) reduce extreme poverty by providing opportunities to poor children to benefit from quality education for higher education achievements; and (ii) boost shared prosperity by introducing direct financing to schools for sustainable education services in poor and lagging communities. Although no scientific studies have been carried out as part of this project, the satisfaction survey carried out among beneficiaries (parents in particular) suggests that preschool has enabled women to free up time for professional activity. Therefore, pre-school is not only a place of learning for children, but it also has a significant impact on the economic empowerment of women. By facilitating their participation in the labor market, it contributes to increasing their income and fostering economic and social progress. This helps create a virtuous circle in which investment in pre-school education translates into broader economic growth. Other Unintended Outcomes and Impacts 56. Scaling up, from regional pilot to nationwide implementation. The new early learning strategy, institutionalization of national standardized assessment and new tools and approaches designed and implemented under the project will now be expanded nationwide. 57. After pilots’ evaluation, the ministry—through the DPFC—has undertaken consultation to develop a National Early Learning Program (PNAPAS). The tools and approaches developed with project support for early reading/writing and mathematics have been officially validated by the scientific council of the Ministry of Education and Literacy. As such, they have been integrated into the National Early Learning Strategy and the resulting operational plan—PNAPAS. 58. The tools and approaches developed under the project and successfully implemented have now been adopted by MENA and scaled up into a new nationwide PRSEP-P177800 which started implementation in 2023. The scaled up activities include: (i) early learning package and standards for standardized national assessment of learning outcomes; (ii) preschool construction through community approach; (iii) new tools (textbooks and student books); and (iv) new teachers training methods (scripted pedagogy) and strengthened coaching of teachers. Implementation of PNAPAS also began in Page 22 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) the 2023/2024 school year and included a new strategy to monitor learning outcomes through national assessments using EGRA and EGMA that were piloted through the project. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 59. Project preparation began in 2017 when GPE announced an allocation of US23.35 million for Côte d’Ivoire. The government and Local Education Group agreed that the Bank would be the supervising entity for the grant. As previously mentioned, the project was the first education operation in Côte d’Ivoire that included a PBC approach. The overall project design used the funding model introduced by the GPE in 2014 which required a “variable part” which was 30 percent of the total grant amount. The objective of this funding modality was to use GPE funds as a catalyst for transformational policies that would have long lasting impact on the education sector. The variable part of the grant developed PBCs and “stretch indicators” based on the ESP which were national level indicators approved by the local education group and GPE which, as indicated, could not show direct attribution to the project. The stretch indicators were based on three domains: (i) equity; (ii) efficiency; and (iii) learning outcomes and used a set of corresponding indicators. The AF GPE grant of US$27.35 million followed the same design principles as the original project. The PBC designed required a verification protocol prior to disbursement. 60. The project’s design was highly relevant, had analytical underpinnings and was informed by lessons learned from other operations in the sub-sector in Africa. The overall TOC assumed when children are prepared for learning and schools have learning-focused inputs, as well as effective teaching and skilled management 18—children are more likely to achieve higher levels of educational attainment, thereby positively impacting learning poverty. The project was anchored in improving equity and quality through a series of activities and interventions related to increasing school readiness by supporting ECE 19 and supporting primary schools in the most disadvantaged rural areas in Côte d’Ivoire. To address these the design included: (i) piloting of ECD programs in communities and primary schools along with establishment of ECD standards; (ii) providing teacher training on the use of scripted lessons to improve early grade reading and mathematics; (iii) expanded opportunities and access for out-of-school children through the inclusion of the bridging program in the AF; (iv) promotion of community involvement in monitoring school activities through the provision of grants for establishing ECD programs and community classroom construction; (v) building of capacity within the system; (vi) and monitoring activities to assess the impact of project interventions. 61. Given the central and regional level of activities, the institutional arrangements were perfectly set up. A Steering Committee (SC) was responsible for overseeing the project which was chaired by the Director of Cabinet of MENA and composed of the Coordinator of the National Education Task Force, Directors of MENA, and representatives of selected ministries and of the LEPG. The SC responsibilities were to: (i) ensure consistency of project activities with sector policies and programs; (ii) validate and monitor progress of project activities; (iii) identify and resolve difficulties in project implementation; and (iv) take proactive measures to ensure effective project implementation. The head of 18The 2018 World Development Report (WDR). 19International research has shown that early childhood education programs can prepare children to develop the social and emotional skills they need to succeed and graduate high school. Other benefits of ECE can include earning higher grades and having a lifelong enthusiasm for reading and learning. The first 5 years of life, positive experiences and warm, responsive relationships stimulate children's development, creating millions of connections in their brains. In fact, children's brains develop connections faster in the first 5 years than at any other time in their lives. Page 23 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) the PIU attended as secretariat. An experienced existing PIU staffed with adequate officers oversaw day-to-day management of the project (coordination, fiduciary, safeguards and M&E). Directorates of MENA were responsible for technical implementation of the project. At the decentralized level, School Management Committees (COGES) were responsible for managing all grants for community educators, performance based and school construction activities. 62. Key risks and appropriate mitigation measures were identified during preparation for both the original project and the AF. The original project’s overall risk was rated Substantial due to challenges related to: (i) piloting the new PBC concept since this was the first education project to implement results-based financing; (ii) capacity constraints within the committees in the most disadvantage areas to implement the community based activities; (iii) implementation of the pilot ECD programs which would require involvement of various ministries; (iv) capacity constraints in fiduciary management; (v) capacity constraints related to collection, analysis, and utilization of data. The overall risk at the time of the AF project was Modest. This was largely due to the successful implementation of the original project activities and capacity development activities that supported overall implementation. Mitigation measures that were included in the original project design included on-going training and technical assistance support at the school/community level and MENA. The operations manual provided detailed implementation guidelines and the Bank would provide technical assistance and ongoing support for fiduciary requirements. B. KEY FACTORS DURING IMPLEMENTATION 63. Factors subject to Government and/or implementing entities control. The project enjoyed continued political support and full commitment from the technical units. World Bank implementation support missions met with the PIU during periodic visits, and report that the technical directors in charge of the different subcomponents took high interest in the project’s progress and provided their full support. There was relatively low turnover in MENA and PIU counterparts, allowing for continuity and building ownership and trust. Issues that impacted implementation were mostly exogenous, related to the currency fluctuation, changes in construction prices, and the effects of the COVID-19 pandemic, which impacted several activities in the project. Institutional arrangements worked well under the direction of the SC. The SC met 10 times during the project implementation The PIU was experienced in project management and the WB’s procedures since it had managed the previous Emergency Basic Education Support Project. It oversaw fiduciary (procurement and financial management), M&E and reporting and safeguards, it worked closely with the National Agency of Environment for safeguards issues. The speed of implementation of the project, sustained disbursements, and overall performance in terms of efficacy are evidence that the overall institutional arrangement worked well. 64. Factors subject to World Bank control. The project benefitted from the fact that there was a consistent Bank team that supported implementation, both local and international. This allowed the team to interact regularly with the PIU/MENA and other implementation units, which allowed for ongoing consultation on the implementation challenges. Additionally, the project’s mid-term review provided the opportunity to review project activities which showed very good project implementation progress and results which led to the GPE AF. The WB’s task team worked with the PIU to ensure the project was closed on time and in an orderly process. 65. Exogenous factors. The Covid-19 health crisis which disrupted the 2019-2020 school year (schools remained closed between March 16 and May 17, 2020). Political tensions linked to the 2020 elections also closed schools for the last semester of 2020 (October 19 and November 15, 2020). These exogenous factors delayed the production and distribution of educational tools for CP1 as well as new tools for CP2. However, the delays did not fundamentally threaten the achievement of the project objectives. In addition, the team at the direction of education and in-service Page 24 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) training (Direction de la Pédagogie et de la Formation Continue - DPFC) worked to develop distance learning courses (radio and video capsules) for virtual classes during school closing period. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 66. The links between the inputs, outputs, and outcomes and the PDO were sound (as shown in the TOC). The original PDO was clearly specified at appraisal and was not modified at the time of the 2020 AF. The M&E design was developed to monitor and report on progress toward meeting the targets of the PDO and intermediate indicators in the Results Framework (RF) through regular routine monitoring and reporting by the PIU. The overall M&E design was not modified at the time of the AF however, it did add a comparative analysis of the two preschool models. For the local stakeholders, the project included capacity development and training on monitoring school activities through participation in community meetings. Finally, the design included use of classroom observations in order to observe teaching practices, identify areas of improved and develop a feedback loop to support teacher improvement in teaching and pedagogical practices. M&E Implementation 67. PIU reporting of data for the RF indicators was done throughout the life of the project. The selected performance indicators were tracked regularly, and the RF was updated on a timely basis for supervision missions including the mid-term review—approximately 60 high quality monthly progress reports (Fiches Synthétiques de Suivi Évaluation) were disseminated within the government, to the WB, donor community, and the public highlighting progress of components (physical and financial). Moreover, data collected was added to the ministry’s broader M&E system which is a well-functioning information system and can generate useful education data. The impact assessment was conducted at the end of the project in the six project supported regions and it included a control group. The assessment included data collection on: (i) community-based preschools and preprimary interventions; (ii) improvements in pedagogical practices in primary schools (classroom observations); and (iii) learning outcomes through the use of EGRA and EGMA. Data collection was the responsibility of MENA’s department, and a consultant was recruited for data analysis and evaluation. The communities were also trained in monitoring school activities and the impact evaluation looked at community involvement in the preschools. M&E Utilization 68. Data was made available regularly for the results framework and was used to inform the analysis of the ICR The PIU use the data through “Fiches Synthétiques de Suivi Evaluation” 20 to monitor and evaluate project impact, inform MENA high management, directors and SC for decision making, and to communicate to shareholders the project’s overall performance. The use of Budgeted Annual Work Programs by directors and PIU to monitor day to day implementation favored appropriation of work plans and activities. At restructuring, all the M&E information (data and studies) was used 20The PIU shared on a monthly basis these monthly monitoring reports with the Wolrld Bank team and other relevant stakeholders. These simplified reports showed, key actions and achievements for the past month, actual disbursement and update of indicators when available. Page 25 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) to inform the AF and scale up activities. This included: (i) EGRA and EGMA assessment data from primary schools; (ii) teacher observations; (iii) use of the teacher observation information to establish a feedback loop for improvement of training materials and teacher pedagogical practices; and (iv) successful community construction of classrooms along with community involvement in setting up ECD facilities and monitoring. Finally, the lessons from the successful implementation of the project were used by the government as it has embarked on the WB-supported PforR operation (PRSEP: 2023-2028) that capitalized on the project’s successes. Justification of Overall Rating of Quality of M&E Based on the above, the rating of quality of M&E is Substantial. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 69. Environment and Social: During preparation, the project was classified as an environmental Category B – Partial Assessment and triggered three World Bank operational policies: OP/BP 4.01 (Environmental Assessment); OP/BP 4.11 (Physical Cultural Resources); and OP/BP 4.12 (Involuntary Resettlement). Based on these, an Environmental and Social Management Framework (ESMF) and a Resettlement Policy Framework (RPF) were prepared and publicly disclosed. 70. Safeguard compliance was overall Satisfactory. During implementation, project activities were implemented in full compliance with the national legislation and safeguards policies triggered by the project. The environmental and safeguards (E&S) screening process was systematic and carried out. An Environmental and Safeguards Management Plan (ESMP)was prepared for school/classroom construction and was monitored by the PIU safeguards specialists in close collaboration with those responsible for activities in the field i.e., inter-mediators, COGESs and technical implementation committees. E&S monitoring missions were also carried out in the field by environmental and social safeguards specialists. The project recorded 13 construction incidents/accidents and a detailed report was sent to the WB specifying how each of these incidents were adequately addressed and resolved. Additionally, the project put in place a preventive communications system for incidents. E&S quarterly reports were regularly produced. The E&S ratings were consistently Moderately Satisfactory or Satisfactory throughout project implementation as reported in the Implementation Status and Results Reports (ISRs). 71. Grievance Redress Mechanism (GRM). The project's GRM was approved by the WB’s task team in December 2020. The project implementation team prepared a GRM communications plan, and 72 local communications actors (15 women) were sensitized on the project’s GRM. Additionally, 753 CP1 directors/teachers received training as well as the facilitators of the bridging classes. There were 35 complaints registered and were resolved through the GRM. The project also developed and implemented an action plan, in collaboration with MENA, for gender-based violence and sexual exploitation and abuse (GBV/SEA) that included mapping of GBV service providers, creating a network of GBV focal points and sensitization on GBV. Overall, there were 3,912 stakeholder (29 percent women) who were sensitized. There was also an evaluation conducted of the capacity of the child protection committees in primary schools and capacity development activities conducted. 72. Fiduciary compliance - Financial management (FM). The financial management (FM) arrangements were adequate and remained unchanged throughout project implementation and FM performance was consistently rated Moderately Satisfactory and Satisfactory. Project implementation progressed according to the requirements outlined in the project’s Grant Agreement. The MENA and PIU was in compliance with financial reporting requirements as Page 26 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) follows: (i) unaudited interim financial reports (IFRs) were deemed satisfactory and submitted on time by the General Inspectorate of Finance that acted as the internal auditor; and (ii) external financial audits were successfully carried out and the reports were submitted on time and were of unqualified opinion (clean reports), underscoring the fact that the project financial resources had been properly managed and judiciously used for their intended purposes. During implementation, FM supervisions identified areas such as updating the procedures manual to include performance-based contracts after the AF and ensuring that audit report recommendations were completed which were all satisfactorily resolved with no negative impact. The final audit report is due on June 30, 2024. By the closing date of December 31, 2023, 95.2 percent of the project’s net financial resources of US$50.70 million had been disbursed. There was US$2.43 million undisbursed and returned to the WB. 73. Fiduciary compliance. Procurement: The WB’s procurement team conducted regular implementation support missions and found project performance to be Satisfactory throughout project implementation. The PRAMS reports consistently indicated that the procurement staff was well trained. The procurement staff routinely prepared, received approval and submitted realistic procurement plans that were aligned with the budget. Additionally, the PIU used the WB’s Systematic Tracking of Exchanges in Procurement (STEP) for managing procurement contracts. The PIU procurement staff were occasionally behind in archiving information and documents in STEP and contract management within the STEP system proved challenging for procurement staff. During implementation, WB’s procurement staff worked closely with the PIU to resolve any challenges that arose and provided capacity development particularly as it related to STEP. C. BANK PERFORMANCE Quality at Entry 74. The project preparation team ensured that the project design was closely aligned with the 2016–19 CPF, Côte d’Ivoire’s NDP 2016–20, GPE’s Strategic Plan 2016-2020 and, importantly, the Local Education Group endorsed ESP. The design included all the necessary ingredients to improve increased access to preschool and improved learning outcomes in primary schools in the project supported areas. The design team drew on national and international evidence that showcased ECD and more effective teaching as a requirement for improving learning outcomes. The preparation team, for the original project and subsequent AF project, identified the appropriate risks, incorporated design features to mitigate them, and included the relevant technical specialists to support project development. Preparation of the project was a consultative process with the government, donor community, GPE and the WB. The project’s design also included the GPE requirements for PBCs and went through the WB peer review process as well as three quality reviews by the GPE. However, the inclusion of the PBCs with national targets that had no connection with the project design which focused on six regions was a problematic design feature. This design feature proved to be challenging for implementation and highlights that targets at appraisal stage should be selected as “achievable” and “measurable” and within the scope of the project. Based on this and the above analysis of the relevance, efficacy and efficiency, quality at entry is rated Moderately Satisfactory. Quality of Supervision 75. Supervision missions were semi-annual, staffed with education specialists, fiduciary, and safeguards staff as well as consultants with needed technical expertise. The project team was actively engaged in supporting the government in its efforts to implement the project. Whenever implementation challenges arose, the WB’s task team worked with the government to find appropriate solutions that would not compromise the integrity of the design. The Page 27 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) supervision teams consistently reported on FM, procurement and safeguards progress during supervision missions and worked with the PIU and other implementation agencies, to build their capacity in these areas. As indicated above, safeguards compliance was monitored regularly. The missions also systematically documented project progress in aide memoires, back-to-office reports, and ISRs, all of which kept WB management informed of progress and provided the foundation for the ICR analysis. Key to the project’s achievements was the consistent supervision by the WB’s task team that included the needed technical expertise to support the various implementation units. Moreover, the WB’s willingness to respond to the government’s request for the AF based on the new GPE allocation for Côte d'Ivoire. As indicated, the WB team worked with MENA to prepare the AF and used the successful implementation of the original project to inform and scale-up of activities in the AF. It should also be noted that at the time of the AF, the WB’s task team asked the GPE to reconsider the use of national level targets for the PBCs. However, the GPE policy at that time remained the same as that of the initial preparation and were not able to alter the PBC criteria based on their policy. As indicated, GPE has subsequently modified their policy to correct this issue. The Bank team also led virtual missions during COVID-19 to ensure that implementation was on track and to ensure that the project was completed on time and as designed. The WB’s steadfast and proactive implementation support to the government ensured the successful achievement of the project’s envisaged outputs and outcomes and the institutionalization of many project activities. Based on this the quality of supervision is rated Satisfactory. Justification of Overall Rating of Bank Performance 76. Based on the above analysis, and the associated ratings of Moderately Satisfactory for Quality at Entry, Satisfactory for Quality of Supervision, Satisfactory for the project’s Overall Outcome Rating. Therefore, the Bank’s Overall Performance is rated Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 77. The risk to sustaining the development outcome is rated Modest. The project covered six regions where the rate of school attendance and school performance were the lowest in Côte d’Ivoire and demonstrated that improving learning outcomes was possible with sustained support and appropriate interventions. The sustainability of project interventions can been seen through: (i) provision of preschool and primary classrooms to increase access to early childhood education; (ii) training of preprimary teachers on the skills needed to teach young children through a play- based approach that encouraged the promotion of more holistic early childhood development; (iii) training of community leaders and parents in the monitoring of school programs in their communities; (iv) training of early grade teachers in early grade reading and mathematics skills and the use of related scripted lessons; and (iv) training of school principals and pedagogical advisors on the use of classroom observation tools that are now being used to help teachers improve their pedagogical practices. Importantly, based on the lessons learned through the project, the government has embarked on the implementation of a WB-supported PforR Operation (PRSEP: 2023-2028) in the amount of US$364.7 million for pre-primary and primary schools that capitalizes and scales up this project’s successes to the national level. Additionally, education represented approximately 12 percent of the government’s 2023 budget. Through the government’s continued strong support to education and project approaches are fully integrated into and maintained through PRSEP implementation, Côte d’Ivoire should be able to reach a new level of performance in education as outlined in the national ESP and the 2023-2027 Country Partnership Framework. V. LESSONS AND RECOMMENDATIONS 78. The main lessons from project preparation and implementation are as follows: Page 28 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) 79. Improving inputs to preprimary and primary schools is effective in increasing participation. Under the project, MENA began implementing the program to help increase enrollment in early childhood education and primary schools. This was done through the provision of classrooms in the project supported communities and support for community training on the importance of ECD and transitioning to primary school. The project provided complementary school kits to girls in Grades 1, 2 and 3, from poor households to ease the financial burden for their families. The project also provided adequate learning materials for teachers along with adequate textbooks for the students. 80. Lesson 2. Ensuring that teachers are equipped with the needed competencies for teaching foundational skills and materials are essential if children are to learn. The project supported training of teachers, principals and pedagogical advisors on the use of scripted lessons which were being piloted in the project. The training focused on foundational literacy with coaching teachers. The teacher materials included guides following explicit linguistic scope and sequences and skilled-based trajectory. Every student was also provided textbooks along with teaching and learning materials (decodable, decipherable with workbooks) that followed explicit linguistic scope and sequences. As indicated, this approach was successful in the project supported schools and is now being scale up under the Bank-supported PforR Operation (PRSEP—P177800—2023-2028). 81. Lesson 3. Non formal pathways to education can be successful if efforts are made to sustain and institutionalize the approach. As indicated, the bridging classes model was successful to the point that the government decided to continue the approach and scale it up within the framework of the PRSEP project (P177800). 82. Lesson 4. Performance-based conditions can be an important mechanism for establishing a culture of project and system accountability if formulated properly. As indicated, the project PBCs were formulated to monitor national level achievement rather than project specific achievement. While these indicators were useful to monitoring implementation of the ESP, they were not directly related to the project. Therefore, disbursements related to the PBCs could only be achieved if there was achievement outside the project scope. Moreover, there could be no direct attribution to the project for any PBC achievement. Again, GPE has subsequently modified its policy on use of national level indicators to project specific PBC indicators. 83. Lesson 5. Effective cooperation and mutual trust between staff of the Bank and the implementing ministry play a significant role in ensuring effective project implementation. The PIU team had been involved in implementation of previous projects and were actively involved in project preparation. The team was familiar with MENA staff and work processes and knowledgeable of World Bank policies and procedures. This cooperation and stakeholder ownership of the project played a significant role in ensuring project performance. In addition, the same Bank team that participated in the project appraisal continued to be involved throughout project implementation. These factors all contributed to the successful completion of the project. . Page 29 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increase Access to Preschool Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of preschools Number 0.00 90.00 342.00 349.00 meeting quality standards 01-Mar-2018 28-Jul-2022 31-Dec-2023 31-Dec-2023 Comments (achievements against targets): Target exceeded (102%). Objective/Outcome: Improve Learning Outcomes in Beneficiary Schools Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Pupils in second grade of Percentage 24.00 32.00 32.00 64.00 primary education meting minimum competency 01-Mar-2018 28-Jul-2022 31-Dec-2023 30-Sep-2023 Page 30 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) threshold for mathematics, in beneficiary schools in poor and lagging communities Of which girls, in second Percentage 47.00 50.00 50.00 51.00 grade of primary education meeting minimum 01-Mar-2018 28-Jul-2022 31-Dec-2023 30-Sep-2023 competency threshold for mathematics, in beneficiary schools in poor and lagging communities Comments (achievements against targets): Target exceeded (200%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Pupils in second grade of Percentage 6.00 25.00 25.00 21.00 primary education meeting minimum competency 01-Mar-2018 31-Dec-2023 31-Dec-2023 11-May-2023 threshold for reading, in beneficiary primary schools in poor and lagging communities Of which girls, in second Percentage 47.00 50.00 52.00 grade of primary education meeting minimum 01-Mar-2018 31-Dec-2023 30-Sep-2023 competency threshold for Page 31 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) reading, in beneficiary primary schools in poor and lagging communities Comments (achievements against targets): Progress from the baseline value observed, though falling short of the end-of-project target. Approximately 84% of the end-of-project target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teachers implementing at Percentage 0.00 80.00 80.00 94.10 least three of the new teaching practices developed 01-Mar-2018 28-Jul-2022 31-Dec-2023 30-Sep-2023 for mathematics in the project, in beneficiary schools Comments (achievements against targets): Target exceeded (117.62%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teachers implementing at Percentage 0.00 80.00 80.00 91.70 least three of the new teaching practices developed 01-Mar-2018 28-Jul-2022 31-Dec-2023 11-May-2023 for reading in the project, in Page 32 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) beneficiary schools Comments (achievements against targets): Target exceeded (114.62%). A.2 Intermediate Results Indicators Component: Improving Conditions for Improved Learning Outcomes Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of community Number 0.00 117.00 117.00 117.00 preschools built by the project 01-Mar-2018 31-Dec-2023 31-Dec-2023 30-Sep-2023 Comments (achievements against targets): Target achieved (100%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of preprimary Number 0.00 225.00 225.00 240.00 classrooms built in beneficiary communities by 30-Nov-2020 31-Dec-2023 31-Dec-2023 30-Sep-2023 Page 33 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) the project Comments (achievements against targets): Target exceeded (107%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Students benefiting from Number 0.00 345,000.00 345,000.00 356,986.00 direct interventions to enhance learning 30-Mar-2018 28-Jul-2022 31-Dec-2023 11-May-2023 Students benefiting from Number 0.00 197,835.00 197,835.00 228,948.00 direct interventions to enhance learning - Female Comments (achievements against targets): Target exceeded (103.5%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Students benefitting from Number 0.00 47,970.00 47,970.00 38,271.00 direct interventions to enhance learning in Early 30-Mar-2018 28-Jul-2023 31-Dec-2023 28-Sep-2023 Childhood Education (ECE) Page 34 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) of which girls Number 0.00 23,985.00 23,985.00 19,113.00 30-Mar-2018 28-Jul-2022 31-Dec-2023 30-Sep-2023 Comments (achievements against targets): Substantial progress observed from the baseline value, though falling short of the end-of-project target. Approximately 80% of the end-of-project target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teachers receiving at least Percentage 0.00 90.00 90.00 100.00 one supervision visit by inspectors or pedagogic 01-Mar-2018 28-Jul-2022 31-Dec-2023 11-May-2023 advisers in beneficiary schools, per quarter Comments (achievements against targets): Target exceeded (111.11%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Teachers recruited or trained Number 0.00 2,250.00 2,250.00 7,582.00 30-Mar-2018 28-Jul-2022 31-Dec-2023 30-Sep-2023 Page 35 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Teachers recruited or Number 0.00 450.00 450.00 1,245.00 trained - Female (RMS requirement) Comments (achievements against targets): Target exceeded (337%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Large-scale Number 0.00 3.00 3.00 3.00 primary/secondary learning assessments completed 30-Mar-2018 28-Jul-2023 31-Dec-2023 11-May-2023 Comments (achievements against targets): Target achieved (100%). Component: Strengthening Education Sector Management and Accountability Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Public schools reporting Percentage 0.00 90.00 90.00 100.00 school grants and student results through community 01-Mar-2018 28-Jul-2022 31-Dec-2023 30-Sep-2023 briefings meetings, at least Page 36 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) once a year, in beneficiary schools Comments (achievements against targets): Target achieved (111.11%). Component: Improving Teaching and Learning Environment for Beneficiary Schools Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of classrooms Number 0.00 675.00 675.00 730.00 rehabilitated/built by the project 01-Sep-2020 28-Jul-2022 31-Dec-2023 31-Dec-2023 Comments (achievements against targets): Target exceeded (108%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of girls receiving Number 0.00 4,500.00 4,500.00 10,508.00 complementary school kits 01-Mar-2018 31-Dec-2023 31-Dec-2023 11-May-2023 Comments (achievements against targets): Page 37 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Target exceeded (233.51%). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of beneficiaries Number 0.00 240,000.00 240,000.00 198,406.00 benefiting from Deworming 01-Mar-2018 28-Jul-2022 31-Dec-2023 30-Sep-2023 Of Which girls Number 0.00 120,000.00 120,000.00 101,180.00 01-Mar-2018 28-Jul-2022 31-Dec-2023 11-May-2023 Comments (achievements against targets): Substantial progress from the baseline value observed, though falling short of the end-of-project target. Approximately 83% of the end-of-project target achieved. Component: Project Implementation and Monitoring and Evaluation Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of policy notes Number 0.00 6.00 9.00 6.00 published by the MENETFP 01-Mar-2018 28-Jul-2022 31-Dec-2023 11-May-2023 Comments (achievements against targets): Page 38 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Target not achieved (66.66% of target reached). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Impact Evaluation report Yes/No No Yes Yes Yes published by the MENETFP 01-Mar-2018 28-Jul-2022 31-Dec-2023 11-May-2023 Comments (achievements against targets): Target achieved. Page 39 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 : Increase access to preschool in participating regions Outcome Indicators 1.Number of preschool meeting quality standards. 1. Number of community preschools built by the project Intermediate Results Indicators 2. Number of preprimary classrooms built in beneficiary communities by the project 1. Classrooms construction (preschool and preprimary) Key Outputs by Component 2.Educators recruitment and training (linked to the achievement of the Objective/Outcome 1) 3. Preschool tools development 4.Quality standards set up Objective/Outcome 2 : improve learning outcomes in beneficiary primary schools in participating regions 1.Pupils in second grade of primary education meeting minimum competency threshold for mathematics, in beneficiary schools in poor and lagging communities 2.Pupils in second grade of primary education meeting minimum competency threshold for reading, in beneficiary schools in poor and Outcome Indicators lagging communities. 3. Teachers implementing at least three of the new teaching practices developed for mathematics in the project, in beneficiary schools. 4. Teachers implementing at least three of the new teaching practices developed for reading, in beneficiary schools. 1.Teachers receiving at least one supervision visit by inspectors or pedagogic advisers in beneficiary schools, per quarter Intermediate Results Indicators 2.Teachers recruited or trained. 3.Large-scale primary/secondary learning assessments completed. 4.Number of classrooms rehabilitated/built by the project. Page 40 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) 5.Number of beneficiaries benefiting from Deworming 6.Number of girls receiving complementary school 1. Classrooms observations 2. Introduction of structured pedagogy and revised curriculum 3. Teacher’s training 4. Learning materials development (decodable, decipherable, teachers guide) Key Outputs by Component 5. Teachers’ supervision and coaching (linked to the achievement of the Objective/Outcome 2) 6. Students assessment (EGRA / EGMA) 7. School Scorecard (with communities’ involvement) 8. Classrooms construction / rehabilitation 9. Deworming 10. School kit for girls Page 41 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Patrick Philippe Ramanantoanina Task Team Leader(s) Maurice Adoni Procurement Specialist(s) Jean Charles Amon Kra Financial Management Specialist Laurianne Marine Anne Lise Le Lem Team Member Ndri Marie Ange Francisca Memel Team Member Aissatou Seck Team Member Catherine Audrey Balife Seya Team Member Frances Mary Beaton-Day Team Member Oumou Coulibaly Team Member Issa Thiam Team Member Abdoulaye Gadiere Environmental Specialist Fatoumata Diallo Team Member Amantchi Jean - Noel Gogoua Team Member Andre Francis Ndem Team Member Germaine M. Ethy Team Member Marie Roger Augustin Counsel Abdoul Wahabi Seini Social Specialist Celine Solange Jeanne Gavach Team Member Page 42 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Assiata Houedanou Soro Team Member Bintou Sogodogo Team Member Supervision/ICR Yves Jantzem, Ali Coulibaly Task Team Leader(s) Abdoulaye Keita, Antro Kenneth Sanvi Procurement Specialist(s) Atioumoutio Edith Kone Epse Zannou Tchoko Financial Management Specialist Kanga Grant Marcelle Armakah Team Member Adjoua Veronique Ouattara Social Specialist Edichi Brigitte Andoh Epse Mobongol Environmental Specialist Frances Mary Beaton-Day Team Member Oumou Coulibaly Team Member Sophie Jackie Danielle Cerbelle Discour Team Member Cedric Cubahiro Team Member Ezechiel Abouro Djallo Team Member Julie Nyamien Messoum Kouame Team Member Nikolai Alexei Sviedrys Wittich Procurement Team Germaine M. Ethy Team Member Aglaia Zafeirakou Team Member Mohammad Ilyas Butt Procurement Team Assiata Houedanou Soro Team Member Felly Akiiki Kaboyo Team Member Bernardo da Cruz Vasconcellos ICR Team/Quality Assurance Sandra Beemer ICR Team/Quality Assurance B. STAFF TIME AND COST Page 43 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY17 8.750 45,673.78 FY18 32.985 217,556.04 FY19 18.321 366,365.74 FY20 18.613 333,859.58 FY21 12.750 222,490.57 Total 91.42 1,185,945.71 Supervision/ICR FY19 0 0.00 FY20 3.125 31,416.39 FY21 11.886 110,718.95 FY22 39.069 318,544.42 FY23 43.949 196,198.87 FY24 15.978 70,767.54 Total 114.01 727,646.17 Page 44 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Improving Conditions for 23.53 24.25 103.1% Improved Learning Outcomes Strenghtening Sector Management and 3.38 4.39 129.8% Accountability Improving Teaching and Learning Environment for 16.64 11.75 70.6% Beneficiary Schools Project Implementation and 7.15 7.87 110.1% Monitoring and Evaluation Total 50.70 48.26 95.2% Page 45 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 4. EFFICIENCY ANALYSIS 1. To assess Côte d’Ivoire Education Services Delivery Enhancement Project’s efficiency in achieving its objectives, two types of analysis are conducted: (i) economic analysis and (ii) implementation efficiency analysis. Considering the findings from these analyses, the project is rated substantial on efficiency. This annex presents the economic analysis of the project. 2. Côte d’Ivoire Education Services Delivery Enhancement Project aimed to increase access to preschool and improve learning outcomes in beneficiary primary schools in participating regions. This objective is reached through support to at least 375 communities, by: (i) developing preprimary education; (ii) improving teaching and learning in reading and mathematics; (iii) enhancing citizen participation in school management; and (iv) improving teaching and learning environments in primary education. The project operates in six regions where primary education enrollment and efficiency rates were the lowest, and where poverty levels were the highest: Bagoué, Béré, Bounkani, Kabadougou, Poro, and Tchologo. 3. The economic analysis conducted during the AF showed that the project is economically cost-effective, with a NPV of US$432.7 million (discounted at 10 percent) and an ERR of 12.9 percent. The current economic analysis, like the one above mentioned, is based on a CBA approach, relying solely on pecuniary benefits of education, because several benefits are not measurable or straightforward to monetize (e.g. teachers career development thanks to the training they’ve received, beneficiaries personal growth, social integration, healthier lifestyle choices, increased civic engagement, enhanced parenting skills, improved quality of life, cultural enrichment, resilience and adaptability). The CBA is performed in a social perspective, and require the number of beneficiaries, the project’s benefits thanks to its impact, and its costs. Unlike the CBA carried out during the AF, where the gains were based on assumptions or the impact of similar projects taken from the literature, this analysis is based on the actual impacts of the various interventions implemented as part of this project 21. Beneficiaries 4. We start the analysis by presenting the number of the project’s beneficiaries. The project benefited various types of individuals, including preschool (i.e. community preschool and public pre-primary) and primary students, teachers, parents, etc. In this analysis, we focus on beneficiaries whose benefits can easily be monetized. Typically, we retain students who benefited from direct interventions to enhance learning in Early Childhood Education (ECE) and students who benefited from direct interventions to enhance learning in primary education. The number of these two types of beneficiaries are 38,271 and 290,549 respectively as shown in the results framework. This brings the total number of beneficiaries that is used in this analysis to 328,820. It should be noted that the total number of beneficiaries used for the AF CBA was 504,000, which combined the number of students benefiting from quality enhancement activities in targeted primary schools (360,000) and the number of students benefiting from the deworming activity (144,000). We believe that this figure was overestimated, since almost all the students benefiting from the deworming activities are already beneficiating from direct interventions to enhance learning. 21A rigorous impact evaluation of the various interventions using the RCT method was carried out. The impacts used in this analysis are taken from the final impact evaluation report. Page 46 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Benefits 5. Côte d'Ivoire has a very young population, with three-quarters (75.6 percent) under the age of 35 (INS, 2022). Unfortunately, its workforce is poorly educated, with 46.1 percent having no formal education and 18.3 percent having only primary education. Figure 1: Distribution of the workforce by level of education 50% 46.1% 40% 30% 18.3% 18.3% 20% 10.3% 10% 7.1% 0% No Primary Lower Secondary Upper Secondary Higher Source: Authors (using EHCVM 2021) 6. However, the income level of waged workers increases significantly with the level of education. Although those with primary education have, on average, almost the same wage level as those with no formal education (respectively 519 and 550 CFAF per hour worked), those with lower secondary education have, on average, 681 CFAF per hour, compared with 957 CFAF per hour for employees with upper secondary education. Employees with post- secondary education are the best paid, with an average of 2,181 CFAF per hour worked. Only workers with at least upper secondary education have, on average, a higher wage than the national average. Moreover, the share of poor people (i.e. those in the first two welfare quintiles) in each level of education of the workforce falls considerably as the level of education rises. It is 44.7 percent (22.8 percent of Q1 and 21.9 percent of Q2) among those with no formal education, while it is only 4.8 percent (1.1 percent of Q1 and 3.7 percent of Q2) among those with post-secondary education. Page 47 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Figure 2: Average wage (in CFAF) by level of education (left) and Distribution of the workforce by level of education and welfare quintiles (right) 300,000 3,000 Higher 200,000 2,000 Upper Secondary 100,000 1,000 Lower Secondary 0 0 Primary Secondary Secondary Higher No Cote d'Ivoire Primary Upper Lower No 0% 20% 40% 60% 80% 100% Average monthly wage (in CFAF) Average hourly wage (in CFAF) Q1 Q2 Q3 Q4 Q5 Source: Authors (using EHCVM 2021) 7. In addition, the Ivorian labor market indicates that investment in education generates significant positive returns that increase with the level of education. The coefficients estimated by the Mincer regression model for the country and for each level of education (Table 1) show that in Côte d’Ivoire, an additional year of education generates a 9.8 percent increase in wages. This rate is slightly lower than that of Sub-Saharan Africa, which stands at 10.5 percent (Psacharopoulos and Patrinos, 2018). The private rate of return to schooling, which is only 7.3 percent for individuals with some primary education, rises to 14.9 percent for those with lower secondary education, 16.6 percent for those with upper secondary education, and 23.1 percent for those with higher education. Disaggregated by gender, returns to education are higher for women compared to men (11.4 percent versus 9.2 percent). This two-percentage point gap is in line with that found by Psacharopoulos and Patrinos. This does not imply that earnings are higher for females, but only that education is a good investment for women and girls and should be a development priority. Table 1: Rate of return to schooling, by gender and level of education Men Women Total Linearized Linearized Linearized Rate of Rate of Rate of Standard Standard Standard Returns Returns Returns Error Error Error Country Côte d’Ivoire 0.092*** 0.006 0.114*** 0.007 0.098*** 0.006 Level of Primary 0.067** 0.028 0.074* 0.042 0.074*** 0.023 education Lower Secondary 0.177*** 0.053 0.078 0.071 0.149*** 0.048 achieved Upper Secondary 0.230*** 0.061 -0.056 0.228 0.166** 0.076 Higher education 0.233*** 0.028 0.213*** 0.067 0.231*** 0.028 *** p < .01, ** p < .05, * p < .1 Source: Authors (using EHCVM 2021) 8. Furthermore, education substantially increases the probability of waged employment (conditional on being employed) which brings the highest earnings. However, it seems to have no effect on the probability of being employed. The average probability of being a salaried employee, which is 0.320 in Côte d’Ivoire, ranges from 0.195 for employees with no education to 0.689 for those with post-secondary education. Page 48 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Table 2: Probability of being employed and probability of being a salaried employee by gender and level of education Men Women Total Probability Probability Probability Probability Probability Probability of being a of being a of being a of being of being of being salaried salaried salaried employed employed employed employee employee employee Country Côte d’Ivoire 0.830 0.395 0.487 0.215 0.641 0.320 Level of No education 0.879 0.249 0.516 0.136 0.648 0.195 education Primary 0.850 0.374 0.508 0.238 0.661 0.314 achieved Lower Secondary 0.739 0.527 0.368 0.360 0.576 0.476 Upper Secondary 0.748 0.635 0.358 0.488 0.607 0.598 Higher education 0.838 0.730 0.452 0.580 0.708 0.689 Source: Authors (using EHCVM 2021) 9. The fundamental question of this analysis is not whether investment in education generates positive returns, but rather whether the project implemented over the six years (2018-2023) was economically cost-effective. 10. Like in the literature (e.g., Angrist et al., 2020; GEEAP, 2023) the project’s impact evaluation shows that most of interventions implemented under this project have significant positive impacts on student schooling and learning. To determine the impact of several interventions that are implemented in combination, Angrist et al. recommend considering the maximum effect of these interventions, rather than simply summing them, and recognize that the combination of several interventions produces more impact than the interventions taken separately. Table 3 below lists the project’s key interventions and their impacts. Table 3: Impact of project’s interventions Impact on schooling Impact on learning (Percentage points increase in (Percentage points increase in test percentage of children who score) attended primary school) Intervention IDELA EGRA EGMA Age 6 Age 7 Age 8 Community Preschool 3 5.3 4 9 15 17 Pre-primary 0 0 0 0 9 0 Primary n/a 5 7 7 12 0 Community Preschool + Primary n/a 2.8 5.4 0 10 0 Pre-primary + Primary n/a 0 0 0 11 0 Notes: 0 percentage point increase means that the impact is not significant at 5 percent level. n/a stands for not applicable. Source: Bastien Michel, 2023. 11. We focus on the impacts on schooling which are higher than those on learning and choose the conservative impact of 7 percent point increase in the percentage of children who attended primary school thanks to the project. Knowing that an Ivorian from the project’s regions who ever attended primary school has, on average, 5.7 years of schooling, we calculated the future and forgone incomes of beneficiaries using average earning for people with some primary education, as estimated from the 2021 West African Economic and Monetary Union (WAEMU) Household Survey (Enquête Harmonisée sur les Conditions de Vie des Ménages de 2021, EHCVM 2021). Table 5 below shows other Page 49 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) key indicators used for this CBA. So as not to take up too many pages, we have chosen not to present here the average monthly wages by level of education and specific age, which have also been used to compute benefits. Table 4: Other keys indicators used to compute benefits Indicator Value Source WDI, latest value for Official exchange rate (LCU per US$, period average) 623.8 2022 Average number of years of schooling for those who ever attended school in Authors’ estimates 5.7 project’s regions using EHCVM 2021 Legal minimum age in labor market 16.0 WDI, latest value for Life expectancy at birth, total (years) 58.6 2021 Authors’ estimates Earning deflator (to adjust EHCVM earnings to consider non-wage workers) 0.78 using ENE 2019 Authors’ estimates Probability of being employed (6-11) 0.032 using EHCVM 2021 Source: Authors Costs 12. The actual cost of the project is US$48.26 million. This cost does not include the preparation cost (US$1.19 million) and the project supervision cost (US$0.71 million). In addition, we calculated the opportunity costs for the project beneficiaries, i.e. the income they would have earned if they had not gone through the project. These costs include education expenses for the time (i.e. years) spent at school because of the project, and the forgone earnings. The annual opportunity cost for a beneficiary is estimated at US$95. This cost is much lower than that estimated in the AF CBA which was US$1,223. We believe that this figure was overestimated because basic education in Côte d’Ivoire is free of charge, and an average household can’t afford more than US$1,200 a year to education a child. Table 5: Opportunity costs (in USD) for an additional year the project’s beneficiaries will spend at school Education Forgone Total Costs earnings For an additional year the project’s beneficiaries will spend at school 54.48 40.23 94.72 Source: Authors (using EHCVM 2021 and ENE 2019) 13. Although schooling is free in Côte d'Ivoire, households nevertheless spend money on tuition and registration fees, subscriptions, school supplies, other school materials, uniforms, school canteen and catering, transport, and other school expenses (tutor, support, etc.). Page 50 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Table 6: Education costs in Côte d’Ivoire in 2021 Percentage of students Average annual total with no spending education cost (in CFAF) Preschool 0.5% 62,294 Primary 0.8% 34,357 Lower Secondary 0.3% 73,842 Upper Secondary 0.1% 124,327 Higher education 0.0% 315,001 Source: Authors (using EHCVM 2021) NPV, BCR and IRR 14. To conclude on the cost-effectiveness of the project, we calculated the NPV and BCR using the 6.25 percent Central Bank (BCEAO) discount rate in force in Côte d'Ivoire since 2009 (BCEAO, 2021), as well as the IRR. Since the AF CBA used the discount rate of 10 percent, we have also estimated the NPV, BCR and IRR using this rate. We make the key assumption that the impact induced by the project remains constant over time, and the average yearly future earning of the beneficiaries will be at least the average yearly income estimated from the EHCVM 2021. The discounted investment total cost (project cost, plus operating costs, plus opportunity costs) amounts to approximately US$62.75 million. Discounted benefits are estimated at US$216.58 million amassed over 42 years (from the legal age of 16 years for entering the workforce until the expected 58 years of life). The corresponding NPV of the operation is US$144.79 million, and the associated BCR is 3.5. The Internal Rate of Return (IRR) of this investment is 19.4 percent. Each of these results (positive NPV, BCR greater than 1 and IRR greater than discount rate) confirms the economic cost-effectiveness of the project, although some benefits, such as the non-pecuniary benefits of education (e.g. teachers career development thanks to the training they’ve received, personal growth, social integration, healthier lifestyle choices, increased civic engagement, enhanced parenting skills, improved quality of life, cultural enrichment, resilience and adaptability) are not easily quantifiable, and were not included in the analysis. Table 7: CBA key parameters and estimates Discount rate 6.25% 10.00% 7 points increase in the 7 points increase in the Project impact percentage of children who percentage of children who attended primary school attended primary school Discounted total costs (present value of US$62.75 million US$62.75 million costs) Discounted total benefits (present value of US$216.58 million US$133.56 million benefits) NPV US$144.79 million US$64.37 million BCR 3.5 2.1 IRR 19.4% 19.4% Source: Authors 15. Based on the analysis presented above, efficiency under the project is rated Substantial. Page 51 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS In an email in June 2024, the Government confirmed its agreement to the findings of the ICR and had no additional comments. The Government provided their final completion report to the World Bank which has been filed in WBDocs for information and reference. Page 52 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 6. COMPARISON OF ORIGINAL AND AF COMPONENT ACTIVITIES Components/Components Interventions in the Initial Project (IP+AF) Interventions added as part of the AF (IP) Component 1: Improving conditions for better learning outcomes Sub-Component 1.1: Pre- Construction and equipping of 117 community Construction and equipping of 225 pre-primary School Development preschools and training of 220 educators and 220 classrooms. assistants. Sub-Component 1.2: Classroom observation, in-service training in 150 In-service training for teachers in 225 additional schools. Improving the Teaching and primary schools. Learning of Reading, Writing and Mathematics Development and distribution of teaching materials in Development and distribution of teaching materials for reading/writing and mathematics (decodable, students, teachers, and pedagogical supervisors in 225 decipherable, scripted lessons, workbooks for additional schools. Distance learning and training. students, pedagogical guide for teachers) for teachers and the pedagogical supervision chain of the 150 Development of bridging classes for 4,500 children schools. Distance learning and training. outside the system. Component 2. Strengthening management and accountability in the education sector Sub-component 2.1: Citizen Training of communities, publication of dashboards, Training of communities, publication of dashboards, participation in school information meetings. in 150 communities information meetings. Community in 225 additional management communities Sub-component 2.2: Results- Not foreseen (additional funding does not cover this Based School Funding Transfer of Grants to 150 Schools pilot activity. The approach and target of the original project remain unchanged). Sub-component 2.3: Capacity- Technical assistance, seminars, studies, missions, and One-year extension of support to MENA and the Inter- building for public policies purchase of office equipment. Ministerial Working Group Component 3: Improvement of the learning environment in beneficiary schools Improvement of infrastructure in beneficiary schools. Construction/rehabilitation of 144 primary Rehabilitation and construction of 675 classrooms to classrooms, 71 latrines, 12 wells, furniture for 17 replace/renovate classrooms built of poor quality and to schools. ensure classrooms’ completeness, sex segregated Supporting girls' education. Distribution of 10,000 latrines, including the equipment of classroom furniture, school kits. and, if necessary, water supply. Page 53 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Components/Components Interventions in the Initial Project (IP+AF) Interventions added as part of the AF (IP) Deworming (school health intervention) to 150 community preschools and 117 primary schools. Component 4. Project implementation and evaluation Sub-component 4.1: Operating Costs Operating costs for an additional 17 months, purchase of Implementation additional equipment, equipment service and maintenance contracts and maintenance of accounting and financial management software (not financed in the initial project). Data collection and analysis for impact evaluation, Impact evaluation including a comparative analysis Sub-component 4.2: recruitment of a firm to implement impact evaluation between the two preschool models (community and Evaluation and contribute to the analysis of results. pre-primary). 2023 Financial Audits Page 54 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 7. ADDITIONAL FINANCING PBC VARIABLE PART MATRIX Reference Performance-based results Performance-Based value Condi�ons (PBCs) 2020 2021 2022 (2017) PBC 1 Objec�ve: Stretch Indicator, Equity Increase the provision of high-quality preschool 1: services to improve the par�cipa�on of children from 22 23.3 23.8 24.2 Children from rural areas the poorest families and regions as percentage of children enrolled in preschool Amount allocated out of total funding: 2,800,000 1,748,460 1,001,540 50,000 Cumula�ve: Cumula�ve: 2,750,000 2,800,000 Indicator verifica�on process: The DSPS database will be the main source used for calcula�ng the target. Disbursement rule: For the verifica�on procedures. The variable component is paid on a pro-rated basis reflec�ng progress. For the first year, the formula follows: Allocated amount * (value atained – reference value) / (target – reference value). For subsequent years, the formula is: Allocated amount * (value atained – previous- year target) / (final target – previous-year target). PBC 2 Reference 2020 2021 2022 Stretch Indicator, value Efficiency 2.1 (2016) Weekly hours taught by Objec�ve: Increase the teaching hours of lower teachers in lower secondary teachers from 15.6 to 18.1 hours 15.6 17.6 18.1 NA secondary educa�on Amount allocated out of total funding: 0 0 0 Performance-Based Reference Performance-based results Condi�ons (PBCs) value (2017) 2020 2021 2022 Indicator verifica�on process: The DSPS database will be the main source used for calcula�ng the target. Page 55 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) Disbursement rule: For the verifica�on procedures. The variable component is paid on a pro-rated basis reflec�ng progress. For the first year, the formula follows: Amount allocated * (value atained – reference value) / (target – reference value). For subsequent years, the formula is: Allocated amount * (value atained – previous- year target) / (final target – previous-year target). PBC 3 Reference 2020 2021 2022 Stretch indicator, value Efficiency 2.2: Comple�on rate / share Objec�ve: The comple�on rate increases while of primary educa�on allocated expenditures for primary educa�on remain current expenditures in stable 1.5 1.7 1.8 1.9 educa�on. Amount allocated out of total funding: 2,600,000 1,053,690 50,000 1,496,310 Cumula�ve: Cumula�ve: 2,550,000 2,600,000 Indicator verifica�on process: The DSPS database and the Finance Act will be the main sources used for calcula�ng the target Disbursement rule: For the verifica�on procedures. The variable component is paid on a pro-rated basis reflec�ng progress. For the first year, the formula follows: Amount allocated * (value atained – reference value) / (target – reference value). For subsequent years, the formula is: Allocated amount * (value atained – previous- year target) / (final target – previous-year target). PBC 4 Reference 2020 2021 2022 Stretch Indicator learning value outcomes 3.1: (2016) Percentage of grade 3 Objec�ve: Improving learning and teaching condi�ons students mee�ng and strengthening educa�onal ac�vi�es and support minimum competency offered to teachers with a view to improving teaching 19% 27% 31% NA threshold for prac�ces in mathema�cs. mathema�cs Amount allocated out of total funding: 0 0 0 Indicator verifica�on process: The na�onal assessment of learning carried out by the DVSP will be the main source of verifica�on. Disbursement rule: For the verifica�on procedures. The variable component is paid on a prorated basis reflec�ng progress. For the first year the formula follows: Amount allocated * (value atained – reference value) / (target – reference value). For subsequent years, the formula is as follows: Amount allocated * (value atained – previousyear target) / (final target – previous-year target). Page 56 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) PBC 5 Reference 2020 2021 2022 Stretch Indicator learning value outcomes 3.2: (2016) Percentage of grade 3 Objec�ve: Improving learning and teaching condi�ons students mee�ng the and strengthening educa�onal ac�vi�es and support minimum competency 23% 31% 34% NA offered to teachers with a view to improving teaching threshold for reading prac�ces in reading. Amount allocated out of total funding: 2,018,640 631,380 1,387,260 Cumula�ve: 2,018,640 Performance-Based Reference Performance-based results Condi�ons (PBCs) value (2017) 2020 2021 2022 Indicator verifica�on process: The na�onal assessment of learning carried out by the DVSP will be the main source of verifica�on. Disbursement rule: For the verifica�on procedures. The variable component is paid on a prorated basis reflec�ng progress. For the first year, the amount allocated is (value atained – reference value) / (target – reference value). For subsequent years, the amount allocated is (value atained – previous-year target) / (final target – previous year target). Reference 2020 2021 2022 value PBC 6 Objec�ve: Improve learning and teaching condi�ons. N/A 5 760 6 507 Stretch Indicator learning outcomes 3.3: Amount allocated out of total funding: 981,360 520,560 Number of teachers 0 460,800 Cumula�ve: recruited 981,360 Verifica�on process: The DSPS database will be the main source used for calcula�ng the target. Disbursement rule: For the verifica�on procedures. The MENETFP sends the data to the World Bank, and the LEPG validates it. US$80 is allocated per teacher recruited. Total funding allocated to the variable part US$ 8,400,000 (original and AF) Page 57 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 8. MODIFICATIONS MADE TO INTERMEDIATE RESULTS INDICATORS Indicator Baseline Original Target AF Revised Comments (2018) (July 2022 Target (December 2023) Component 1: Improving conditions for improved learning outcomes 1. Community preschools built by the project 0 117 117 End target date revised (Number) No change to new closing date of December 2023. New AF 2020 Indicator 0 225 This indicator was added 2. Preprimary classrooms built in beneficiary to monitor the number communities by the project (number) of preprimary classrooms built in beneficiary communities under the project 3. Students benefitting from direct 0 144,000 345,000 Target scaled up and interventions to enhance learning (number) (No target was (170,000) target date revised. AF (number female) (Corporate Indicator) established for moved from PDO level females) indicator to intermediate indicator New AF 2020 Indicator 0 47,970 New corporate indicated 4. Students benefitting from direct (23,985) added at the time of the interventions to enhance learning in Early AF. Childhood Education (ECE) (number) (number female) (Corporate ECE indicator) Removed in 2020 0 10.320 Indicator replace by the Number of children enrolled in community new corporate indicator preschools built under the project (number “number of students girls) benefiting from direct interventions to enhance learning in ECE” 5. Teachers receiving at least one supervision 0 90% 90% End target date revised visit by inspectors or pedagogic advisers in No change to new closing date of beneficiary schools, per quarter (Percentage) December 2023. Corporate Indicator 0 1,350 2,250 AF moved from PDO 6. Teacher recruited or trained (number) (675) (995) level indicator to (female) (Definition includes teachers from intermediate indicator. Page 58 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) project supported community and Targets increased and preprimary schools and primary schools.) end target date revised to new December 2023 closing date. 2020 AF Removed indicator 0 900 Indicator was removed Number of teachers trained in beneficiary because it was similar to schools, based on classroom observations’ the existing corporate results indicator “number of teacher recruited or trained” Strengthening Education Sector Management 7. Public schools reporting school grants and 0 90% 90% The end target date was student results through community briefings No change revised to December meetings, at least once a year, in beneficiary 2023 to align with the schools (Percentage) project closing date. Improving Teaching and Learning Environment for Beneficiary Schools New AF 2020 Indicator 0 675 Indicator added to 8. Primary classrooms rehabilitated/built by monitor the number of the project (number) primary classrooms built by project. 9. Number of girls receiving complementary 0 2.500 4.500 Target increased due to school kits (Number) scale-up activities. End target date revised to December 2023 to align new closing date. 10. Number of beneficiaries benefiting from 0 240.000 No Change End target date revised Deworming (Number) (number female) (120,000) 240.000 to December 2023 to (120,000) align with project closing date Project Implementation and Monitoring and Evaluation 11. Policy notes published by the MENETFP 0 8 9 The target value has (number) been revised as one additional policy note will be published by the Ministry under the AF. The end target date revised to December Page 59 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) 2023, to align with new project closing date. 12. Impact Evaluation report published by the No Yes Yes End target date revised MENETFP (Yes/No) (no change) to December 2023 to align with project closing date Page 60 of 61 The World Bank Cote d'Ivoire: Education Service Delivery Enhancement Project (P163218) ANNEX 9. SUPPORTING DOCUMENTS Project Appraisal Document, World Bank, 2018. Project Paper, World Bank, 2020. Aides-memoire and Implementation Status Reports 2018–2023. APAP Completion Report, Ministère De l’Education Nationale Et De l’Alphabétisation, 2023. Michel, B.; Evaluation Finale du Projet de Developpement des Competences pour la Croissance (PRODEC), 2023 Page 61 of 61