Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) Report Number: ICRR0023930 1. Project Data Project ID Project Name P152910 CG -Education Sector Support Project Country Practice Area(Lead) Congo, Republic of Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-58060 31-Jul-2023 27,837,752.28 Bank Approval Date Closing Date (Actual) 28-Apr-2016 30-Jun-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 30,000,000.00 0.00 Revised Commitment 30,000,000.00 0.00 Actual 27,964,822.56 0.00 Prepared by Reviewed by ICR Review Coordinator Group Livia Maria Angelica Judyth L. Twigg Susan Ann Caceres IEGHC (Unit 2) Benavides Matarazzo 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective (PDO) as stated in the Financing Agreement and in the Project Appraisal Document (PAD) was "to improve education outcomes of primary and lower secondary school children and to increase the effectiveness of select management systems." Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) This Review concurs with the ICR (p. 16) that the project included three objectives: (a) improve education outcomes of primary school children, (b) improve education outcomes of lower secondary school children, and (c) increase the effectiveness of select management systems. The project was restructured twice, in August 2019 and June 2022. Neither the PDO nor the outcome indicators were modified, but targets were changed to ensure greater realism with respect to the country context at the time of restructuring. The ratings are the same against both the original and revised outcome targets, as shown in the ICR. Therefore, this Review will not carry out a split assessment. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components The project design included three components: Component 1: Providing quality education for all (appraisal: US$35 million, of which International Development Association [IDA] US$14 million; restructuring: US$11.5 million, fully financed by IDA). This component aimed to improve the quality of education by focusing on providing teachers and students with an appropriate teaching and learning environment. The five subcomponents under this component were: Subcomponent 1.1: Curriculum revision and implementation (appraisal: US$5 million, of which US$5 million IDA; restructuring: US$10 million, fully financed by IDA). Activities were to include the revision of primary and lower secondary school curricula, with a focus on French, mathematics, and science, followed by the printing and distribution of curriculum materials and training of relevant stakeholders. This subcomponent would also support the strengthening of the National Pedagogical Research and Action Institute, responsible for curriculum development and procurement of learning materials. During the August 2019 restructuring, this component was combined with subcomponent 1.2. Subcomponent 1.2: Provision of learning materials (appraisal: US$9 million, of which no IDA funding). Two rounds of procurement of textbooks were planned under this component, the first of which would be reprinting of existing textbooks to ensure a 1:1 coverage of French and mathematics for all primary school students, and a 1:1 coverage in French, mathematics, and science for all public lower secondary students. The second round of textbook procurement would be carried out for new textbooks that would be aligned with the new curricula. Other materials were expected to be distributed to support learning, such as books and laboratory kits, as well as scripted lessons in French and mathematics to all teachers in public primary schools. This subcomponent was combined with the previous one during the August 2019 restructuring, and the production of textbooks was replaced by the production of activity books and workbooks in primary and lower secondary levels. Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) Subcomponent 1.3: Improved assessment (appraisal: US$5 million, fully financed by IDA; restructuring: US$1.5 million, fully financed by IDA). The focus of this subcomponent would be on launching a national assessment of French and mathematics (at primary and lower secondary levels) to be conducted at least twice during the life of the project. In addition, the subcomponent would support improving the national examination system as well as measuring learning outcomes. This subcomponent was combined with subcomponent 1.4 during the 2019 restructuring. Subcomponent 1.4: Expand use of remediation in schools and classrooms (appraisal: US$4 million, fully financed by IDA). This subcomponent would support remediation activities, building on the results from learning assessments and research on existing teaching practices. Activities were also planned to develop a comprehensive policy framework for inclusive education targeted to the needs of vulnerable populations, and the development of local accountability mechanisms. This subcomponent was combined with the previous one during the August 2019 restructuring. Subcomponent 1.5: Improvements to school environments (appraisal: US$12 million with no IDA funding). The project was to include support for construction of classrooms and rehabilitation of infrastructure with a focus on primary schools to ensure universal primary enrollment. This subcomponent was cancelled during the August 2019 restructuring. Component 2: Improving scope, quality, and management of human resources (appraisal: US$18 million, of which IDA US$3 million; restructuring: US$9 million, fully financed by IDA). The two subcomponents under this component were: Subcomponent 2.1: Personnel recruitment, deployment, remuneration, advancement, and exit (appraisal: US$1 million, of which no IDA; restructuring US$3 million fully financed by IDA). The project would support the creation of a digital personnel database to identify personnel and to track them with respect to actual positions, careers, professional development, deployment, advancement, and retirement. In addition, this subcomponent would support the development of objective standards for recruitment into training institutions, and support efforts to track teacher mobility in order to make recommendations with respect to teacher deployment policies. Subcomponent 2.2: Training and professional development (appraisal: US$17 million, of which IDA US$2 million; restructuring: US$6 million, fully financed by IDA). The project would support training for all key education personnel, building on training needs assessments carried out as part of the previous project, but with a vision towards a long-term system reform. Component 3: Improving system performance (appraisal: US$10 million, fully financed by IDA; restructuring: US$9.5 million, fully financed by IDA). The four subcomponents under this component were: Subcomponent 3.1: Improved monitoring and evaluation (appraisal: US$1.5 million, fully financed by IDA; restructuring: US$4.5 million, fully financed by IDA). The project aimed to support the creation of an education management information system (EMIS) that would allow the collection, analysis, and use of data for informed decision making. This subcomponent was combined with subcomponent 3.2 during the August 2019 restructuring. Subcomponent 3.2: Institutional strengthening and capacity building (appraisal: US$2.5 million, fully financed by IDA). Based on a functional analysis of the key institutions in the sector, the project expected to support strengthening of the capacities of key functions at the central and regional levels. Activities would focus on creating an evaluation culture and evidence-based decision making, fostering greater Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) transparency and accountability. This subcomponent was combined with the previous one during the August 2019 restructuring. Subcomponent 3.3: Project management (appraisal: US$5 million, fully financed by IDA; restructuring: US$4.5 million, fully financed by IDA). This subcomponent aimed to finance the project coordination unit (PCU), responsible for the day-to-day management of the project. Subcomponent 3.4: Additional strategic studies (appraisal: US$1 million, fully financed by IDA; restructuring: US$0.5 million, fully financed by IDA). This subcomponent would finance additional studies and technical assistance as needed in support of education reforms and improved governance, quality, and quality assurance. The original project design also included an allocation of US$3 million for price and physical contingencies. As part of the August 2019 restructuring, these funds were reallocated among the above subcomponents. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates The original project cost was US$70 million, of which US$40 million would be financed by the government and US$30 million would be financed through an IDA single currency credit (EUR 27.6 million). At appraisal, there were already signs that the government was facing significant fiscal constraints. This was confirmed during the early implementation phase. As a result, counterpart funds were cancelled during a first restructuring process, and the total project cost was reduced to the credit amount. There were exchange rate gains of US$0.32 million, and actual disbursements were US$27.96 million, that is, 92 percent of the final credit amount of US$30.32 million. A Project Preparation Advance (PPA) of US$3 million was approved and partly used to carry out preparatory activities including studies, study tours, and the development of the Project Implementation Manual. The project was approved on April 28, 2016, became effective on August 4, 2017, had a mid-term review on May 27, 2019, and closed on June 30, 2023. The project was restructured twice, as follows.  Based on the results of the mid-term review, the project was restructured on August 2, 2019, at which stage US$4.42 million had been disbursed. The restructuring addressed the lack of counterpart funding by cancelling the government's full allocation and recalibrating the design to the new project amount. Specifically, the restructuring encompassed: (a) the cancellation of activities related to rehabilitation and construction of classrooms, and of a Competitive Fund for Action Research in Education; (b) simplification of the project subcomponents by merging subcomponents 1.1 and 1.2, 1.3 and 1.4, and 3.1 and 3.2; (c) reallocation of funds among disbursement categories, including the remainder of the unused funds from the PPA and the unallocated funds category; (d) revision of the results framework; and (e) advancement of the closing date from July 31, 2023 to June 30, 2022.  The second restructuring was carried out June 27, 2022, when US$14.58 million had been disbursed. It responded to delays in implementation that had been further exacerbated by the COVID-19 pandemic. The restructuring included: (a) provision of workbooks instead of textbooks to children to ensure learning continuity during the school closures during the pandemic; (b) extension of the project from June 30, 2022, to June 30, 2023; (c) reallocation of funds across components Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) and disbursement categories to reflect updated costs and the extension; and (d) revision of the results framework to adjust targets to the extension period. 3. Relevance of Objectives Rationale At the time of appraisal, the Republic of Congo had seen improvements in the education sector but was still facing major challenges. Gross enrollment rates had grown from 90 percent in 2005 to 104 percent in 2011. Primary completion rates had increased from 74 to 80 percent over the same period, although, at the same time, repetition rates increased from 18 percent in 2010 to 23 percent in 2012. Lower secondary enrollment also grew from 59 to 72 percent between 2005 and 2013, with near gender parity (parity index 0.96). Dropout rates were high, however, at 17.6 percent, while the number of available slots in the upper secondary schools was insufficient to accommodate the increasing enrollment and completion rates in primary schools. Infrastructure was deficient, with an insufficient number of classrooms and limited access to latrines. Textbook availability was very limited, with one reading textbook per five students, and one mathematics textbook per ten students. Almost 39 percent of primary school-age children were enrolled in private schools, and four percent in subsidized écoles conventionnées. Learning outcomes were not readily available, but a nationwide study carried out in 2012-13 of 2,000 students showed that the average score in French was 33 percent, with a range of 21 to 61 percent, depending on the region. The average score for mathematics was 26 percent at the national level, with a range of 18 to 40 percent. Boys outperformed girls both in French (35 versus 31 percent) and mathematics (27 versus 24 percent). On the other hand, there was little difference between rural and urban students. Private schools performed better than public schools and écoles convetionnées. While these data contributed to the overall assessment of education in Congo, there were sampling issues with this study, making the results only indicative. There were system-wide challenges affecting the capacity of the education sector to provide quality education for students. Students had classes for an estimated 23.5 weeks per year, while the standard was 29 school weeks. This was attributed to a high number of examinations as well as reduced instructional hours due to double shifts in schools. The curricula had not been updated in the last 15 years. Human resources management was the responsibility of multiple institutions and, as a result, personnel information on teachers was not readily available. Estimates indicated that there were between 4,700 and 6,900 civil service teachers at the primary level, while there were between 2,500 and 2,900 volunteer teachers (bénévoles) for this same education level. Overall, the sector was not managing adequately key functions such as strategic planning, data systems, finance and budgeting, human resource management, construction, evaluation, communication, and accountability mechanisms. The project's objectives were aligned with the government's Education Sector Strategy for 2015 to 2024, which had been recently completed at the time of appraisal. The strategy focused on: “(a) providing quality education for all; (b) responding to the need for quality human resources in an emerging economy; and (c) creating a well-performing education system” (PAD, p. 2). At appraisal, project objectives were relevant to the Bank's Country Partnership Strategy (CPS) for FY 2013-2016 (Report No. 71713-CG), specifically Pillar 2: Vulnerability and Resilience. This pillar specifically Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) addressed “Deepening Reforms in the Education Sector,” including improving quality of education through teacher training and provision of textbooks, as well as systems reforms to achieve greater efficiency and equity in the sector. The objectives were also aligned with the Country Partnership Framework for FY20-24 (Report No. 126962-CG), specifically with Focus Area 2: Build Human Capital and Enhance Resilience for Social Inclusion and Sustainable Growth. This Focus Area included a specific objective on “Improved Quality and Equity in the Education System,” which addresses needed system-wide reforms to ensure better access and quality of education, particularly for girls and vulnerable populations. The objectives were appropriately pitched, given the Bank’s previous experience in the country and sector. The previous Support to Basic Education Project (P084317, 2005-2013, US$35 million) had aimed to improve efficiency in the allocation and management of resources, improve the quality of the education services offered, and reduce inequities in the provision of these services. The predecessor project was implemented in a post-conflict situation, where the country faced significant challenges in institution building and sustainability. Its objectives had aimed to contribute to the improvement of basic education by increasing primary education completion rates and reducing primary level repetition rates. However, the objectives were not clearly defined. "Education outcomes" would seem to imply access and improved education quality considering the supported activities, though the outcome indicators used to measure achievement of the first and second objectives (completion, transition, and repetition rates) focused on education access and efficiency rather than education quality. The PAD (p. 11) stated repeatedly that the project intended to "improve the quality of education," providing the definition of "improved education outcomes" used by this Review. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Improve education outcomes of primary school children Rationale The theory of change for this objective stated that improved education outcomes at the primary level would be achieved through (a) the revision of the French and mathematics curricula for all primary grades, (b) training of primary level teachers and inspectors on the implementation of the revised curriculum, (c) the preparation and distribution of textbooks in French and mathematics, and (d) the implementation of remediation activities in the classroom. Outputs Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) The French and mathematics curricula for the primary grades were revised, piloted, and validated. Accompanying materials, including teachers’ guides for all primary grades, were developed. The number of additional qualified primary teachers resulting from project interventions was 17,828 teachers, exceeding the end target of 4,000 teachers. The target number was a rough estimate due to the lack of verifiable information on the number of teachers in the system at the time of appraisal. 646,094 primary students benefited from workbooks and activity books (as reported by the project team). Originally, the project aimed to distribute a textbook in French and mathematics to at least 90 percent of students. This target was reduced to 54 percent of primary students as part of the first restructuring process (August 2019). Subsequently, during the restructuring of June 2022, the indicator on the distribution of textbooks was eliminated because, in the context of the COVID-19 pandemic, the government requested that the Bank instead support the production of workbooks and booklets as part of the learning continuity effort in the country. 884,722 students at the primary and lower secondary levels benefited from direct interventions to enhance learning, exceeding the target of 659,883 students. Of the total number of students benefited, 495,597 were female students, that is, 56 percent of the total, exceeding the target of 329,942 female students. Between 2016 and 2023, 578 classrooms were constructed, above the target of 250 classrooms. Construction was carried out independently from the project, as the corresponding sub- component was cancelled in 2019, and was instead financed by the Global Partnership for Education, with the United National Education, Scientific, and Cultural Organization (UNESCO) as the grant agent and the United Nations Children's Fund (UNICEF) as the implementing agency, as well as from private donations. Outcomes The primary completion rate at closure was 93 percent as compared to a baseline of 73 percent, exceeding both the original and formally revised targets of 75 and 74 percent, respectively. The female primary completion rate at closure was 91 percent, as compared to a baseline of 73 percent, also above the original and revised targets of 75 and 74 percent, respectively. The transition to lower secondary at closure was 93 percent as compared to a baseline of 67 percent and above the original and formally revised targets of 72 and 71 percent, respectively. The corresponding rate for female students at closure was 93 percent, as compared to a baseline of 67 percent, exceeding the original and formally revised targets of 72 and 71 percent, respectively. The primary repetition rate at closure was 14 percent as compared to a baseline of 21 percent, exceeding the target of a reduction to 17 and 18 percent for the original and formally revised targets, respectively. There was improvement in students achieving at least minimum proficiency level in math (33.6 to 35.1 percent) and French (7.7 to 11.7 percent) at the end of primary schooling. This information was based on participation in the PASEC regional large-scale assessment in 2019 and a national large-scale assessment carried out at the end of the project period June 2023, as reported by the Bank team on June 28, 2024. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) Although the outcome targets were met the ICR lacked information to show interventions to improve quality (i.e. curriculum reform and teacher training) were having a positive effect and learning gains in math and French were modest. Achievement of this objective is therefore rated Modest. Rating Modest OBJECTIVE 2 Objective Improve education outcomes of lower secondary school children Rationale This objective aimed to improve education outcomes for lower secondary school children through the revision of the curriculum with a focus on French, mathematics, and science. Subsequently, school materials, including textbooks, would be produced, with corresponding training of relevant stakeholders at the school level, including teachers and inspectors. Outputs The curricula were revised for the four grades in lower secondary (6éme, 5éme, 4éme, 3éme) for reading, science, and math, together with accompanying materials and teacher training programs. 1,146 lower secondary teachers were trained on the new teacher guides for reading, science, and math (as reported by the project team). 238,628 lower secondary public school students received workbooks and activity books in French, science, and mathematics, as compared to a target of 50,051 students. It should be noted that, as part of the 2022 restructuring, the provision of textbooks was replaced by the provision of activity books. In terms of percentage, 45 percent of lower secondary public school students received activity books in science and mathematics, as compared to a target of 20 percent. Outcomes The transition rate to upper secondary decreased from the baseline of 67 percent to 56 percent at closure, below the original and revised targets of 72 and 71 percent, respectively. In the case of female students, the final outcome was 57 percent, below the baseline of 67 percent and of the original and revised targets of 72 and 71 percent, respectively. The ICR (p. 20) attributed the failure to achieve this outcome to two factors. First, there was an increase in students completing lower secondary education, thus increasing the denominator. Second, the availability of upper secondary spaces was limited, thus restricting the number of students who could progress to this level. Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) There was an increase in children passing the end-of-lower-secondary examination from 46.35 percent in 2016 to 60.5 percent in 2023 (reported by Bank team on June 28, 2024). Achievement of this objective is rated Substantial, with the caveat as more children complete lower secondary the availability of upper secondary spaces is limited. Rating Substantial OBJECTIVE 3 Objective Increase the effectiveness of select management systems Rationale This objective aimed to increase the effectiveness of two management systems, learning assessment and human resource management. The increase in the effectiveness of the learning assessment system would be achieved through the establishment and implementation of a first national assessment for both primary and lower secondary levels as well as the publication of annual statistics. Improved effectiveness of the human resources management system would be achieved through the introduction of a biometric system that would allow for the registration of volunteer teachers (bénévoles), trained teachers who had not yet been recruited by the government (finalists), and civil servant teachers. Outputs A national learning assessment instrument was created and implemented. In addition, two large-scale primary/secondary learning assessments were completed, the 2015 Programme for the Analysis of Education Systems (PASEC) and the 2023 PASEC-type national assessment for grades 2 and 6, exceeding the target of one assessment. The PASEC-type assessment was carried out at the national level, while the PASEC is a regional assessment sponsored by the Conference of the Ministers of Education of French-Speaking Countries. The percentage of volunteer teachers (bénévoles) registered with biometric data was 90 percent, below the target of 95 percent. In terms of the number of registered bénévoles, 9,468 were registered, above the target of 2,500 bénévoles. Originally, biometric registration was to be carried out on a pilot basis in three departments. Subsequently, the system was implemented at the national level in all 12 departments in the country. Despite the larger scope, the 95 percent target was kept. The number of finalists registered was 1,783, below the target of 10,000 finalists. The number of civil servant teachers registered was 21,176, exceeding the target of 14,000 civil servant teachers. Annual statistics reports were published for the school years 2017-18, 2018-19, and 2019-20. Subsequently, these yearbooks were not produced due to constraints linked to the COVID-19 pandemic. Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) Annual consultations with indigenous groups (in collaboration with associations for indigenous peoples) were not carried out as planned. Outcomes A national assessment system was established for learning assessment at the primary level. The utility of the learning assessment was deemed to be 3, above the formally revised target of 1 and meeting the original target of 3. Utility was measured as follows: “(i) the assessment data were fully analyzed and reported to education policymakers, and (ii) the results were reported for all required subgroups (gender, urban/rural, and geographic region)” (ICR, p. 20). The utility of the assessment system was measured on a scale of 1-4. The criteria were: (i) assessment data analyzed, and results reported to education policy makers; (ii) results reported for at least one student subgroup (gender, urban/rural, geographical region); and (iii) assessment exercise repeated at least once every five years for the same subject area and grades. If all criteria were deemed as yes, the utility value would be 4; if two were yes, the value would be 3; if one criterion was yes, the value was 2; if none were present, the value would be 1. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The first objective, to improve education outcomes of primary school children outcome targets related to transition, repetition, and completion were met, but the ICR lacked information to show interventions to improve quality (i.e. curriculum reform and teacher training) were having a positive effect and learning gains in math and French were modest; therefore, the objective was rated modest. The second, improve education outcomes of lower secondary school children is rated substantial due to the increase in children passing the end-of-lower-secondary examination, but with a caveat as the system is constrained with available space in upper secondary, thus, transition rate decreased. The third objective, to increase the effectiveness of select management systems, was achieved but with some shortfalls in reaching output targets, and is rated Substantial. The aggregate achievement is indicative of a Substantial rating for overall efficacy, but with one objective rated modest. Overall Efficacy Rating Substantial 5. Efficiency At appraisal, a cost-benefit analysis was carried out for the project. The analysis assumed that students would earn a premium wage by completing primary or lower secondary education as compared to those who would not Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) complete the corresponding education cycle. The wage differential would be due to increased productivity resulting from the project's intervention. The net present value (NPV) and the internal rate of return (IRR) were estimated for three scenarios, taking into account the constrained fiscal situation of the government at the time of preparation. The low-case scenario considered that only IDA financing would be available (US$30 million). The medium-case scenario included a US$40 million government counterpart contribution, as per the original project design, for a total project cost of US$70 million. Under the high-case scenario, the government would finance US$60 million, resulting in a total project cost of US$90 million. The IRRs for these scenarios were 13.1, 34.9, and 23.3 percent, respectively. The NPV was 67.5, 218.6, and 257.1 million dollars, respectively. This analysis assumed a discount rate of the annual benefits of 8 percent. It also assumed that graduating students would earn a premium across a 40-year work lifecycle after graduating from school. In addition, the analysis indicated that there would be economic benefits to students' families because they would not need to purchase school materials and, in rural areas, pay for the salaries of bénévole teachers, estimating that 6,250 households would benefit from savings of around 50 dollars per year. The ICR also included a cost-benefit analysis, updating the one carried out at appraisal under the low-case scenario, as only IDA funding was provided for the project. The base scenario assumed a 5 percent increase in earnings with an 8 percent discount rate. The IRR for this scenario was 22 percent with an NPV of US$59 million. A second scenario assumed a 15 percent discount rate. In this case, the IRR was 26 percent, and the NPV was US$82 million. With respect to implementation efficiency, the ICR noted some shortcomings that reduced overall project efficiency. The project was approved on April 28, 2016, but became effective only on August 3, 2017, that is, 16 months after approval. There were significant delays in the revision of the curriculum, leading to a two-year delay in the production of teaching and learning materials and teacher training. There were also delays in the implementation of the financial audits’ recommendations for improvements. Finally, the project faced implementation challenges in procurement, particularly towards the end of the project, when staffing was insufficient and procurement processes were delayed. On balance, however, these shortcomings were only moderate, and combined with the positive economic analysis, resulted in an efficiency rating of Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  13.10  Not Applicable 100.00 ICR Estimate  22.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) 6. Outcome Relevance of objectives is rated Substantial, as the objectives were fully consistent with the country context, the national education strategy, and Bank strategy, but the objectives to "improve education outcomes" were not clearly defined. Efficacy is rated Substantial based on rating of two objectives and one of the objectives did not have sufficient evidence with respect to improvement of education quality and was rated modest. Efficiency is rated substantial as returns were significant, but with some challenges during the life of the project that reduced implementation efficiency. These ratings indicate that there were moderate shortcomings in the project's preparation, implementation, and achievement, consistent with an overall outcome rating of Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The project contributed to the strengthening of institutional capacity through the development of a new curriculum for primary and lower secondary, the creation of a national assessment system, and the implementation of a biometric registration system for teachers. In addition, as reported by the task team, the bénévole teachers are currently being paid by the government thanks to their registration in the biometric system. Thus, the main risks to development outcome are linked to continuity in policy support and availability of resources to sustain achievements under this project. For example, teacher training would still be necessary to ensure full implementation of the new curriculum, and school materials would need to be produced on a continuous basis, especially in the case of activity books, which cannot be easily reused by subsequent cohorts of students. This risk could be mitigated through the follow-on project, Transforming the Education Sector for Better Outcomes and Results (TRESOR, P179410), currently under preparation. This proposed project aims to continue supporting access to quality basic education and strengthening of education sector management systems. 8. Assessment of Bank Performance a. Quality-at-Entry The project was aligned with the government's education sector strategy for 2015 to 2024 as well as with Bank strategy. The focus on access to education was appropriate, given the challenges that the country was facing at the time. Also, the project design addressed two critical systems to be supported, implementation of a student performance assessment system and development of biometric registration of teachers and civil servants. Project design also included the creation and implementation of an EMIS, given the lack of adequate data management system in the sector. Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) Project design included a large contribution of counterpart financing (US$40 million), at a time when the country was already facing a significant reduction in its revenues (PAD, p. 1). The main source of revenue was the export of oil, accounting for 90 percent of exports and 75 percent of government revenues. These revenues had been declining at a rate of 6.2 percent per year. In this context, the expectation that the government could provide counterpart funding was too optimistic, as was confirmed during the implementation process, and this risk was evident at the time of project preparation. The project had to be restructured to reflect the cancellation of counterpart funds with the consequent changes to project activities and reduction in the number of subcomponents and targets in the results framework. The results framework was not adequate. In the case of objectives 1 and 2, indicators were not designed to measure achievement of outcomes as defined in the PAD. Furthermore, intermediate indicators could have been disaggregated by education level, i.e. primary and lower secondary, to ensure that these could be assessed separately (for example, for the number of teachers trained). Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The World Bank team carried out regular implementation support missions twice a year in person and, during the COVID-19 lockdown, virtually. Missions included both technical and fiduciary support. The Bank also provided technical support through frequent meetings and hands-on training on fiduciary aspects of project implementation. The team carried out two restructurings to reflect the changing context in the country, such as the lack of counterpart funding and the need to provide learning continuity during the COVID-19 lockdown. The team also effectively simplified project design by merging several sub- components. On the other hand, the team could have addressed the shortcomings in the original design of the results framework. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The PAD (p. 24) recognized the challenges faced in the country with respect to the availability of data to establish a solid baseline and estimate projections. For example, the number of children who would benefit from the project was not clear, as there was no reliable source of information on this demographic. The data on the number of teachers (both civil servants and bénévoles) was unknown. Consequently, project Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) design included several activities to improve M&E systems so as to support sectoral decision making as well as monitoring progress under the project. The project aimed to support the creation and functioning of a reliable and credible EMIS to facilitate informed decision making. In addition, a statistical yearbook was to be produced. At the project level, the objectives were not clearly defined; it was necessary to consult the PAD (p. 11) to determine that "improved education outcomes" meant both "improvements in education quality” and improvement in access. The indicators in the results framework did not reflect the results chain leading to achievement of the objectives defined as such, lacking specific output, intermediate or outcome indicators on improved quality at the primary and lower secondary levels, as well as curriculum reform and teacher training for lower secondary. b. M&E Implementation At the sector level, the project supported the development of a standard instrument to collect statistical data, allowing for the production of statistical yearbooks for 2017-18, 2018-19, and 2019-20. Subsequently, these yearbooks were not produced due to constraints arising from the COVID-19 pandemic. In addition, the project supported implementation of the civil servant biometric registration system, implementation of a Service Delivery Indicator survey, and creation of the national assessment system. Still, the ICR (p. 28) indicated that capacity was not sufficient to implement a sustained M&E system in the sector, and further progress is expected under the proposed TRESOR project. The new teacher biometric system provided data on existing teachers under all three categories. At the project level, M&E was under the responsibility of the PCU. Initially, the PCU's data reporting was limited, but over time it improved. The Bank provided additional support in order to obtain the data and elaborate the end-of-project numbers for the PDO indicators, compensating for the shortcomings in project M&E. However, no action was taken to add indicators to measure achievement of the first and second objectives. c. M&E Utilization Although the project included efforts to produce reliable data for decision making, challenges continued in the collection, processing, maintenance, dissemination, and use of data (TRESOR PID Report No PIDA0042). In addition, no budget line has been allocated for future student learning assessments. Despite these limitations, data were sufficiently available for the purposes of restructuring of the project and to inform the ICR. M&E Quality Rating Modest 10. Other Issues a. Safeguards Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) The project was rated as environmental assessment Category B, partial assessment, at the time of appraisal. The main trigger for category B was related to the potential adverse environmental and social impacts associated with construction and rehabilitation of schools. The specific safeguard policies triggered were OP/BP 4.01 Environmental Assessment, OP/BP 4.11 Physical Cultural Resources, OP/BP 4.10 Indigenous Peoples, and OP/BP 4.12 Involuntary Resettlement. As part of the preparation process, an Environmental and Social Management Framework and a Resettlement Policy Framework were prepared. As part of the 2019 restructuring, the construction and rehabilitation of classrooms were removed from the project description. Nonetheless, the Bank continued monitoring environmental and social safeguards. Ratings were Moderately Satisfactory or Satisfactory during the implementation period, closing as Moderately Satisfactory. Risk ratings were downgraded from Moderate at appraisal to Low at project closing. The project included a Grievance Redress Mechanism (GRM). Implementation of the GRM started relatively late and was operational only in January 2022. A total of 517 complaints were received, processed, and resolved. Complaints were mainly related to the non-receipt of new materials and implementation of the biometric registration of teachers. b. Fiduciary Compliance Financial management (FM) of the project faced challenges throughout the implementation period. Shortcomings were mainly related to budget forecasting and internal controls, including ineligible expenditures. The Bank’s FM specialist provided assistance prior to project closing in order to reduce the amount of ineligible expenditures. Audits were carried out in accordance with the Financing Agreement, and opinions were unqualified. Interim financial reports were deemed acceptable. Financial statements were prepared by the PCU on an annual basis. Ratings were Moderately Satisfactory throughout project implementation, but closed as Moderately Unsatisfactory. The Bank team reported on July 30, 2024 the final audit report confirmed the project complied with financial management. “There are no outstanding Financial Management or Procurement issues related to the Project at this stage.” “In our opinion, during the period under review, the mission did not note any non-compliance with the terms of the credit agreement and the World Bank's instructions.” “The amount deemed ineligible was reduced after receipts and other proof was provided by the Project Coordination Unit, albeit there still remained at the end some ineligible expenditures (FCFA 300 797 078,40 or USD 497,185) that were ultimately reimbursed to the World Bank by the Government.” In the case of procurement, the PCU regularly used the Systematic Tracking of Exchanges in Procurement (STEP) system, and procurement plans were submitted to the Bank as required. While procurement staffing was adequate throughout project implementation, towards the end of the project, the procurement specialist left the PCU and was not replaced. As a result, the STEP platform was not updated, and there were some problematic contracts that required Bank support to ensure that they were closed adequately. Procurement was rated as Satisfactory or Moderately Satisfactory during the implementation period, but closed as Moderately Unsatisfactory. Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) c. Unintended impacts (Positive or Negative) None reported. d. Other N/A 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Shortcomings in measuring and Outcome Satisfactory Moderately Satisfactory attaining progress toward the achievement of Objective one. Overall Bank Performance Bank Performance Satisfactory Moderately Satisfactory aligns with the outcome rating. The results framework did not include measures of improvements of education Quality of M&E Substantial Modest quality as defined in the PAD and supported activities, at either the primary or lower secondary levels. The analysis focused on the assessment of whether indicator Quality of ICR --- Modest targets were reached, rather than more broadly on achievement of the objectives. 12. Lessons The ICR (pp. 31-32) provided insightful lessons, some of which are restated below. Teacher buy-in on curricular reforms is essential to ensure successful implementation. Teacher training under the project demonstrated that the new curriculum would require a shorter time to prepare classes and would allow for greater student participation in class. Teachers welcomed these changes, particularly because of the greater student participation and interaction. The ICR recognized that the process was still in its initial stages, and that further training was planned under the new TRESOR project by expanding on content knowledge and pedagogical practices. Continuous capacity assessment is necessary to ensure satisfactory implementation. At appraisal, implementation capacity risk was correctly assessed as Substantial due to the country’s historically weak implementation capacity and persistent challenges in the use of country Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) systems. Fiduciary missions were carried out systematically; however, towards the end of the project implementation period, both procurement and financial management ratings were downgraded to Moderately Unsatisfactory because of deficient financial management and procurement processes detected as a result of in-depth reviews. Good quality data is essential for project preparation, monitoring, and evaluation. The PAD rated the technical design risk of the project as Substantial mainly because of the lack of reliable data on the school-age population as well as on teachers working in the system either as civil servants or bénévoles. Consequently, the project included activities that intended to create an EMIS and a personnel management system, thus addressing both these challenges. At closing, some progress had been achieved on both counts, but further work was needed to fully implement both systems. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was well written and concise, but it had some limitations in terms of the assessment of outcomes. The ICR provided a good description of project implementation process, including an in-depth analysis of the two restructuring processes and how they modified the planned activities and the results framework. The ICR constructed a theory of change that was based on the original results framework in the PAD, but it was organized around the components rather than around the objectives. The analysis focused on an assessment of whether indicator targets were reached rather than more broadly on the achievement of objectives. As a result additional reports and data not contained in the ICR were reviewed by IEG to finalize the validation. The relevance section included a section on relevance of design, which would have been more appropriately addressed in the sections on Efficacy and Quality at Entry. Given that information systems were deemed to be critical at the time of preparation, the ICR could have provided more information on the development of the EMIS. Finally, the ICR did not provide straightforward statements on compliance with the Bank's safeguard and fiduciary policies. The Bank team subsequently provided information from the final audit to clarify compliance. a. Quality of ICR Rating Modest Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG -Education Sector Support Project (P152910) Page 18 of 18