Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) Report Number: ICRR0023388 1. Project Data Project ID Project Name P133333 SN Quality and Equityof Basic Education Country Practice Area(Lead) Senegal Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-52750,IDA-62520,TF-15232,TF- 31-Jan-2018 125,691,393.20 15285,TF-15311,TF-B5169 Bank Approval Date Closing Date (Actual) 13-Jun-2013 30-Jun-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 20,000,000.00 56,356,258.40 Revised Commitment 133,206,314.74 54,336,057.62 Actual 125,691,393.20 52,979,849.58 Prepared by Reviewed by ICR Review Coordinator Group Susan Ann Caceres Judyth L. Twigg Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective was to improve learning outcomes for early grades, increase access to the science and mathematics tracks of secondary schools, and improve equity in access to basic education. These objectives were stated consistently in the lending agreement (p. 5) and Project Appraisal Document (PAD) (p. iv). Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) Based on IEG/OPCS guidelines, no split rating is required. At a 2018 restructuring and Additional Financing (AF), some outcome targets were revised, but most were revised upward. In the one case where an outcome target was revised downward, both the original and revised targets were exceeded. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 09-May-2018 c. Will a split evaluation be undertaken? No d. Components Initially, there were three components with several sub-components; at a 2021 AF, a fourth component was introduced: Component 1: Improving Quality of Basic Education (estimated costs US$56.2 million; 2018 AF US$48 million; actual costs US$86.68 million). This component aimed to improve the quality of learning at schools through four subcomponents: Sub-component 1: Performance-based financing education grants. This sub-component was to support performance-based contracts between all 14 regional education authorities and the 56 educational districts. With AF, this activity was expanded to 16 regional education authorities. Sub-component 1.2: School Quality and Management Improvement. This sub-component was to support two main activities: (i) local communities' participation and oversight; and (ii) funds to support the implementation of quality improvement agreements between districts and schools. With AF, a literacy pilot for members of school management communities was added. Sub-component 1.3: Implementation of a national program to develop teaching and learning of science and math at the middle school level. This sub-component was to distribute 33,000 teaching guides, implement the revised middle school curriculum, and train middle school teachers to use the teaching guides and low-cost and effective information and communication technology. It was also to support renovation of the eight existing Science and Technology Blocks and construction and equipment of 20 additional Blocks at middle schools. With AF, two specialized high schools focusing on math and science, in Kaolack and Sedhiou, were to be developed. Criteria for selecting these two regions were low rates of access and completion, high poverty rates, and proximity to other regions. The AF also scaled up math and science training for teachers at both primary and secondary levels, and implemented activities to improve enrollment of girls in science and mathematics tracks. Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) Sub-component 1.4. Improving pre-service teacher training. This sub-component was to improve the qualification of teachers entering into the education system at the primary and middle school levels via construction and equipment of four new regional training centers in Diourbel, Kaolack, Kaffrine, and Louga, and hire consultants to design a new training program aligned with the curriculum. With the AF, additional activities focused on in-service training, headmaster training, and use of classroom observation tools. Component 2: Strengthening Equity in Access to Education (estimated US$15.3 million; 2018 AF US$8 million; actual US$25.84 million). This component aimed to reach out-of-school children with quality education focusing on less advantaged areas. It also focused on replacing all schools that were under shelters. Sub-component 2.1. Construction and rehabilitation of schools in lagging or underserved areas. This sub- component was to build schools in lagging areas, rehabilitate/replace shelters used as classrooms via public-private partnerships, and construct 200 new primary schools. It was expected that 144 schools in Kaffrine, Diourbel, Tambacounda, Louga, and Matam, all regions with a relatively high proportion of out-of- school children, would create spaces for an additional 30,000 children. Sub-component 2.2. Grants to Daaras. This sub-component was to support 100 Daaras with resources for rehabilitation of infrastructure, additional French teachers' salaries, learning materials, and incentives for the headmasters. The supported schools were to use a specific Ministry of Education (MOE) curriculum in reading and math and to measure results. With the AF, an additional 400 Daaras were to receive performance-based grants. Sub-component 2.3: With the AF, a catch-up program for young out-of-school children (ages 8-14) was to be implemented to allow these children to integrate into the school system in the five regions with the lowest enrollment rates. Component 3: Project Management and Capacity Building (estimated US$8.5 million; actual US$7.45 million). This component was to support: (a) project coordination, supervision, financial management, communication and outreach, and implementation of the safeguard documents; (b) communication, monitoring and evaluation of the project, including through the reinforcement of MOE's information technology system; and (c) a program of activities to build the capacity of education inspectorates in the areas of leadership, human resources management, financial management, planning, monitoring and evaluation (M&E), and communication. Sub-component 3.1: Project Implementation and Management. This subcomponent was to finance staff training, preparation of monitoring reports, and consultants to several departments in the MOE. Sub-component 3.2: Development of an M&E system. This sub-component was to build an M&E system both at the sector level and at the regional and district levels so that the various levels of the MOE had data to monitor indicators and guide decisions with the use of early grade learning assessment data. It was to support an expansion of education management information system (EMIS); a Service Delivery Indicator (SDI) exercise to provide better data on teacher behavior and performance; capacity building in policy analytics at the ministerial level; and continued to support the internal audit department and M&E. Sub-component 3.3 Strengthening the decentralization of the education sector. This sub-component addressed capacity-building to strengthen decentralized levels in the MOE, including inspectorate services Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) (regional and district). The project was to finance consultants, workshops, and equipment for regional and district levels to develop capacity in financial management, planning, M&E, and communication. Component 4: Empowerment of Girls and Young Women in Education (AF, US$6.9 million; actual, US$5.72 million). This component was added with the final additional financing in May 2021 and supported the empowerment of girls and young women in 300 junior and upper secondary schools in the five poorest performing regions: Kedougou, Kolda, Tambacounda, Kaffrine, and Matam. Activities included: adapting the performance-based contract system to ensure it was gender informed; establishing a system to detect girls at-risk of dropping out; elimination of gender-based violence via codes of contact for personnel; provision of remediation; training to provide psycho-social support for adolescent girls; and provision of menstruation hygiene supplies and rehabilitation of school infirmaries. The project used a disbursement-linked indicator (DLI) approach to finance the implementation of activities, based on annual work plans for each component and achievement of DLIs. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: Project costs were US$136.6 million (original amount US$69.7 million and AF of US$66.9 million). Actual costs were US$125.69 million, representing 92% disbursement. With exchange rate losses, the undisbursed amount returned to the Bank was US$1.18 million. Costs for component two (117%) were above estimated costs with AF, while costs for components one (90%), three (64%), and four (82%) were below estimated costs. The ICR did not explain why less was disbursed than planned for components one, three, and four. Financing: The parent project was financed by separate sources totaling US$69.7 million, comprised of an IDA credit of US$20 million, a Global Partnership for Education grant of US$46.9 million, and an Education Sector Capacity Building Trust Fund Grant for US$2.8 million. Two AFs were provided. The 2018 AF was funded by an IDA credit of US$60 million, and the 2021 AF of US$6.9 million was funded from the Umbrella Facility for Gender Equality Multi-Donor Trust Fund. Overall, the total funding was US$136.6 million equivalent. Borrower Contribution: The Borrower did not contribute, as noted in the Lending Agreement. The ICR (p. 12) explained that while the Project Appraisal Document indicated the Borrower was expected to contribute US$ 148 million, the Financing Agreement and Grant agreements indicated that IDA and the other financing sources were to fully finance the Project. Dates: The operation was approved by the Board on June 13, 2013 and became effective on February 10, 2014. The operation closed on June 30, 2022. The operation was expected to close on January 31, 2018, but closed three and a half years later on June 30, 2022. Over the course of the operation there were five level two restructurings that were predominantly used to change the closing date (restructurings on October 30, 2015; March 1, 2016, April 23, 2018) and give time for the processing of the Additional Financing (April 23, 2018). Additional Financing were added on May 9, 2018 and May 6, 2021. During the restructuring changes were also made to the results framework and allocation between components. The restructuring also enabled the Government to finalize construction of the four regional training centers, construction of Science and Technology Blocks and equipment to the Blocks, and build 10 remaining primary schools (ICR p. 18). Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) 3. Relevance of Objectives Rationale The objectives were highly relevant to country context. Regional disparities in school enrollment and completion were evident, given lack of adequate infrastructure and parental preference (ICR, p. 7). In five out of 14 regions in the country, children were lagging behind in education due to parental desire for a closer alignment of the education system with their religious beliefs. A large proportion of students attended Daaras (Koranic schools), where students memorized the Koran but did not acquire adequate numeracy and literacy skills. The number of youth who have never attended school is high. Learning outcomes were low. Only 62 percent of students in Grade 5 obtained the required skills in mathematics, based on the francophone regional student learning assessment (Program for Analyzing Education Systems of the Conference of Education Ministers). There were also issues with teaching and learning. Teachers were not well qualified and not familiar with up-to-date curricula, and initial training received in Teacher Training Institutes was inadequate. The aims of the government’s Sector Plan and National Strategy for Economic and Social Development were consistent with the objectives of the operation. The government’s Sector Plan (2013-2025) promoted achieving universal primary completion and increasing access to secondary and tertiary education; improving education quality at all levels, including learning outcomes in primary grades; and eliminating disparities associated with region, gender, poverty status, and rural/urban location, and inclusion of children and youth with disabilities. The government’s National Strategy for Economic and Social Development (2013-2017) acknowledged the importance of quality education. The objectives were highly relevant to country context and Bank strategies throughout the life of the operation. The objectives of the World Bank Country Partnership Strategy (FY13-17) and World Bank Country Partnership Framework (FY20–FY24) focus on equity, improving human capital, and improving results. The project aligned with the 2021 World Bank Africa West Strategy targets to reduce learning poverty from 80 percent to 76 percent and increase girls' secondary school enrollment from 42 percent to 47 percent. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Improve learning outcomes for early grades Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) Rationale Improved learning outcomes for early grades were to be achieved through implementation of performance- based contracts, which supported the establishment and utilization of learning assessments, improved financial management at the school level, and system-wide improvements to teacher management. Training in school management was to enable administrators to adequately assess school performance, provide feedback to teachers, and assess professional development needs. Capacity building at all levels of the system was intended to strengthen the connections between performance-based contracts, data systems, and teaching and learning. The results chain for this objective also recognized the importance of system coherence of all levels of the system (central, regional, district, and school) and stakeholders, consistent with the World Development Report (2018). Including the root cause of inadequate teaching skills (pre-service training) was an important aspect included in the project; however, the initial activities supporting pre-service training were limited, and with AF the subsequent activities further emphasized on-the-job training, which did not address the root cause. Outputs Teacher and Head Teacher Training All newly recruited teachers were trained with the new program for teaching science and math and certified in regional training centers, which met the target. This was a disbursement linked indicator. The ICR (p. 24) reported these new training programs were aligned with the curriculum, but it did not explain or provide evidence of application by teachers in classrooms. 21,056 teachers received on-the-job training, missing the target (25,000). The ICR (p. 24) clarified that training was impacted during the global pandemic in 2020. All school heads were trained, which exceeded the target (60 percent). Regional Training Centers in Diourbel, Kaolack, Louga, and Fatick provided in-service training that focused on scripted lessons (which is helpful for teachers with lower levels of teaching capacity and content knowledge) and a classroom observation tool. The ICR reported that the use of the tool had a positive impact on classroom instruction. Pre-service Teacher Training A new pre-service teacher training curriculum is currently used by the Regional Training Centers. Student Learning Assessments A learning assessment system and Early Grade Reading Assessment and mathematics testing were carried out for primary grades during the operation. The National System for the Evaluation of Internal Returns of Senegal measured performance thresholds in French, mathematics, and life skills in grade 2 and grade 4. The ICR noted, “the system included but was not limited to: (i) establishment of a technical team to develop a strategic approach to learning that included student evaluation, teacher interviews, and lesson observation; (ii) development of standardized criteria for grade progression; (iii) administration of pre-tests at the beginning Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) of the school year; (iv) administration of two formative evaluations during the school year; (v) sharing of assessment results with focal points in the region and districts as well as schools that were used to develop roadmaps for teacher training; and (vi) development of remedial programs where necessary" (p. 23). Thus, a notable part of this process was additional support provided by the National Research Institute for Education to decentralized levels to use the data for decision-making and feedback. The data from learning assessments were part of the performance-based contracts, which according to the ICR (p. 23), “has improved promotion rates and success rates of the primary schools," which is plausibly related to this operation. During the years of the operation, Senegal participated in a regional assessment Education Systems Analysis Program that evaluated learning outcomes for grades 2 and 6 of primary education. Overall project and decentralized levels The project benefited 2,700,000 students, of whom 56 percent were female. 2,059,073students benefitted from direct interventions to enhance learning, which exceeded the target (1,875,000). Of the students benefiting from direct intervention to enhance learning, 553,500 were girls, which missed the target (571,500). The ICR did not describe the intervention, or how benefit and improvement were measured. 77 percent of districts achieved all least 95 percent of established targets, missing the original (80 percent) and revised (100 percent) targets. The ICR (p. 24) reported that the contracts containing the "established targets" were a roadmap that connected training needs based on student evaluation, which is a good practice to improve education quality and learning. 98 percent of schools had Quality Improvement Plans, which exceeded the original target (90 percent) but slightly missed the revised target (100 percent). This was a disbursement linked indicator. The ICR (p. 24) noted that areas of focus of the Plans were girls’ enrollment parity, establishment of pedagogical groups, teacher training, and integration of information and communications technology. All school have a functional school-based management (SBM) committee, which exceeded the original target (90 percent) and met the revised target (100 percent). This was a disbursement linked indicator. The ICR (p. 25) noted that the government has such confidence in the committees that it has increased the allocation to schools from less than US$1 to US$7 per student. The committees manage the funds. The ICR also reported learning from this experience. It noted (p. 12) that a Japan International Cooperation Agency (JICA)- financed pilot included the creation of a national SBM technical team within the Primary Education Directorate (Direction de 'Enseignement Elementaire) that developed the tools, tested them, and carried out an evaluation. JICA continued to provide technical assistance for scale-up of the SBM model as well as cellphone facilities, software, and database systems to allow communities to report on the quality of service delivery, notably school functioning, resource transfers, concerns about the utilization of funds, and teachers' presence and absenteeism. 13,941 school community members were trained using the materials in local languages, which nearly met the target (14,000). The ICR clarified that the training was impacted due to the global COVID-19 pandemic, when training had to be suspended. Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) All Academic Inspectorates and Inspectorates of Training and Education have at least two officials in human resources, statistics, and budget management trained to handle their roles and responsibilities, which met the original (95 percent) and revised (100 percent) targets. The ICR noted that the transfer of these roles and responsibilities has allowed the regional and departmental levels to improve the management of the education system and monitoring at the decentralized level. A human resources, statistics, and budget management system, including software and equipment, was established in all Academic Inspectorates and in all Training and Education Inspectorates. An EMIS was created that includes dashboards with data on location of schools, condition of school structures, students, teachers, textbooks and materials, financial resources, and exams, and is tailored to each level in the system. This information system has been institutionalized. At the school level, the dashboards were piloted in 100 elementary schools, 100 middle schools, and 31 high schools. The project also supported training on the system for the 16 regions and 59 districts. A regional education report is produced yearly about the education system at the regional level. MOE used this report for its sector review. During the COVID-19 pandemic, MOE produced a report on the pandemic situation in Senegal and provided strategies and health measures for the reopening of schools. Outcomes The percentage of students who, by the end of grades two and four of primary schooling, demonstrated the ability to read and comprehend grade level texts increased from 34 percent to 73 percent for grade two, and 52 percent to 75 percent for grade four, which exceeded both the original targets (39 percent for grade two and 56 percent for grade 4) and revised targets (63 percent for grade two and 65 percent for grade four). The percentage of students who, by the end of grades 2 and 4 of primary schooling, demonstrated the ability to compute and solve grade level math problems increased from 51 percent to 74 percent for grade two, and 46 percent to 70 percent for grade four, which exceeded the original targets (56 percent for grade 2 and 51 percent for grade 4) and revised targets (65 percent for grade 2 and 65 percent for grade 4). The percentage of students who, by the end of grade 1 of primary schooling, demonstrated the ability to read and comprehend grade level texts increased from 54 percent to 66 percent, exceeding the original target (60 percent) and the lowered revised target (55 percent). The percentage of students who, by the end of grades 2 and 4 of primary schooling, were able to integrate experimental approaches or reasoned investigation into learning increased from 0 percent to 67 percent in grade 2, and from 2 percent to 62 percent in grade 4, which exceeded the target for grade 4 (50 percent), but not the target for grade 2 (80 percent). The target was not achieved “largely due to the fact that the primary school programs that were to be revised were not completed in time to reinforce the experimental approaches" (ICR, p. 23). Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) The COVID-19 pandemic had an impact on the final months of project implementation. The activities most impacted by the pandemic were civil works and the in-service teacher training, because teachers could not participate in short-term training programs. Rating Substantial OBJECTIVE 2 Objective Increase access to science and mathematics tracks in secondary schools Rationale The ICR (p. 25) notes the areas that were expected to contribute to improvements in the objective included: (i) provision of teaching guides and training of all middle school teachers on the guides; (ii) implementation of a revised middle school curriculum; (iii) performance-based contracts to middle schools to implement the reform programs related to science and mathematics; (iv) construction and renovation of 28 Science and Technology Blocks and provision of equipment; (v) pilot technology-based teaching methods for science and mathematics and utilization of new equipment; and (vi) construction of two new Lycées focused on science and mathematics. These activities would not be sufficient to address the barriers to lack of access to science and mathematics for all. The barriers included teachers’ competencies (which implies changes in pre-service training, as well as robust on-going training and support to address weak content and pedagogical skills); curriculum; behavior/attitudinal changes to ensure teachers provide mathematics and science for all, including girls, and not just to those academically advanced; resources/equipment; and measures to ensure coherence at all levels of the system. Changing gender norms required additional strategies that were needed from the initiation of the operation; the addition of these elements at the AF was too late for meaningful change to take hold during the project's lifetime. Outputs School infrastructure and facilities The project supported the construction and/or rehabilitation of 28 Science and Technology Blocks. Two new Lycées were not constructed as planned. The number of students enrolled in project-supported Lycées was 0, which did not meet the target for number of total students (600) or the percentage of female students enrolled in project-supported Lycées (50 percent). The ICR (p. 26) stated that the Lycées will be completed by the government. Teacher Training 92 percent of middle school teachers were trained to know how to use the new teaching guides, which nearly met the target (100 percent). This was a disbursement linked indicator. The ICR did not describe the evidence Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) to substantiate utilization of the training, which was another disbursement linked indicator (percentage of middle schools implementing the new science and mathematics-oriented program). Overall education system and decentralized levels Districts signed performance contracts with 300 secondary schools (that included the 28 new Science and Technology Blocks) to promote science and mathematics through activities such as science and mathematics careers days; communications and outreach programs through radio, flyers, leaflets etc.; organization of science clubs; and teacher training. The government developed a digital technology strategy that was piloted in 100 primary schools, 300 lower secondary schools, and 30 high schools to strengthen science teaching throughout the system. The project financed: 6,699 laptops; 332 laptops that converted into tables for academic teams; 247 video projectors; 231 computer whiteboards, 16 Wi-Fi routers; and other equipment. The ICR noted that this pilot was successful and will be further implemented in the follow-up operation. The ICR (p. 26) did not state how the pilot was evaluated or what specifically was learned from the experience. Outcomes The percentage of middle school students enrolled in math and science tracks (S1, S2) in secondary level remained nearly constant, at 32 percent and 31 percent at baseline and endline, not meeting the target (45 percent). The percentage of female middle school students enrolled in math and science tracks (S1, S2) in secondary level increased from 24 percent to 54 percent, exceeding the target (30 percent). The ICR does not explain the contradiction of girls' enrollment increasing from 24 percent to 54 percent while the overall rate of enrollment remained consistent, which would seem to have implications for boys' enrollment in the math and science tracks. The ICR (p. 26) attributed the limited progress on this objective to delay in the government’s national science improvement strategy and teachers’ attitudes that focused upon those skilled in science, instead of encouraging scientific thinking and experiences among all students. Since the close of the operation, the national science improvement strategy has been completed. The government has taken action to modify the bylaws of the teachers' Orientation Committee to promote science and mathematics among all students. The government has also trained inspectors and trainers on the aligned curriculum and evaluation. The curriculum will be revised during the Bank’s follow-up operation. Rating Modest OBJECTIVE 3 Objective Improve equity in access to basic education Rationale Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) Improvements in equity were to be achieved through: (i) construction of new primary schools in less advantaged areas, referred to as “lagging areas”; (ii) provision of performance-based contracts to Daaras that expressed an interest in implementing a special curriculum that focused on reading and mathematics; (iii) implementation of a catch-up program for out-of-school children; and (iv) implementation of programs to encourage girls to remain in school as well as return to school. These activities were targeted in regions with enrollment challenges, and the overall approach of the project -- performance-based contracts and alignment of actions of all levels of the education system (data, stakeholders) -- were integrated with these activities. Outputs School infrastructure 82 schools were built by the project to extend the primary education network. 144 new schools in five lagging regions (in terms of access) were built. This was a disbursement linked indicator. The regions and distribution of schools was: (i) 37 in Kaffrine, (ii) 28 in Diourble, (iii) 30 in Tambacounda, (iv) 20 in Louga; and (v) 29 in Matam (ICR, p. 27). Gender 20,000 girls directly benefitted from socio-emotional health and well-being interventions, which met the target. The ICR did not describe these interventions in detail or how benefit was measured. 1,593 action plans integrating gender and adolescent protection were implemented (at the institution, district, and regional levels), which exceeded the target (350). 5,000 education personnel were trained on gender-informed management approaches, which met the target. The number of school-based focal points who were trained on gender-based violence reporting and mitigation measures increased from 20 to 200, which met the target. 1,486 institutions developed action plans that integrated gender with implementation of those plans. Other activities that supported enrollment of adolescent girls and removed barriers included: training of 40,000 girls on self-development and leadership; provision of 12,900,066 disposable menstrual hygiene kits for middle school and high school girls; provision of 22,773 cash transfers for high school girls in vulnerable situations that might lead to dropping out; and provision of 1,862 bicycles to girls for transportation to school, which was done in connection with communities. A strategy to strengthen girls' leadership in schools was developed by the National Center for Vocational School Guidance. 2,500 manuals and 2,000 workbooks on this strategy were distributed. 280 female school leaders received capacity development on gender, administrative, and financial areas. Lagging regions Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) In Kafrine, the number of non-repeaters increased from 35,365 students in grades 1 and 4 to 46,590 students, which met the original target (43,611) but not the revised target (49,721). In Tambacounda, the number of non-repeaters increased from 65,398 students in grades 1 and 4 to 90,728 students, which exceeded the original (81,172) and revised target (86,086). In Matam, the number of non-repeaters increased from 51,496 students in grades 1 and 4 to 66,684 students, which exceeded the original (64,063) but not revised target (71,547). In Louga, the number of non-repeaters increased from 66,834 students in grades 1 and 4 to 86,748 students, which exceeded the original (83,034) but not revised target (92,692). In Diourbel, the number of non-repeaters increased from 69,324 students in grades 1 and 4 to 123,081 students, which exceeded the original (86,103) and revised target (109,907). 82 percent of Daaras achieved 75 percent of the agreed targets in their performance contracts, which exceeded the original target (80 percent), but slightly missed the revised target (85 percent). The ICR noted that to be eligible, Daaras had to establish a Daara management committee that was fully participatory and introduce teaching in French. This was a disbursement linked indicator. 527 performance contracts were signed between local districts and Daaras (Koranic school owners), which exceeded the target (500). 1,580 members of the Daaras management committees were trained, and 527 Daaras buildings were rehabilitated with at least one shed structure, one waterpoint, and toilets. Performance contracts included reading and mathematics textbooks and teaching guides, iron supplements and deworming for children, and creation of school canteens in each Daaras. During COVID-19, additional support such as hygienic kits and sanitary products, nutritional support, and awareness campaigns were provided to 1,043 Daaras, as they remained open during this period. Out-of-school children 56,869 out of school children returned to schools in the system, which exceeded the target (15,000). Out of the children who returned to schools in the system, 42 percent were girls, which exceeded the target (30 percent). Notably, MOE supported the return of children via school management committees conducting outreach to parents and children, and implementation of catch-up programs to re-integrate these children. Facilitators were hired for four years to support the catch-up program. Distance learning Distance learning was supported during the project via provision of computers and video equipment to provide teaching from national television. TeckTal (a mobile learning platform that provides a turnkey plug- and-play solution for teaching and learning) was leveraged during COVID-19-related shutdowns. 300 teachers were equipped with tablets and trained to use the platform, which resulted in 1,000 courses for the platform that were validated by the teaching team and put online. The ICR did not report on which groups of students were able access the course. Outcomes Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) The impact evaluation (IE) conducted of the Daaras under the project suggests that the project’s interventions plausibly contributed to reading and mathematics improvements for Daaras children. The ICR reported: “The IE showed that children who attended Daaras with a teacher who provided approximately eight months of instruction in basic mathematics and French demonstrated higher levels of learning outcomes as well as social-emotional skills than those who attend Daaras without such a teacher. In French, the children who had eight months of instruction had scores that were 23.2 percent higher, while in mathematics children scored 25.7 percent higher. The IE also showed that the impact of the interventions in mathematics and French was greater for girls than boys. Interestingly, the IE also showed that while the impact of teaching mathematics and French was more impactful for girls, the impact on socio-emotional skills was more significant for boys, particularly related to the areas of exploring new horizons.” School census data presented in the ICR shows gender parity for the 2022 school year across all levels of education, preprimary to upper secondary. The ICR did not present trends to show whether gender parity was achieved across the life of the operation, or this was a change. The ICR did not present disaggregated enrollment trends data showing that equity in access improved during the life of the operation. It should be noted that the MOE, with support from the West African Development Bank, built 4,967 new classrooms as well as latrines and other needed facilities during this operation's time frame, which likely also contributed to achievement of this objective. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The overall rating for efficacy is substantial. Two out of three objectives were rated substantial. The project has provided a foundation for further improvements to the education system and learning. The project supported the decentralization of the system using performance-based contracts that had a strong focus on quality improvement, learning, and equity. Notably, the minimum proficiency in reading and mathematics increased in early grades. Several other critical inputs into the education system were realized: data systems and student learning assessment systems. Access improved in the disadvantaged areas as well as in Daaras with the provision of additional classrooms and improved facilities. Supported Daaras are teaching French and mathematics, which has improved learning, and a large number of out-of-school children have returned to school. Overall Efficacy Rating Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) Substantial 5. Efficiency The economic justification for the investment was based on higher returns and earnings and better employment opportunities for workers with higher levels of skills. For example, for those with some post-secondary education, earnings are more than four times those of an individual with no formal education (ICR, p.30). At the national level in Senegal, one additional year of schooling increases income by 10 percent (ICR, p.30). A cost- benefit analysis that estimated the net present value (NPV) and internal rate of return (IRR) for the whole project was not conducted at appraisal. Instead, the PAD provided an overview of economic, fiscal, and social context. An economic analysis that estimated the benefit from the Additional Financing as well as the associated cost was conducted at the time of the AF. The analysis was based on component-specific intervention design, beneficiary targeting, and quantifying the benefits, which were adequate assumptions. Three sets of cost-benefit analyses were conducted related to the AF, related to: quality improvement (component 1), equity in access (component 2), and capacity building (component 3). The analysis under the base scenario found the aggregated IRR for the total AF at 21 percent. The present discounted cost was US$24 million, with the present value of benefits estimated at US$122.7 million. The resulting NPV was US$98.8 million. At closure, the estimate added the fourth component along with the other three components. The lower bound of the present discounted value of benefits for the overall project was US$312.9 million. The present discounted value of costs was estimated to be US$75.6 million. The corresponding NPV was US$237.3 million. The internal rate of return (IRR) associated with this NPV was 24 percent. The benefit-cost ratio was 3.9 at the ICR stage. This suggests positive returns. There was near full disbursement of the credit (92 percent). The use of project funds was adequately monitored over the course of implementation by regular reporting and internal controls and external financial audits regularly submitted to the Bank. However, implementation delays impacted activities. For example, there were delays in developing mathematics and science teaching strategies, and the government had to finish school construction after the operation closed, which created additional costs for the government. Delays in the civil works for the Lycées were due to issues with procurement capacity, contract validation, and topographical studies. Several extensions to the closing date were needed. Moreover, after training teachers and inspectors on the science curriculum, it was decided that the curriculum would be revised during the follow-up operation, after a curriculum assessment conducted late in the operation revealed weaknesses, suggesting an inefficient planning and implementation process. All of these aspects suggest inefficiencies in implementation. On balance, the project's investments produced value for money, but there were significant implementation inefficiencies. Efficiency Rating Modest Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 100.00 ICR Estimate  24.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives is assessed across the entire project and is rated High, given high relevance and alignment of the objectives with government and Bank strategies over the life of the operation. Efficacy is rated Substantial overall, but with moderate shortcomings related to modest achievement of the objective to increase access to science and mathematics tracks in secondary schools. The operation laid a foundation to improve learning, equity, and access to math and science for girls. The support to decentralized levels of the education system had a strong focus on learning and equity. Notably, there have been improvements in lower-grade proficiency in reading and mathematics. Efficiency is rated modest across the entire project due to implementation inefficiencies. Thus, the outcome is rated Moderately Satisfactory, indicative of moderate shortcomings in the operation’s preparation, implementation, and achievement. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Several project activities have been institutionalized. The original capacity risks were mitigated through the institutional strengthening within the MOE's governance structure, and a robust data system is now in place and data are used by all levels of the system for decision-making. Performance-based contracts are being continued under the follow-up project, as is expansion of support to other Daaras. Distance learning capacity has been improved and will be expanded under the Bank’s follow-up operation. These aspects contribute to the likelihood of project interventions being sustained. However, Senegal still has substantial challenges related to access, particularly for out-of-school children. At the secondary level, the country still needs to increase learning outcomes. Capacity issues persist, as illustrated by this project's implementation delays in the mathematics and science teaching strategy and in the civil works for the Lycées. 8. Assessment of Bank Performance Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) a. Quality-at-Entry There were several strengths to preparation. Design was based on strong contextual understanding of the education system and decentralized structure in Senegal. The project's design was informed by lessons learned from the predecessor operation, the Senegal Second Quality Education For All (QEFA2), and experience from other operations using DLIs and involving multiple stakeholders to improve learning. For example, the Bank team drew upon past experience in: (i) developing DLIs to be a mechanism to create incentives and accountability; (ii) designing an operation with adequate decentralized resources; (iii) reinforcing school leadership and local authorities to contribute to improvement in learning outcomes; and (iv) empowering local communities to participate in the management of service delivery and providing feedback. The Bank team identified appropriate risks and appropriate mitigation measures. For example, limited capacity of project staff in monitoring interventions such as performance-based contracts was addressed by adding field-based task team members to provide the necessary support and technical assistance. However, there were design weaknesses. As noted in the monitoring and evaluation section, the results framework lacked outcome indicators to measure improved equity of access. For example, one of the outcome-level indicators for the 2021 AF measuring equity-related outputs rather than outcomes. In addition, there were design weaknesses. Project activities were not sufficient to address the barriers to lack of access to science and mathematics for all. It was also late (final year of implementation) to begin the gender empowerment work that should have been embedded into the approach and design from the start. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The Bank team was actively engaged in supporting the government in its efforts to implement the project over the nine-year implementation period. The Bank worked closely with the Directorate of Planning and Education Reform, which was responsible for overall coordination of the operation, as well as other technical units such as the Directorate for Elementary Education and the Directorate of Middle and General Secondary Education, who were responsible for monitoring implementation of the performance-based contracts and primary and middle school project activities. Project coordination by these entities was in addition to their other ministerial responsibilities, which had a negative impact upon project implementation and resulted in the Bank team proactively providing additional technical support. The Bank also identified a remedy when it requested that MOE create a team solely responsible for overseeing project implementation with clear roles and responsibilities. This change improved implementation (ICR, p. 36). The Bank was responsive to the realities facing Senegal during the global pandemic. There was supervision continuity with the MOE and the Bank team (composed of both international and local staff), which allowed the Bank team to regularly meet with the MOE and the various implementation units. The Bank supported the government’s request to extend the project to complete civil works that had been delayed because of issues related to procurement, counterpart funding, and the COVID-19 pandemic. Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) The Bank secured two Additional Financings for the operation. Notably, the second one was aligned with the government’s focus on gender empowerment and its analysis, but this was late (final year of implementation) to begin what should have been embedded into the approach from the start. The Bank effectively monitored risks during implementation and was conservative with ratings to ensure appropriate mitigation measure were in place. The Bank and the implementing agencies enhanced implementation by communicating and engaging with a variety of stakeholders to understand the value of project activities. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project's objectives, components, and outcome measures were logically linked, but there were weaknesses with the results chain and measurement with one objective (i.e., increased access to mathematics and science). The results framework was predominantly composed of output indicators, but these measures assisted in tracing the results chain toward achievement of the objectives. There were also appropriate outcome indicators, such as increased learning outcomes. However, indicators added in the final AF were weak, as they did not measure changes in gender equity; instead, they were outputs measuring implementation of activities (i.e. number of education personnel trained on gender-informed management approaches; number of school based focal points trained on gender-based violence reporting and mitigation measures; number of girls directly benefitting from socio-emotional health and well-being interventions; and number of action plans integrating gender and adolescent implementation). Additional efforts, beyond indicators in the results framework, were also part of monitoring and evaluation. The project supported establishment of a monitoring system applicable for each level of the system. Notably, design also included student learning assessments that measured improvements in learning, and an impact evaluation of the Daaras was implemented. Finally, there was independent validation to confirm achievement of DLIs. Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) b. M&E Implementation The Division of Planning and Education Reform was responsible for monitoring and evaluation and overseeing capacity building activities (as well as overall project coordination). The operation built the technical and managerial capacities of the regions and districts so that they had adequate data management skills. There was a clear timeline for data collection and analysis and explicit assignment of responsibility. Indicators and performance-based contracts were monitored regularly. During the operation, additional evaluations were undertaken. A Service Delivery Indicators (SDI) survey of elementary schools with a second-grade elementary class was updated. There was a diagnostic of regional training institutions. In addition, an impact evaluation was conducted to examine the impact of the intervention in Daaras. This evaluation provided evidence of learning among children who attended Daaras and had a teacher who provided approximately eight months of instruction in basic mathematics and French. c. M&E Utilization There were multiple examples of how the monitoring and evaluation activities conducted under the project were utilized by the government. For example, the government used the annual sector report during its review meeting. The results of the impact evaluation informed future support in Daaras. Data from the data platform were used among schools and at the district, regional, and central level to aid in decision-making. Finally, the performance-based contracts were reviewed to inform follow-on contracts. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The environmental assessment category was B at appraisal and remained category B with the two Additional Financings (as no activity was expected to have a significant negative impact). The project triggered: OP/BP 4.01 Environment Assessment; OP 4.11 Physical Cultural Resources, and OP 4.12 Involuntary Resettlement. A Resettlement Policy Framework was prepared. The operation updated the Environmental Social Management Framework previously prepared under the predecessor project. The safeguards were monitored during implementation and were consistently rated satisfactory and moderately satisfactory by Implementation Status and Results Reports (ISRs). The project employed additional safeguard protections. First, a Grievance Redress Mechanism was established in May 2020.This process was used twice to resolve complaints registered in relation to recruitment of project staff and another related to a Koranic School. The second AF put in place additional activities to mitigate gender-based violence through codes of conduct, training, and awareness-raising activities. Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) b. Fiduciary Compliance Financial Management Ratings were satisfactory or moderately satisfactory for fiduciary management in ISRs for most of the life of the operation. Financial reports were regularly submitted and of acceptable quality. The required audits were submitted to the World Bank, and the auditors’ opinions were clean over the project period. However, issues emerged and ratings were downgraded in 2017. The issues included: (i) delays in the transfer of resources by MOE to the regions and districts due to lack of a financial unit, and (ii) delays in provision of financial reporting and technical reports that led to a lag in disbursements from the project accounts. These shortcomings were resolved when qualified staff were hired and a financial unit was established that had greater linkages to the regional level. During the life of the project, the Bank’s financial management team provided training to build financial management capacity in the implementing agency. The ICR (39) noted that borrower contributions to the sector were a problem in 2017 and 2018; this operation was fully financed by the Bank, GPE, and Trust Funds. Procurement Ratings for procurement were satisfactory or moderately satisfactory in ISRs throughout the life of the operation, with one exception. In this one case (December 2020), the rating was lowered due to lack of documentation. This situation was resolved when a new procurement staff was hired. From that point, data accurately reflected contract progress, which contributed to better contracts management and timely delivery of goods and works. World Bank procurement staff provided training and technical guidance to the implementing agency. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderate shortcomings in Bank Performance Satisfactory Moderately Satisfactory project and M&E design. Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons IEG has synthesized the lessons presented in the ICR (pp. 40-41) as the following: Establishing a feedback loop from central, decentral, and school levels that uses the information from student learning assessments shifts the focus to improved learning, which is a necessary step. One of the mechanisms the project used was performance-based contracts that resulted in schools, head teachers, and school management committees examining learning data, after receiving capacity development to discuss how to use resources provided by MOE to improve learning. This was complemented with other actions to support learning by central and decentral levels: training (to understand the data and how to improve teaching practices) and inspection. The government has expanded the approach throughout the country. These actions are important and necessary, but not sufficient, to improve student learning. Other actions are also needed across the system to ensure system coherence in support of learning: curriculum and teaching, capacity across all levels of the system, and political commitment. Sound policy dialogue based on evidence and contextual understanding can play a role in creating a pathway between the conventional education system and the Koranic education system. This project strategically launched a competitive process to support Daaras that were willing to integrate French and mathematics into their traditional curriculum. This was done through awareness-raising, continuous dialogue, the involvement of Koranic teachers, and patience, as well as contextual understanding. These sustained efforts led to the introduction of French and mathematics in the project-supported Daaras. The project’s evaluation showed the approach was working and opened an important dialogue on expansion of the activities to other Koranic schools. This is important for the country to have children with literacy and numeracy skills, since the level of education is related to individual future earnings and poverty. This also shows the importance of the Bank’s contextual understanding and dialogue. Emergency crisis situations such as the global pandemic require flexibility and can present opportunities for change. The outbreak of COVID-19 showed that the Bank team was able to react quickly to unforeseen events and to find the means in the financing agreement to help the country deal with unexpected situations. During the pandemic, formal schools were closed; however, Daaras remained open. Because of the health concerns and needs of the Daaras, the project supported 1,000 Daaras with food, sanitation kits, and sensitization on social distancing. Additionally, the project team and MOE held discussions to establish mechanisms to ensure learning continuity during the pandemic. As a result of these timely discussions, the Bank's follow-up operation aims to provide learning continuity throughout the education system through digital technology, broadcast, and radio. Changing attitudes and access to mathematics and science for all students -- regardless of ability and gender -- requires a comprehensive approach. This project had an incomplete set of activities to improve access to math and science, which was a design weakness. Other activities directly changing teachers’ skills and attitudes and behaviors would also be needed. Moreover, the Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SN Quality and Equityof Basic Education (P133333) government’s national science improvement strategy was delayed. As a result, some on the Orientation Committee viewed access to mathematics and science as only for capable learners. In a decentralized system such as Senegal's, central-level actions are necessary to support access to mathematics and science in the form of well-defined strategies and decrees, but implementation also requires building capacity among decentralized levels to ensure the competencies and awareness that all learners need access to mathematics and science. It also requires provision of resources and training for decentralized levels and teachers. Project implementation shows the importance of engaging with diverse group of stakeholders, particularly in relation to gender norms. 13. Assessment Recommended? No 14. Comments on Quality of ICR The quality of the ICR was substantial, but additional information in some areas of the ICR would have made the validation easier. The ICR provided a concise description of the project, its activities, and the factors affecting implementation, but it was overly repetitive. It analyzed shortcomings in design, preparation, and Bank performance. The rating for efficiency was not explained well, particularly implementation efficiency. The ICR reported indicator data and explained when targets were not met and provided a broader narrative of achievement of objectives. It did not report on all disbursement linked indicators noted in Annex 7 (D1, D10 and D11). While not required, it would have enhanced the presentation of efficacy if trend data from enrollment and census data were included to provide evidence of changes in equity for groups, including gender parity. a. Quality of ICR Rating Substantial Page 21 of 21