Document of The World Bank Report No: ICR00003564 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-44010 TF-94223) ON A CREDIT IN THE AMOUNT OF SDR, EURO 12.76 MILLION (US$20.0 MILLION EQUIVALENT) TO THE REPUBLIC OF YEMEN FOR A SECONDARY EDUCATION DEVELOPMENT AND GIRLS ACCESS PROJECT November 24, 2015 Education Global Practice Middle East and North Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective September 30, 2015) Currency Unit = Yemeni Rial YER 1.00 = US$ 0.005 US$ 1.00 = 215.1 YER FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS BEDP Basic Education Development Project CCT Conditional Cash Transfers DFID Department for International Development DP Development Partners EFA-FTI Education for All - Fast Track Initiative EIE Education in Emergencies EIRR Economic Internal Rate of Return EKN Embassy of the Kingdom of the Netherlands ERDC Educational Research and Development Centre FTA Female Teacher Assistants GER Gross Enrollment Rate GIZ Gesellschaft für Internationale Zusammenarbeit ("German Corporation for International Cooperation") GTZ Gesellschaft für Technische Zusammenarbeit ("German Technical Assistance Corporation") GOY Government of Yemen IBRD International Bank for Reconstruction and Development IDA International Development Association IFR Interim Financial Report KfW Kreditanstalt für Wiederaufbau ("Reconstruction Credit Institute") M&E Monitoring and Evaluation MDGs Millennium Development Goals ii MENA Middle East and North Africa MOE Ministry of Education MOHESR Ministry of Higher Education and Scientific Research MOTEVT Ministry of Technical Education and Vocational Training MSI Management Systems International NGO Non-Governmental Organization NGSES National General Secondary Education Strategy NOL No Objection Letter PAU Project Administration Unit PDO Project Development Objectives PWP Public Works Project SMC School Management Committees TPM Third Party Monitoring Senior Global Practice Director: Claudia Costin Sector Manager: Safaa El-Kogali Project Team Leader: Tomomi Miyajima ICR Team Leader: Tomomi Miyajima ICR Primary Author: Joel Reyes ICR Secondary Author: Keiichi Ogawa iii REPUBLIC OF YEMEN SECONDARY EDUCATION DEVELOPMENT AND GIRLS ACCESS PROJECT CONTENTS Data Sheet A. Basic Information ................................................................................................................................. v B. Key Dates.............................................................................................................................................. v C. Ratings Summary................................................................................................................................. vi D. Sector and Theme Codes ..................................................................................................................... vi E. Bank Staff............................................................................................................................................ vii F. Results Framework Analysis .............................................................................................................. vii G. Ratings of Project Performance in ISRs ............................................................................................ xii H. Restructuring (if any) ........................................................................................................................ xiii I. Disbursement Profile ......................................................................................................................... xiii 1. Project Context, Development Objectives and Design ............................................................................. 1 2. Key Factors Affecting Implementation and Outcomes ............................................................................. 5 3. Assessment of Outcomes ........................................................................................................................ 12 4. Assessment of Risk to Development Outcome ....................................................................................... 16 5. Assessment of Bank and Borrower Performance.................................................................................... 17 6. Lessons Learned...................................................................................................................................... 19 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ......................................... 22 Annex 1. Project Costs and Financing ........................................................................................................ 23 Annex 2. PDO Impact and Outputs by Component .................................................................................... 24 Annex 3. Economic and Financial Analysis ............................................................................................... 31 Annex 4. Bank Lending and Implementation Support/Supervision Processes ........................................... 34 Annex 5. Beneficiary Survey Results ......................................................................................................... 36 Annex 6. Stakeholder Workshop Report and Results ................................................................................. 37 Annex 7. Summary of Borrower's ICR ....................................................................................................... 38 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders .................................................... 42 Annex 9. List of Supporting Documents .................................................................................................... 43 Annex 10. Education Impact of the Political Crisis and Armed Conflict in Yemen and Education in Emergencies Lessons Learned .................................................................................................................... 44 Annex 11: Timeline of World Bank Supervision and Issues Addressed ................................................... 47 iv A. Basic Information Secondary Education Country: Yemen, Republic of Project Name: Development and Girls Access Program Project ID: P089761 L/C/TF Number(s): IDA-44010,TF-94223 ICR Date: 11/24/2015 ICR Type: Core ICR Lending Instrument: SIL Borrower: REPUBLIC OF YEMEN Original Total XDR 12.76M Disbursed Amount: XDR 12.76M Commitment: Revised Amount: XDR 12.76M Environmental Category: B Implementing Agencies: Ministry of Education Cofinanciers and Other External Partners: DFID, EKN, KfW, GIZ B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 07/05/2006 Effectiveness: 01/06/2009 01/06/2009 04/05/2012 06/11/2013 Appraisal: 12/17/2007 Restructuring(s): 05/11/2014 11/08/2014 Approval: 03/18/2008 Mid-term Review: 02/02/2014 02/02/2014 Closing: 01/31/2015 05/31/2015 v C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: High Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any Quality of Supervision Yes None time (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 6 8 Secondary education 88 88 Sub-national government administration 6 4 vi Theme Code (as % of total Bank financing) Education for all 67 67 Gender 33 33 E. Bank Staff Positions At ICR At Approval Vice President: Hafez M. H. Ghanem Daniela Gressani Country Director: Asad Alam Emmanuel Mbi Practice Manager/Manager: Safaa El Tayeb El-Kogali Mourad Ezzine Project Team Leader: Tomomi Miyajima Ayesha Vawda ICR Team Leader: Tomomi Miyajima ICR Primary Author: Joel E. Reyes F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To improve gender equity, quality, and efficiency of secondary education in selected districts with a particular focus on girls in rural areas. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Gender Parity Index of Secondary Education Access Rate in the Project Districts Value quantitative or 0.42 0.66 0.60 0.63 Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 vii Comments Original baseline and target for nine governorates. Scaled down project to five (incl. % governorates with a revised baseline of 0.43 and target of 0.60. Surpassed Expected Target. achievement) Indicator 2 : G10 to G12 male retention rate in project districts (%) Value quantitative or 80.6 85.4 81.0 78.7 Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Original baseline and target for nine governorates. Scaled down project to five (incl. % governorates with a revised baseline of 73.0 and target of 81. Increased 5.7 percentage points (2.3 percentages points below expected target). achievement) Indicator 3 : G10 to G12 female retention rate in project districts (%) Value quantitative or 84.2 89.1 84.3 84.5 Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Original baseline and target for nine governorates. Scaled down project to five (incl. % governorates with a revised baseline of 78.3 and target of 84.3. Increased 6.0 percentage points (0.3 percentages points above expected target). achievement) Indicator 4 : Improved learning outcomes in Mathematics in G12 (%) Value quantitative or 19.3 (out of 100) 40 19.3 31.2 at mid-term Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 02/02/2014 Comments Original baseline and target for nine governorates kept the same for scaled down project to five governorates. Achievement test, at time of mid-term evaluation, showed (incl. % increment of 10.9 percentage points—from an average of 19.3 to 31.2 points (out of achievement) 100). Followed up test not conducted due to 2015 armed conflict. viii (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Indicator 1 : Civil works program completed in 90 schools Value (quantitative 0 90 44 43 or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Original target is for nine governorates. Revised target is for scaled down project to five (incl. % governorates (97.7% achieved). achievement) Indicator 2 : 6,800 students attending in project schools using school community grants (SCG) Value (quantitative 0 6,800 5,200 8,141 or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Original target is for nine governorates. Revised target is for scaled down project to five (incl. % governorates (156.6% achieved). achievement) Indicator 3 : 108 female staff financed by the School Community Grants (SCG) Value (quantitative 0 108 89 89 or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Original target is for nine governorates. Revised target is for scaled down project to five (incl. % governorates (100% achieved). achievement) Indicator 4 : 30,000 teachers trained in subject modules (cumulative) Value 0 30,000 17,267 14,349 ix (quantitative or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Original target is for nine governorates. Revised target is for scaled down project to five (incl. % governorates (83% achieved). achievement) Indicator 5 : 60% of trained teachers using at least 60% of teaching skills taught through training Value (quantitative 0 60% 0 or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 Comments Assessment of teachers in classroom dropped from project interventions and indicators (incl. % due to 2015 conflict and insecurity crisis. achievement) Indicator 6 : Number of schools equipped with science labs Value (quantitative n/a n/a 50 50 or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Additional IO added as part of project restructuring targeting only five out of the (incl. % original nine governorates within the project (100% achieved). achievement) Indicator 7 : Number of schools equipped with computer labs Value (quantitative n/a n/a 50 50 or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Additional IO added as part of project restructuring targeting only five out of the (incl. % original nine governorates within the project (100% achieved). achievement) x Indicator 8 : Number of schools equipped with libraries Value (quantitative n/a n/a 50 50 or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Additional IO added as part of project restructuring targeting only five out of the (incl. % original nine governorates within the project (100% achieved). achievement) Number of teachers from project district schools trained on pedagogy training Indicator 9 : certification Value (quantitative n/a n/a 1,500 n/a or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Additional IO added as part of project restructuring targeting only five out of the (incl. % original nine governorates within the project. Training was postponed (0% achievement). achievement) Indicator 10 : Average performance of teachers from the 50 schools in classroom observation test Value (quantitative n/a n/a 70% n/a or Qualitative) Date achieved 02/19/2008 01/31/2015 05/31/2015 05/31/2015 Comments Additional IO added as part of project restructuring targeting only five out of the (incl. % original nine governorates within the project. Training was postponed (0% achievement). achievement) Indicator 11 : Curriculum framework policy document prepared Value Policy Document Dropped from (quantitative No Policy Document Available Project or Qualitative) Date achieved 02/19/2008 01/31/2015 04/05/2012 xi Comments IO dropped as part of cancelation of original component 3 of project (Supporting (incl. % Secondary Education Policy Development), due to 2011 political and security crisis. achievement) Indicator 12 : Feasibility assessment for private sector regulatory framework undertaken Value Framework Dropped from (quantitative No framework available project or Qualitative) Date achieved 02/19/2008 01/31/2015 04/05/2012 Comments IO dropped as part of cancelation of original component 3 of project (Supporting (incl. % Secondary Education Policy Development), due to 2011 political and security crisis. achievement) G. Ratings of Project Performance in ISRs Actual Disbursements Date ISR No. DO IP (USD millions) Archived IDA-44010 TF-94223 1 06/30/2008 Satisfactory Satisfactory 0.00 2 12/23/2008 Satisfactory Satisfactory 0.00 3 04/07/2009 Satisfactory Satisfactory 0.00 0.0 4 12/29/2009 Satisfactory Moderately Satisfactory 0.08 0.0 5 04/07/2010 Satisfactory Moderately Satisfactory 0.08 0.0 6 01/08/2011 Moderately Satisfactory Moderately Unsatisfactory 0.58 0.0 7 08/26/2011 Moderately Unsatisfactory Moderately Unsatisfactory 1.13 0.88 8 04/02/2012 Moderately Unsatisfactory Moderately Unsatisfactory 1.13 0.88 9 09/19/2012 Moderately Unsatisfactory Moderately Unsatisfactory 1.92 0.95 10 06/26/2013 Moderately Unsatisfactory Moderately Satisfactory 5.13 2.74 11 01/11/2014 Moderately Unsatisfactory Moderately Satisfactory 7.96 4.59 12 06/23/2014 Moderately Satisfactory Moderately Satisfactory 12.30 6.62 13 12/22/2014 Moderately Satisfactory Moderately Satisfactory 15.90 6.62 14 05/31/2015 Moderately Satisfactory Moderately Unsatisfactory 19.13 6.62 xii H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions Review project components, 04/05/2012 N MU MU 1.13 financing, and results framework. Reallocate funds to civil works 06/11/2013 MU MU 5.13 component This restructuring is to change both : (i) baseline and (ii) annual/end targets of the PDO indicators 1, 2 and 3, using data 05/11/2014 MU MS 10.96 from 5 project governorates alone, to ensure accuracy of measuring project’s progress towards PDO achievement. Extension of the project closing 11/08/2014 MS MS 14.23 date for additional 4 months I. Disbursement Profile xiii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. At the time of appraisal, the Secondary Education Development and Girls Access Project (SEDGAP) was fully aligned with the country and education sector needs in Yemen, the education sector policies, and the assistance strategies of the World Bank and other donors. 2. Yemen placed in the lowest world socio-economic rankings and was struggling to improve its education sector. In 2006, Yemen was unlikely to meet the Millennium Development Goals (MDGs) and ranked 150 out of 177 countries in that year’s Human Development Index, and 121 out of 140 countries in the Gender Development Index. Given this context, education remained a high priority and Yemen had endorsed at least five different strategies to address challenges in basic, secondary, higher education, technical education and vocational training. Of these priorities, secondary education had received the least investment, especially in rural areas and for girls. The National General Secondary Education Strategy (NGSES), approved by the Cabinet in July 2007, provided the policy framework for systematic development support at this level. It aimed to address gender and rural inequity; inefficiencies due to high repetition and dropouts; and poor quality and ineffective teacher deployment, among other challenges. 3. Gender and regional inequity was considered a crucial education obstacle in Yemen, especially at the secondary level. Female education in Yemen remained a key challenge, especially for adolescents and youth in rural areas. Obstacles were economic and conservative norms. In secondary education, low general access was compounded by considerable gender gaps. The 38.3% gross enrollment rate (GER) in secondary education was extremely low by international standards. This translated to a GER of 59.8% for boys and 25.9% for girls. While low availability of schools and teachers constrained access of boys and girls to secondary education, demand-side attitudinal factors and practices such as early marriage—especially in rural areas—compounded the obstacles for female students, especially after grade 9. Moreover, the provision of secondary schooling in rural areas had to take into account efficiency issues, given a sparse population density and smaller number of students (of the existing secondary schools at the time, only 21% had more than 180 students). In terms of quality, the theoretical and overloaded secondary education curriculum and limited instructional materials hampered learning. Improving quality and efficiency was also affected by the lack of relevant teacher qualifications and deployment policies. In May 2006, Yemen linked teacher posts to the school rather than individuals, providing a leverage to better map teachers’ posts to secondary education demand. At the time, only 7.5% of secondary rural teachers were female and the quality of pre-service preparation for secondary teachers was limited. 4. Yemen’s education policies and strategies sought to address challenges in basic, secondary and higher education. In secondary education, the National General Secondary Education Strategy (NGSES) aimed to provide high quality education in an equitable and cost-effective manner. Especially, it sought to address gender and rural access inequities and inefficiencies due to high repetition rates. The NGSES pointed to the needs of improved curriculum content and delivery, trained teachers, and female representation in the teaching force. In basic education, the National Basic Education Development Strategy (NBEDS) provided a ten-year program for achieving the education related MDGs in the country. This was followed by the approval of the National Vocational Training and Technical Education Development Strategy and the National Strategy of Higher Education. In terms of governance, the education sector in Yemen recognized that decentralization should be advanced further after the 2000 Local Authority Law (LAL). This law entrusted governorates and districts with the delivery of education, and provided the framework to strengthen school-based management and community participation. 5. The Bank’s rationale for involvement in Yemen’s education sector rested in its long-term commitment as a major development partner (DP) in the country. Examples of this includes Bank support for the Basic Education Expansion Project (BEEP) from 2000 to 2007, and for the Basic Education Development Project (BEDP) from 2004 to 2012. A second phase of this latter project (BEDP II) was initiated in 2013. Moreover, the International Development Association (IDA) maintained a close relationship with the education sector through its analytical and advisory support, facilitating a long-term strategic dialogue for education sector development. The secondary education program was a natural extension of past investments under the development of the NGSES, and partnerships with development partners. It benefitted from lessons learned on demand-driven interventions such as conditional cash transfers (CCT) and community participation in schools. These were in line with the Country Assistance Strategy (CAS) which supported investments to reduce poverty through economic and social development, including in education. 6. Donor commitment to Yemen was strong, especially within the context of the Education for All Fast Track Initiative (EFA-FTI) and support for sector-wide approaches (SWAp). Of the 23 countries around the world invited to participate, Yemen was the only country from the Middle East and North Africa (MENA), and one of the first group of 12 countries to be endorsed for participation in the global initiative. Donors were eager to pool their resources to support social programs towards the achievement of the Millennium Development Goals (MDGs) and through SWAps. The Bank played a key role in laying a strong foundation for donor harmonization in Yemen for the past decade through close collaboration with the Local Education Group (LEG) that convened more than 20 partners including multilateral, bilateral donors and national and international NGOs. Backed up by such a strong presence of LEGs, the impetus to support the Millennium Development Goals (MDG) for both education and gender equity, provided a strong foundation for donor harmonization in Yemen. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 7. To address the integrated challenges of education access to quality learning and gender equity, the original project development objective was: “To support the Government of Yemen (GoY)’s program for improving gender equity, quality, and efficiency of secondary education in selected districts with a particular focus on girls in rural areas.” 8. The original key outcome indicators focused on nine governorates with the following targets to be reached in a period of six years: • Gender parity index of secondary education access rates in the project districts increased from 0.42 to 0.66 1; • Grade 10 to Grade 12 retention rate in project districts improved from 80.6% to 85.4% for males and from 84.2% to 89.1% for females; and • Improved learning outcomes in Mathematics in Grade 12 (to be obtained using independent curriculum-based learning assessments conducted in randomly selected schools in project years two and five). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/ justification 9. After the 2011 security and political crisis, and also because of the withdrawal of financing from two of the four original SEDGAP donors, 2 the project reduced its scale from nine to five governorates. The PDO remained unchanged, given its high relevance. Only the scale of the project was adjusted through two “Level Two” restructuring. In April 2012, the first restructure revised the end-of-project targets for both the PDO and intermediate outcome indicators to align to the lower number of participating governorates. 1Dividing the access rates of females by its value for males, where “1” would indicate full parity. 2 In the midst of the 2011 political and security crisis, DFID withdrew its participation in the project along with US$44 million in financing. The Netherlands reduced its original commitment from US$14.8 million to US$6.6 million. 2 In May 2014—to more accurately assess project progress and achievements—the PDOs’ original baselines were revised to reflect those belonging only to the five remaining governorates, and some additional intermediate indicators for component one were added. The approved revisions were as follows: Changes to PDO baselines and targets: • Gender parity index baseline adjusted from 0.42 in the 9 governorates with an end target of 0.66, to 0.43 in the 5 remaining governorates with an end target of 0.60; • Grade 10-12 male retention rate baseline from 80.6% in the 9 governorates with an end target of 85.4, to 73% in the 5 remaining governorates with an end target of 81; and • Grade 10-12 female retention rate baseline from 84.2% in the 9 governorates with an end target of 89.1, to 78.3% in the 5 remaining governorates with an end target of 84.3. Changes to intermediate outcome targets: • Number of schools to receive civil works was reduced from 90 to 44; • Number of students attending in the project schools was reduced from 6,800 to 5,200; and • The number of female teachers financed by the project was reduced from 108 to 89. New intermediate outcome indicators added in Component 1: • Number of schools equipped with science laboratory was targeted to 50 schools; • Number of schools equipped with computer laboratory was targeted to 50 schools; • Number of schools equipped with libraries was targeted to 50 schools; • Number of trained teachers from project district schools with pedagogy training certifications was targeted to 1,500; and • Average performance of teachers from the 50 schools based on a classroom observation test was targeted to 70%. 10. In October 2014, the Bank provided a four-month extension to the closing date (from January 31 to May 31, 2015) to allow the project to finalize all of its activities in the face of the renewed armed conflict in 2015. 1.4 Main Beneficiaries 11. The original project beneficiaries were nine governorates divided into two phases: (1) Abyan, Hadramout, Ibb, Laheji, and Taiz, followed by (2) Aden, Dhamar, Hodaidah, and Sana’a City. In those 9 governorates, intervention was planned for 30 districts and approximately 90 schools. Originally, the project was to benefit approximately 6,800 students through school community grants, 30,000 teachers through a training program, and 108 newly hired female teachers in rural areas. The nine governorates and districts were chosen based on a set of criteria such as regional balance across the country, poverty rate, and low level of Gender Parity Index (GPI) as described on section 2.1 of the SEDGAP operations manual. 12. During the April 2012 restructuring, the project reduced its scope to the phase one five governorates: Abyan, Hadramout, Ibb, Lahej, and Taiz. These included 17 districts and 50 schools, across the southern and northern regions of Yemen. The revised targets were 5,200 students benefited by school community grants, 17,000 teachers trained, and 89 female teachers hired. 1.5 Original Components (as approved) Component 1: Improving equity and reducing gender gaps through demand- and supply-side interventions (US$74 million total; IDA US$15 million) 3 • Subcomponent 1.1. Upgrade and furnish school facilities including extensions, rehabilitations, and repairs of classrooms, science and computer laboratories and multi-purpose rooms. • Subcomponent 1.2. Finance the provision of learning equipment and learning materials appropriate to the existing curriculum. • Subcomponent 1.3. Finance the technical assistance to develop the Operations Manual for the school community grant scheme, as well as for conducting awareness campaigns and building capacity at the governorate, district and community levels (including mothers’ and fathers’ councils) to manage, supervise and monitor the use of the School Community Grants. • Subcomponent 1.4. Provide School Community Grants to finance targeted incentives through community school grants to arrange transportation to school, finance eligible contractual female staff in co-educational schools; and conditional cash transfers to girls in grades 4-12 and to boys in grades 10-12 in selected districts within the project area. Component 2: Enhancing the quality of service delivery (US$23.4 million total; IDA US$4 million) • Subcomponent 2.1. Build and improve teaching and learning practices in the classroom through: (a) the identification and development of upgraded teaching and learning materials for the existing curriculum based on the findings of the textbook revisions from grades 1 to 12; (b) the development and delivery of training to support teachers and school teams, including the related training and professional development of trainers and supervisors; (c) the strengthening of school principals’ capacity to lead and manage schools; and (d) the strengthening of supervisors’ capacity to monitor service delivery. • Subcomponent 2.2. Strengthen the learning assessment system by introducing new examinations in the school system and strengthening the capacity at the school level to prepare and conduct assessments of student performance based on the existing syllabi, texts, and learning materials, through the provision of consultant services, training and goods. • Subcomponent 2.3. Finance capacity building at the central and governorate levels to improve planning, programming and data collection and analysis, through the provision of consultants’ services, training and goods. Component 3: Supporting Secondary Education Policy Development (US$1.5 million total; IDA US$0.3 million). 13. Support the secondary education policy development for longer-term reform through the following studies: (i) review of in-service and pre-service training of teachers; (ii) preparation of a curriculum framework policy document based on a comprehensive review of the current streams and curriculum; (iii) a feasibility assessment for developing a regulatory framework to encourage private sector to invest in secondary education; and (iv) tracer studies on secondary education graduates to assess the effectiveness of secondary education outcomes. Component 4: Project Management and Monitoring (US$4.4 million total, IDA US$0.8 million). 14. Support and strengthen the capacity of the PAU to carry out the fiduciary, monitoring and reporting duties required for the Project’s management. 1.6 Revised Components 15. While maintaining the original Project Development Objectives (of gender parity in access, improved retention rates, and improved learning outcomes using math scores as proxy), the scale of the project was reduced from nine to five governorates (those in its initial phase). The following project scope revisions were made: 4 Component 1: Improving Equity and Reducing Gender Gaps. The overall component cost was reduced from US$74 million to US$33.6 million. There were no changes in the component description, but the targets of the intermediate results indicators were revised. Component 2: Enhancing the Quality of Service Delivery. The overall component cost was reduced from US$23.4 million to US$11.1 million. In addition, the component activities were simplified as follows: • Provision of learning equipment and materials to selected schools; and • Capacity building of teachers, principals and supervisors in selected schools to improve teaching and learning practices in the classroom. Component 3: Supporting Secondary Education Policy Development. This component was cancelled during the project restructuring in 2012. However, some of the initial activities were started before restructuring (for example, study tour to Jordan to review its secondary academic streaming strategy) and US$0.1 million were disbursed. Component 4: Project Management. The overall cost was reduced from US$4 million to US$2.4 million. There were no changes in the component description, but estimated costs for project management were revised to better reflect PAU’s staffing and operating cost requirements. 1.7 Other significant changes 16. The changes in the scale and scope of the project responded to significant context changes outside the control of the Government and the Bank. Yemen went through two acute periods of armed conflicts in 2011 and 2015. In addition two of the major DPs for SEDGAP reduced their financing to the project. Interviewed stakeholders for this ICR noted that regional and sectoral priorities were shifting in donor countries and the 2011 crisis accelerated their final decision to withdraw. These context changes did not impact the high relevance of SEDGAP PDOs and interventions; at the contrary, they became more relevant. For example, the community-based interventions (such as school-community grants and conditional cash transfers) provided relevant support and incentives for learners, especially girls, to continue their studies in the 2011-2015 inter-crises period. 2. Key Factors Affecting Implementation and Outcomes 17. SEDGAP was a unique project as it attempted to address education and gender issues in an integrated way. This approach was much demanded and owned by the GoY. It was implemented successfully after initial difficulties and expected learning curves for a project targeting a complex—but needed—set of access and gender equity education goals. As an innovative integrated approach, it faced risks but also rewards. Impressively, it achieved most of its revised PDO targets in the face of extremely difficulties beyond the control of the implementing partners. This section analyzes the key factors and events that impacted the project cycle. 2.1 Project Preparation, Design and Quality at Entry 18. Overall, the preparation phase of the project was sound based on its assessment of quality and gender equity needs, alignment to national education policies, and facilitation of government ownership and commitment. Attentive to its background analysis—and emerging from the priorities outlined in the NGSES—the preparation team (i) linked the project design to evolving secondary education policies; (ii) facilitated a strong stakeholder dialogue at the central and governorate levels and with DPs; and (iii) sought strong Government commitment and donor alignment/harmonization. In addition, the 5 project design used lessons learned on community participation at the primary education level (from the Basic Education Development Project, BEDP), which were expanded to secondary education, especially in rural areas and for girls. 19. The preparation team aimed to keep the design of SEDGAP simple, while developing an integrated theory of change to achieve its quality and gender equity goals. The project appraisal report of February 18, 2008, developed a clear causal model and theory of change for the design of SEDGAP. Addressing the highly relevant education and gender equity needs for Yemen, it integrated supply interventions (school infrastructure and quality inputs for access in hard-to-reach rural areas) and demand support (school community grants and CCTs). Given the socio-cultural constraints to girls’ education in the country, this integrated approach was the most relevant to increase secondary education access in rural areas for females. Yet the project team was mindful of simplification. The pre-appraisal aide memoire of October 20-November 5, 2007 noted that the project design had been simplified to take into account: (i) the capacity issues related to implementation, managing and monitoring of the project; (ii) the system-wide and decentralized interventions at the governorate, district and school levels; and (iii) the support for long-term secondary education policy development. The table below shows the areas of complexity and the ways the design team sought to address them. Design Complexity Areas Simplification Approach by Design Team Need to combine supply (schools and Anchor supply and demand interventions within existing teachers) and demand (school community MOE programs and harmonize technical assistance with grants) interventions other donors. Need secondary education services for a Map each school to serve different communities and dispersed population in rural areas. support students’ mobility through transportation grants. Need to promote cultural and attitudinal Use community participation, hiring of female teachers, change in favor of girls’ education CCTs and other school grants to incentivize new behaviors towards girls’ education. Simplify delivery through an integrated MOE strategy (for basic and secondary education). Need for longer-term secondary education Include a “policy studies” component (#3) to provide on- policies going evidence and dialogue for secondary education reform. Initial implementation will require Extend implementation period from five to six years to learning curve mitigate associated risk of slow initial implementation. 20. The design of SEDGAP attempted to mitigate implementation risks, but overestimated the experience in donor harmonization and government capacity to manage pool funds. The project made use of existing implementation offices with deep experience in World Bank-supported projects, such as the Project Administration Unit (PAU) and the Public Works Project (PWP) office in order to cope with a few complex interventions that were new to the secondary education sector, such as the design of cost-effective secondary schools, the design of a new secondary education teacher training program, and the set up and training of new School Management Committees, SMC. As it was expected in context of fragility, design also embraced the potential benefits of harmonization of development partners within one sector wide program. Co-financing agencies (3 DPs: Dutch, DFID, and KfW) signed Memorandums of Understanding (MOUs) to formalize their contributions and responsibilities. Supporting towards the common PDOs under a SWAPs framework, the Netherlands (through the Embassy of the Kingdom of the Netherlands, EKN) and DFID would contribute funds through a Multi-Donor-Trust Fund (TF) administered by the World Bank (TF-94233). Germany (through KfW, GIZ and GtZ) were to provide parallel financial and technical support, especially to the quality component. Furthermore, the original aim was for all DPs to share leadership in organizing collective field missions and project supervision. As will be noted in the implementation section, next, some DPs withdrew from this alliance and the capacity of the MoE, PAU and 6 PWP to initially manage pooled funds was overestimated; as in retrospect, it proved to be more difficult than anticipated. 2.2 Implementation 21. SEDGAP confronted unexpected obstacles, notably the withdrawal of DPs financing pledges and two security and armed conflict crises. Yet the relevance of SEDGAP’s innovative design to achieve integrated education and gender equity objectives was upheld. The project achieved most of its development objectives in the midst of a detrimental financial and security context, in part, by proactively scaling down the project and targeting only five governorates. The project was launched in January 2009 and closed May 31, 2015 and its six years of operation can be described in three different phases: (i) a slow start up in 2009- 10; (ii) the 2011 security and political crisis and 2012 project restructuring; and (iii) the 2013-2015 accelerated progress (see Annex 11 on specific chronology). In addition, in 2015, the project closed in the midst of renewed armed conflict and IDA portfolio-wide-suspension in Yemen. The factors that impacted the project cycle within each of these phases are presented below. 22. Bank supervision was initially optimistic in 2009 but slow disbursements continued into 2010, resulting in downgrading the project implementation rating to ‘moderately unsatisfactory’ yet the team swiftly increased technical assistance to cope with the challenge. In 2009, SEDGAP disbursed less than US$200,000, yet the ISR rated its implementation as satisfactory pointing to the expected learning curve of the implementing agencies. Also, supervision reports noted as positive the Governmental efforts to build its alliances with, and capacity of the nine targeted governorates and their districts for the implementation of SEDGAP. The first year of operations also required extensive contact at the community level, especially to promote girls’ secondary education, which is traditionally not a socio-cultural priority in rural areas. In terms of disbursement rates, the initial delays were experienced due to the various reiterations of school standard designs for secondary education schools in line with the efficiency needs in rural areas (such as one school serving various communities). Also, the donor harmonization strategy to rotate DPs leadership was limited by the lack of tools and capacity by some DPs to lead the technical and/or fiduciary supervision of the project. The management of pooled funding also proved to be more difficult than anticipated due to its rigidity and complex fiduciary transaction for which substantive capacity building was required for PAU. In 2010, the project performance was downgraded to MU due to the slow start-up, against which MOE, PAU and PWP renewed efforts to accelerate implementation and the ISRs #4 and #5 noted increased external technical assistance to the local implementing agencies, and the completion of procurement processes for the first batch of 49 schools. 23. The political and security crisis of 2011 hampered the efforts of the MOE, PAU and DPs to revamp project implementation. The context of political instability and insecurity affected the entire country, including its education sector. Schools were closed, education delivery was interrupted, providers’ contracts were suspended, and the safety of students and teachers was compromised. During this time, the MOE and PAU also experienced various changes in leadership and staff, reducing the capacity to sustain project implementation and transparency (for example, the PAU director, financial manager and procurement officer resigned in 2011). The Netherlands and DFID also confirmed their decisions to withdraw their support which resulted in reduction of the financing to more than 50% of the original project costs. In light of the political and security environment in Yemen, most DPs suspended or withdrew disbursements. As of July 28, 2011, the World Bank also suspended disbursements for its entire IDA portfolio, consisting of 21 operations with $882 million in commitments. In addition, most DPs’ expatriate staff, including from the World Bank, were evacuated from Yemen. 24. SEDGAP implemented an emergency plan during 2011 to face the detrimental impact of the political and security crisis and the cancellation, reduction and suspension of donor financing. The task team, together with PAU/PWP/MoE made a proactive move to face the implementation obstacles emerging from the 2011 political and security crisis and withdrawal of support by two of the major DPs. 7 The table below presents key processes of the 2011 SEDGAP emergency plan. The crisis was especially unfortunate given the efforts during 2010 to complete all required school construction and training designs, and launch the demand based program (grants to SMCs). The emergency plan allowed the PAU and PWP to remain active –albeit understaffed—during the 2011 crisis and contractors were able to re-engage quickly once the emergency period ended. Project supervision was undertaken remotely from Washington DC and through “reverse missions” in Amman. KfW/GiZ played a critical role as one of the only DPs that remained in the country and continued to provide in-country technical support and financial flows to SEDGAP. In turn, the World Bank team facilitated various problem-solving workshops, bringing together different MOE departments in Amman (given the impossibility of Bank staff travel to Yemen). SEDGAP restructuring in 2012 upheld its project development objectives by scaling down project investments only within the first phase of five governorates. Project Emergency Management of Disbursement Suspension Key Process Strategy Planning for the Crisis The PAU and the World Bank prepared a one year projection of Period minimal activities that could be sustained with available cash flows in the Designated Account. The goal was to slow down investments during the suspension and be prepared to resume normal activities after one year Prioritizing Activities The priority was to maintain the demand-based program and continue During the Crisis to develop the quality component with the support of KfW/GiZ. Negotiating with Contractors Negotiate with contractors changes to the implementation timelines to avoid penalties and maintain positive relations for the recovery period. Harmonizing with Donors Coordinate with donors who remained in the country for direct support for Complementary Support to SEDGAP. KFW and GiZ remained active during the 2011 crisis and supported the program in the country through this difficult period. Staffing The PAU was downsized to only essential personnel. Reverse supervision and The World Bank supported SEDGAP through reverse missions in support from DP in Yemen Jordan and coordinated closely with KfW/GiZ staff remaining in Yemen. Scale down project while Maintain the important focus on quality and equity of the project, but maintaining original PDOs scale down to only five governorates, instead of the original nine. 25. After the 2011 crisis, SEDGAP’s innovative, integrated design (supply and demand-based investments) supported the post-crisis education needs of the targeted governorates and districts. Given a detrimental shift in the political and security context in Yemen—as well as withdrawal of approximately 50% of original project funding—the project was scaled down from nine to only the five governorates that had been selected for the first phase of implementation: Abyan, Hadramout, Ibb, Laheji, and Taiz. These represent different regions in southern and northern Yemen. All have been affected by the conflict and caught in the middle of the confrontations with various armed forces (such as Abyan in 2012 and Laheji in 2014). SEDGAP, however, continued to provide support to these provinces (with some limitations at time, such as inability to access Laheji after the 2014 armed confrontation).The preparatory activities before the 2011 crisis (completed school designs, training modules, signing of contracts, etc.) as well as the “emergency plan” put in place during the crisis, contributed to the successful revamping of SEDGAP activities in the post 2011 and inter-crises period. The integrated design supplying school infrastructure and supporting education demand from communities proved to be even more relevant in the midst of the conflict. For example, school-community grants and CCTs were especially useful to support the permanence of boys and girls in targeted schools after the 2011 crisis. Transportation grants, for example, reached 4,192 girls and 3,949 boys. In 2013 and 2014 the school construction program and the training interventions showed remarkable acceleration (43 schools built and 14,349 teachers trained). Disbursements increased from 6% in July 2012 to 58% in May 2014, and almost 80% at the end of the 8 same year. By 2015, overall disbursements reached 95% and the project closed successfully on May 31, 2015, unfortunately in the midst of a new armed conflict period in Yemen. 26. In the post-2011 crisis period, donor harmonization improved and integrated technical support to the MOE, PAU and PWP increased. KFW/GiZ and the World Bank worked together diligently during this last phase of implementation. Germany led support to the quality teaching and learning component and provided complementary in-country follow up. The Government of Netherlands left approximately 50% of its original Trust Fund support, in the amount of US$6.62 million. During the post-2011 crisis, these funds were utilized first and by early 2014 were fully disbursed as shown on the Table G on the data sheet. The Mid-term review of February 2014 captured these advances and provided guidelines to continue the accelerated implementation and a DP harmonized approach (mainly the World Bank, GiZ and KfW). This improved DP alliance helped monitor the PDO indicators until project completion and accelerated school construction, teacher training, and disbursement of grants to schools and communities. Ultimately, the innovative approach to integrate supply and demand support investments proved correct to improve the PDOs related to more equitable access and retention of girls in secondary education in the targeted governorates. 27. SEDGAP was better able to navigate the 2015 renewal of the armed conflict in Yemen, which again had a detrimental impact on the education sector. In September 2014, the Houthis (known as Ansarullah in Yemen) led a takeover of Sana’a, which led to a full military confrontation since March 2015, including the involvement of a coalition of ten countries led by Saudi Arabia. Street bombings, rationed fuel and electricity, and armed conflict escalated across the country. Together with the lack of food, water and fuel, the war has highly affected the education sector and left more than 1.8 million students (62% of enrollments) out of school during most of the second semester period of the 2014/15 school year. By January 23, 2015, as a precautionary measure, the Bank again suspended all missions to Yemen and relocated its staff to Cairo and Washington, DC, followed by the IDA disbursement suspension announced on March 11, 2015. An advisor to PAU contracted by KfW remained in Yemen and continued to provide direct support to SEDGAP, while keeping a constant communication with the World Bank task team. Despite such an extremely difficult situation—but through the efforts of the MOE, PAU, PWP and staff at the sub-national levels (Governorates and Districts)—by May 31st, 2015, the final project activities were accomplished. 28. SEDGAP’s innovative and integrated interventions achieved the expected outcomes, in the face of extreme challenges and risks. As with any innovative approach, risks actualized during project implementation (including unanticipated ones such as the 2011 and 2015 crises, and the changes in DP financing priorities). However, potential rewards were also high. SEDGAP showed that an integrated supply-demand approach was better able to navigate difficult and rapidly changing contexts. It also provided some lessons learned for future programs that may also face contexts of adversity, noted in section 6, Lessons Learned. Annex 10 also summarizes feedback from MOE and SEDGAP stakeholders interviewed for this ICR noting the new set of education demands in the midst of the present emergency. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 29. M&E Design. The M&E strategy was well designed, identifying the key impact and intermediate indicators of the program, as well as baselines for the original nine governorates to benefit from the project. The M&E mechanisms for SEDGAP were adequate to measure the PDOs and intermediate outcome indicators. It used education statistics at the governorate level—enrollment, attendance, dropout and gender indicators—as well as data from project implementation at the field level, including the school construction program, the school and community grants, and the teacher training program. The 2012 “level two” restructuring adjusted PDO targets given the scale down of the project. To better capture project impact, the 2008 baselines for only the five remaining governorates were noted in 2014. 9 30. M&E Implementation. M&E implementation required coordination with the implementation units, including the PWP, the CCT unit, and the Girls Education Department. The PWP especially supported the infrastructure program for various Bank-supported projects including the BEDP (until it closed on 12/31/2012), BEDP II and SEDGAP. This provided an institutionalized strategy to integrate information for all school infrastructure contracts and their implementation. The implementation of the M&E strategy was affected at times by PAU staff rotation and limited access to sources of data, especially from the governorates most affected by the political instability and insecurity. After the 2011 crisis, the Government and the Bank agreed to hire a third party monitoring (TPM) agency to support field based M&E activities, especially to monitor specific indicators and progress for school construction, and to follow up on CCT activities.. The third party monitoring by Management Systems International (MSI) reported findings to the MOE, PAU, PWP and the World Bank. This was done in three cycles: (i) January-June 2013; (ii) February-June 2014; and (iii) January-June 2015. MSI monitoring included the governorates with great instability, such as Abyan, and Lahj. The monitoring team within the PAU and PWP also coordinated with the third party monitoring agency and with the Sanaa-based advisor from KfW. 31. M&E Utilization. The M&E system was able to provide sufficient data to make equity-based decisions in selecting the five most vulnerable governorates, especially those with the lowest retention rates for boy and girls. The M&E system also allowed the PAU and PWP to closely monitor the intermediate outcome indicators, as well as implementation and disbursement progress. After the project restructuring of April 2012, the M&E system included additional indicators and targets to monitor the provision of science and computer laboratories and library books. During the periods of acute crisis in Yemen (in 2011 and 2015), the third party monitoring strategy was especially useful in maintaining monitoring of activities in the field. TPM brought not only M&E expertise, but also security strategies to maintain their staff and project beneficiaries safe during supervisions. The M&E system, in general, provided adequate data to assess achievements by the end of the project even in a difficult implementation environment (see Annex 2). 2.4 Safeguard and Fiduciary Compliance Environmental Safeguards 32. The project environmental category was rated B (OP4.01), since no new schools in new locations were to be built. The original project design defined the intervention as upgrading schools facilities which included extension, rehabilitation and replacement of existing schools. The PAD states the following: The civil works investments in rural secondary schools identified central schools for rehabilitation and expansion, which could serve various nearby communities. Laboratories, latrines and replacement of other physical spaces took place in targeted schools. The Environmental Management Plan (EMP) for SEDGAP was prepared, based on lessons learned with similar IDA supported investments at the basic education level. This EMP was followed during implementation and provided clear guidelines to avoid environmental violations. Rehabilitating or expanding infrastructure in existing locations required extensive consulting with communities, as well as deciding which community would serve as central hubs for secondary education services, addressing the cost-efficiency requirements for the more sparse population density in rural areas. The integrated approach to implement education infrastructure contracts within the PWP provided an efficient way to provide support to one government agency working across different Bank- financed projects. Social Safeguards and Stakeholder Consultation 33. The project was assessed for cultural and attitudinal obstacles and assets toward the education of girls in Yemen. It provided a clear diagnosis on the crucial role of community participation, especially to increase equitable access and learning in the rural areas. It pointed to the already developed community participation experience of the education sector, especially at the basic education level and in the provision 10 of CCTs. The assessment recommended continuous awareness-raising efforts at the community level, to continue to promote secondary education and the participation of girls (ref. PAD Annex 10). Community involvement was also part of project design, as the consultation on education needs with stakeholders and communities has become a standard practice within World Bank operations. SEDGAP complied with these social safeguards, involving multiple stakeholders at the central, governorate, and community levels in diagnosing secondary education needs, especially in rural areas and for girls. Financial Management 34. Implementation. The Project Management Unit of the Public Works Project (PWP) and PAU (MOE) maintained adequate financial management arrangements, which were assessed as Moderately Satisfactory during the project life. The arrangements included (i) qualified staffing; (ii) well-functioning automated accounting system following the cash basis of accounting; (iii) acceptable FM manual; (iv) monthly reconciliation; and (v) timely financial reporting. The PMU and PAU maintained segregated Designated Accounts at the Central Bank of Yemen for the related IDA fund (44010) and used the traditional SOEs as the main disbursement method along to the Direct Payments method. During the project life, both implementing agencies were responsible for reporting and documenting all project expenditures based on Interim Financial Reports (IFR). All withdrawal applications were processed with no major obstacles. All project audit reports were submitted in a timely manner to the Bank throughout the project life, except for the last submitted 2014 report which, due to the war in Yemen, was delayed for a week time. The final audit report for the year 2015 (including the grace period) is expected to be submitted by November 30, 2015. 35. Ineligible expenditure: The auditor has identified, in a management letter issued for the year 2013, monthly payments made to the members of the Steering Committee (civil servants). The auditor clarified that fixed payments to civil servants are not in compliance with the financing agreement. This type of expense is not part of the expenses set eligible in the agreed definitions. Further, the fixed payments made to the steering committee members were discussed during the September 2014 mission. The PAU was advised by the Bank team to refund a total amount of USD 88,800 to IDA and other DPs. Due to the Bank’s suspension of disbursements in January 2015 which was applied to the whole portfolio in Yemen, the letter requesting the refund of ineligible expenditure from the Government of Yemen was put on hold, and therefore, the refund is currently pending formal communication with the Government of Yemen. 36. Disbursement. As of September 30, 2015, the client connection shows that the project disbursed a total amount of XDR 12.756 million (equivalent to USD 19.125 million), with a percentage of disbursement of 99.97% from the total Grant financing allocation of XDR 12.76 million. Procurement 37. Implementation. SEDGAP included procurement for works, goods, consultants and non- consulting services. Civil work activities were implemented using the PWP capacity and databases developed with IDA technical assistance. Goods procured included office and target school furniture, equipment and other educational materials. Standard bidding documents used were those in line with previous IDA financed projects, especially BEDP. Bank procedures were also utilized for consultant services. Procurement processes were rated moderately satisfactory throughout the project, given implementation delays experienced. In addition to prior review supervision carried out from the IDA offices, post reviews were conducted as well; they were also rated moderately satisfactory. Towards the end of 2010, SEDGAP began to accelerate its implementation through the tenders for the expansion and/or rehabilitation of 50 schools. The implementation of these contracts was delayed during the 2011 crisis but the MoE and PAU were able to negotiate with selected firms the postponement of activities until 2012. These negotiations facilitated the fast -resumption of the school construction program after the crisis. 11 38. Capacity Building. The World Bank procurement staff also provided close and constant technical assistance to their counterparts within the PAU and PWP teams, including training on the preparation of TORs and the review of Bank procurement guidelines. This exemplifies the dual role of the Bank’s procurement staff in terms of both borrower capacity building and fiduciary assurance. 2.5 Post-completion Operation/Next Phase 39. There are currently no plans for a follow-up operation specific to Yemen’s secondary education sector. Despite the current crisis, the Bank has maintained an on-going education sector dialogue, including the preparation of a review of education decentralization lessons learned in federal countries to support the Post-2011 Crisis National Dialogue in Yemen and an agreement to transfer from a unitary state consisting of 21 governorates and 333 districts, to a federal state made up of six regions. Further, a year before the closure of SEDGAP, the MoE proactively established a taskforce to discuss potential new project to support secondary education sector, however, with the onset of the 2015 armed conflict, however, the country is back to a state of emergency and the work was put on hold. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Ratings: Overall: High 40. Objectives. The development objectives of SEDGAP continue to be highly relevant for Yemen and its education sector. SEDGAP development objectives, indicators and education interventions were gender-sensitive. SEDGAP was able to help boys and girls stay in school in spite of a difficult local context. Quality education, another important SEDGAP objective, continues to be demanded in the country, and will require a focus not only on academic achievement and productive skills, but also on positive values, psychosocial wellbeing and social cohesion. SEDGAP’s focus on equity is even more relevant today as it can contribute to social cohesion across the regions, communities and people of Yemen. A quality and equitable secondary education will be especially important for youth that are exposed to the on-going conflict today and who will need to support the peaceful reconstruction of their country. 41. Design. The design of SEDGAP is also highly relevant as it shows that access, quality and gender equity can be addressed in an integrated way. The post-2011 crisis showed that a combination of both supply and demand education interventions can better support education communities through emergencies. While schools were being rebuilt and teachers trained in the governorates targeted by the project, the school and community grants provided needed resources to help students stay in school. School attendance provided protection and a sense of normalcy during the most acute periods of the armed conflict in Yemen. The combination of supply and demand education investments will be highly relevant after the present crisis, as well. School-community alliances remain highly relevant for follow-up programs to SEDGAP. 42. Implementation. Implementation arrangements during project design (such as using existing units within MOE, providing a longer initial implementation period) as well as proactive strategies once implementation was underway—including SEDGAP’s emergency response plan in 2011—are still highly relevant today. Moreover, they are lessons learned for future education programs, especially in the fragile context of the country. 12 3.2 Achievement of Project Development Objectives Ratings: Before 2012 Restructuring: Unsatisfactory After 2012 Restructuring: Moderately Satisfactory Overall: Moderately Satisfactory 43. The original PDO targets for the originally nine targeted governorates were in effect for 5% of disbursements (from 2009-2011). Following the “Level Two” restructuring of PDO targets for only five targeted governorates and related adjustments made on PDO indicators, 95% of IDA was disbursed (2012- 2015). For this reason, the assessment of PDO achievement will be separated into two sections, one for the original PDO targets in nine governorates and one for the revised targets in five governorates. 44. Unsatisfactory achievement of PDOs targets prior to the 2012 restructuring. Given the slow start-up of the project in 2009, the withdrawal of donor funding in 2010, and the 2011 crisis, the project was not able to implement the proposed interventions at the community and school levels to affect the PDOs in the nine targeted governorates. By July 2012, it had only disbursed 6% of its proposed investments (mainly for the preparatory and technical activities for the larger investments). After the 2011 crisis, SEDGAP reduced its scope from nine to five governorates and adjusted its PDO targets, baseline and indicators. The post-2012 restructuring period is assessed separately next. 45. Satisfactory achievement of PDOs with some moderate shortcoming after the 2012 restructuring. The PDO indicators, set to measure impact in terms of gender parity, school retention, and learning outcomes in Mathematics were mostly met. The two moderate shortcomings were (i) the achievement of boy’s retention slightly below the expected target, and (ii) the inability to conduct the final mathematic assessment in 2015 due to the armed conflict (but the first test and mid-term test showed an improvement trend). • Surpassed Expected Target: The Gender Parity Index baseline increased from 0.43 to 0.63, surpassing its expected end of project target of 0.60. • Improved But Did Not Meet Expected Target: The grade 10-12 male students’ retention rate baseline increased from adjusted from 73% to 78.7% but did not meet its expected end of project target of 81%. • Surpassed Expected Target: The grade 10-12 female students’ retention rate baseline increased from 78.3% to 84.5%, surpassing its expected end of project target of 84.3%. • Surpassed Expected Target by Mid-Term but Final Measure Not Possible Due to 2015 Armed Conflict: At the time of the Mid-Term Evaluation the mathematics assessment scores increased from a baseline of 19.3% to 31.2%, surpassing the 24% target that was expected for the mid-term. However, at the end of the project, in 2015, the end-line survey on the learning assessment was not conducted due to the on-going armed conflict. The expected end-of-project target was 40 % points. 46. SEDGAP integrated supply and demand-based interventions contributed to more gender equity in secondary education access and retention in targeted governorates and districts. The intermediate outcome indicators monitored the delivery of school construction, training of teachers, school and community grants, and hiring of female teachers. These interventions were proposed as crucial to the result chain to impact PDO targets. Towards the end-of-project, these services had been delivered only with some minor shortcomings in teacher training (14,349 trainees as opposed to the 17,267 target proposed). The key result chain interventions delivered in order to achieve the PDOs included: • Out of the restructured target of 44 schools to benefit from civil works, by the end of the project 43 were completed. 13 • The target of 5,200 students benefiting from the different types of grants was surpassed, reaching 8,141 (this is even above the original project target—before restructuring—of 6,800 students). • 14,349 teachers were trained in subject modules, 17% short of the 17,267 target. • 89 new female teachers were hired, in line with the adjusted targets. 3.3 Efficiency Rating: Substantial 47. SEDGAP was considered at appraisal and even more today to be an economically efficient intervention. When SEDGAP was prepared in 2008, the Economic Internal Rate of Return (EIRR) for the project was estimated as 13.7%. Using actual data, the cost and benefits streams were updated for the targets of the restructured project. The updated benefit of the project, as estimated, still far exceeds its actual costs, with an EIRR of 17.0%. This indicates that the economic viability of the project is maintained. In addition, as SEDGAP has contributed to the education of more than 14,587 female students in the five targeted governorates, there are several associated social benefits and externalities with such increased in education for females. Annex 3 shows the updated economic and financial analysis, based on today’s assumptions and targets achieved by the Project. 3.4 Justification of Overall Outcome Rating Rating: Before 2012 Restructuring: Unsatisfactory After 2012 Restructuring: Moderately Satisfactory Overall: Moderately Satisfactory 48. The achievement of the PDO is rated ‘”Unsatisfactory” before and “Moderately Satisfactory” after the 2012 project restructuring, which changed the scope and scale of the project targeting only five governorates, instead of the original nine. Given the parameters for evaluation for relevance, efficacy and efficiency for 94% of disbursements, the overall rating is proposed as “moderately satisfactory. It is also important to note that this was an innovative project that assumed certain risks that paid off in terms of achievement of its integrated education and gender equity objectives. The slow startup of project and implementation constraints can be mostly attributed to factor out of the control of the GoY and the Bank (conflict crises and changes in DP financing priorities). The following table shows the split and overall rating before and after the 2012 restructuring. Against Original PDO Against Revised PDO Overall Targets Targets Rating Unsatisfactory Moderately Satisfactory - Rating Value 2 4 - % disbursed 6 94 100 pre/post 2012 Restructuring Weighted Value 0.24 3.76 4 Final Rating - - Moderately Satisfactory 49. The integrated ratings on relevance, efficacy and efficiency is show on the table below. Project’s objectives and design were and still are today highly relevant to Yemen and its education sector. The efficacy rating reflects the more gender-sensitive access and retention, as well as improved learning environment, attributable to the project. Outcomes were largely achieved despite all the unforeseeable incidents of the political and security crises and the changes in DPs’ financial commitments. Efficacy can be further rated by component, where component 1 (supply-demand) was highly successful, while quality 14 (teacher training) had more modest achievements. Efficiency is rated satisfactory at appraisal and even more today, based on the project outcomes in spite of the reduced financing during the life of the project. As the table below notes, most of the outcomes were achieved mainly after the 2012 restructuring as noted above. The efforts of the GoY, implementing agencies, and the Bank were especially exemplar in monitoring the risks and adapt the project in order to uphold the PDOs, which were finally achieved. 3 Period Relevance Achievement of PDO – Efficacy Efficiency Overall Rating (Objectives /Design) Comp 1 Comp 2 Comp 3 Comp 4 Summary Pre- Restruc High Modest Low Low Modest Low Modest Unsatisfactory turing Post- Substan- Substan- Moderately Restruc High High Modest Cancelled Substantial tial tial Satisfactory turing 3.5 Overarching Themes, Other Outcomes and Impacts 50. Based on the interviews conducted with SEDGAP stakeholders (MOE, PAU, PWP, DPs), there are additional outcome-related themes worth noting, including two unexpected outcomes: (a) Poverty and Gender Impact/Aspects 51. Project achieved its objectives in one of the poorest countries in the world, targeting a group traditionally excluded from secondary education: rural girls. The project impacted equitable access of girls and their retention in secondary education, especially in the rural areas. Of the approximately population of 25 million in Yemen, 65% are in rural areas and almost half of the rural population live under the poverty line. In this context, the project provided support to more than 15,000 rural girls to attend secondary education. There is a multiplier effect of educating girls; those that complete primary and secondary education are more likely to earn income, have fewer unwanted pregnancies and break the cycle of poverty. 4 (b) Institutional Change/Strengthening 52. The implementing agencies for SEDGAP became resilient in the face of political and security crisis. Beyond the traditional learning curve of working with various donors and new pooled funding mechanisms, the implementing agencies for the project faced the constraints of a growing armed conflict (for example: working without electricity and fuel; facing daily threats to their physical and psychological wellbeing; and managing a volatile and unstable political situation). Resiliently, they were able to navigate these adversities by being flexible and proactive. They defined an emergency plan, and were ready to revamp implementation and achieve the PDOs immediately in the post-crisis periods. Donors that remained in Yemen also learned to integrate their support—based on their comparative advantages—not only by pooling funds (at times difficult) but by integrating efforts to the common SEDGAP objectives. 3 The MU rating for implementation for the last ISR of the project, reflects the overall Bank portfolio in Yemen, after disbursement suspension during the 2015 armed conflict. It does not reflect the overall achievements of the project. The other rating on the PDOs achievement, which reflects more project-specific evaluation, was rated MS. 4 The World Bank’s 2012 World Development Report on Gender Equality and Development 15 (c) Unintended Outcomes and Impact 53. The lower than expected retention rates of boys. The PDO targets show that boys achieved a retention rate of 78.7%, compared to 84.5% for girls. In-depth analysis of the gap between boys’ and girls’ retention rates was not possible due to lack of access to relevant data and inability for the ICR team to visit the field as per the travel restriction to the country. However, the context of crisis may have affected boys’ attendance in ways that need to be identified. 54. The greater demand for community and school grants during the post-2011-crisis period. The intermediate indicator for students benefited by SMC grants surpassed target expectations, reaching 8,141 students above the target of 5,200 (156% achievement). This shows the active role of community-school alliances in the 2011 post-crisis to address education needs at the community level, supported by the SMC grants. It also epitomizes community and school alliances as a protective factor for students during emergencies, bringing teachers and parents together to plan critical inputs such as availability of female teachers and safe transportation for girls. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 55. Given the on-going political and security crisis in Yemen, all missions to the country have been suspended. There was no safe possibility of inviting project beneficiaries to a workshop or visiting communities; even reverse missions, or missions to a third-country, involved prohibitive security risks given the current security situation. However, available evidence collected by the MoE and TPM confirm that the package of demand-side interventions (CCT, SCG, support for MFCs) were highly effective in raising awareness towards girls education and to actually change behavior of the parents, to send and retain girls at secondary schools. With strong request from the Bank team, the final round of TPM conducted CCT beneficiary interviews and the anecdotal evidence points that parents, teachers and students perceived CCT intervention effective and useful in continuing their education. MoE also proactively conducted field visits while the Bank team was not able. The video that recorded voices of SCG and CCT beneficiaries provide a strong basis for the impact of demand-side interventions in improving female students’ attendance and performance at school. 4. Assessment of Risk to Development Outcome Rating: High 56. The project restructuring of 2012 revised the project risks, stating as ‘high’ those related to the political and security turmoil, political interference, and delays in counterpart funding. This rating has been confirmed given the 2015 acute conflict experienced in Yemen. Today the risk to the sustainability of project outcomes is high as the armed conflict can lead to a setback of program outcomes, such as potential damage to school infrastructure investments and lack of funding for salaries of the newly hired female teachers. The risks to the development outcomes of SEDGAP call for an institutional approach by the GoY, DPs and the World Bank to provide relevant support in the midst of the current crisis. This requires not only traditional access and quality programs (building schools, training teachers, providing community grants), but also alternative and flexible education services that are sensitive to risks associated with armed conflict and its impact on education actors (see Annex 10 with further examples of relevant education emergency services). 16 5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues) 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 57. The moderately satisfactory quality at entry rating combines assessments for (i) soundness and background analysis and relevant design; (ii) organizational and implementation arrangements, and (iii) co- financier and partners arrangements and pooled financing. The design factors are assessed as ‘satisfactory’ based on evidence of the Bank’s support for a sound diagnosis and secondary education policy discussions in the country, a systemic and rigorous approach to define the theory of change of the project, and an innovative integration of supply and demand education interventions for classroom and community change. The organizational factors assessed as “moderately satisfactory” are based on the efforts to simplify local organizational arrangements by institutionalizing project implementation within existing organizations and teams managing other Bank-financed projects (BEEP and BEDP), which benefited from the operational lessons learned from previous Bank financed programs. The donor harmonization factors, although highly relevant in a fragile context, can be assessed ‘moderately unsatisfactory’ given the overestimation of donor harmonization readiness especially on the aspect of rotating leadership for technical and operational support and handling a rigid pooled financing. Although outside of the control of the design teams, the assumptions of a well-functioning DP sharing of supervision and pooled funding mechanisms did not hold during implementation. (b) Quality of Supervision Rating: Moderately Satisfactory 58. The moderately satisfactory rating of project supervision is substantiated by the commitment and quality support for the implementation of the project during extremely difficult times, while noting some initial moderate shortcomings. In general, the positive factors affecting supervision include : (i) close Bank relations with the Government and LEGs including, at times, Bank staff based in the country; (ii) responsive and proactive support during the 2011 crisis, including the preparation of an emergency plan to sustain essential activities in the face of withdrawn, reduced or suspended disbursements; (iii) strong collaboration with KfW/GiZ, that enabled continuation of key activities such as CCT even during the crisis,; and (iv) proactive actions taken to restructure the project several times to swiftly adopt to constantly changing operating environment, with effort to maintain the integrated investments to achieve the PDOs; and (v) During the 2011 and 2015 crises, the Bank’s task team proactively provided close guidance to PAU and PWP to handle fiduciary and project management issues. Also it is commendable the effective transition and flexible support across the three task team leaders that supported SEDGAP during its six years of implementation. However, the slow disbursement of funds during the first two years of implementation was considered “normal” with over-optimistic satisfactory ratings and the fact that the task team was not possible to disburse IDA until later in the project. Given those shortcomings, quality of supervision is rated Moderately Satisfactory. (c) Justification of Rating for Overall Bank Performance Rating: Overall Moderately Satisfactory 59. The moderately satisfactory overall rating is in line with the above dimensions ratings for quality at entry and supervision. This rating reflects the commitment and efforts of both project preparation and supervision teams, while noting the slow disbursements during 2009 and 2010. World Bank response was especially proactive during the two political and insecurity crises that affected Yemen in 2011 and 2015, as well as changes in donor contribution announced during the second year of project implementation (2010). The Bank country management unit provided mechanism (TPM: third party monitoring), to ensure M&E 17 on the ground. After the 2011 crisis, upon lifting of the disbursement suspension, the task team and Bank management promptly resumed operation based on the ceaseless guidance and support provided to the PAU and PWP during the suspension period. The education teams in the MENA region are capturing the lessons learned from this and other projects to provide more relevant education services in the face of crises (see Annex 10). 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 60. Government performance is rated moderately satisfactory based on the following reasons: In general, the GoY demonstrated strong ownership and commitment to SEDGAP, providing a relevant institutional framework and supporting a decentralized and participatory processes throughout the project life. The GoY’s commitment in general was strong and illustrated by the extreme efforts exerted after the 2011 crisis to revamp the implementation of SEDGAP and achieve its PDOs. Government efforts are specially recognized in (i) providing the policy framework (NGES) to guide the design and implementation of SEDGAP; (ii) assigning a seasoned MoE senior advisor who is fully dedicated to the project; (iii) avoiding independent and isolated projects, by bringing them all under the umbrella of the PAU and PWP; (iv) providing effective leadership and coordination across MOE departments (teacher training, CCTs, Girls Education, etc.) and subnational entities; and (v) supporting essential project activities during the 2011 crisis to revamp project progress in the post-crisis period. However, early in the project and especially towards the end, Government counterpart funding was not made available for outstanding payments mainly for civil works activities. The GoY is still unable to contribute the funds due to severe fiscal crisis stemming from the long-lasting armed conflict and dissolvent of industries. In addition, the GoY was not able to hire 89 female teachers as civil servant, hence risking their status and sustainability of the project impact. Based on those deficiencies, the government performance is rated MS. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 61. The PAU, PWP and sub-national implementing stakeholders are rated satisfactory, while noting some minor shortcoming mostly outside their control. In general, PAU and PWP showed a resilient and stable performance for project implementation during difficult situations. They maintained good working relations across the different Government implementation agencies, technical departments, DPs and other key stakeholders, and provided proactive support during the 2011 crisis through the elaboration and implementation of an emergency plan. Both PAU and PWP sought a timely resolution of implementation issues, including the delays of school designs and site selection, training modules, and declining or suspended DP financing. The implementing agencies did suffer turn-over of its staff, especially in the financial management and procurement teams. This required significant amount of time and effort to build capacity of the new staff, for which the PAU made the most of support provided from the Bank fiduciary team and quickly adopted to the changes. PWP also demonstrated its strong performance throughout the project and successfully carried out civil works activities despite all challenges such as price changes and inability to go into certain sites due to insecurity. Both agencies provided timely progress reports and M&E from the field while the Bank team was not able to conduct field supervision during the conflict. In general, the implementing agencies were those mostly responsible for the successful revamping of project activities after the 2011 crisis and closing the project successfully in the midst of the 2015 armed conflict. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 62. The overall Borrower’s moderately satisfactory rating reflects the above dimensions ratings on Government and implementing agencies’ performance. The project completed satisfactorily most of its 18 restructured targets. The rating recognizes a successful revamping of project implementation and progress, although there were major delays during initial project implementation. Moreover, the two most crucial issues that affected project implementation—the 2011 and 2015 crises and the withdrawal of DP financial commitments—were outside the control of the Borrower. The long lasting crisis severely affected the fiscal capacity of the government which caused the insufficient counterpart funding towards the end of the project. Recognizing the continued effort exerted to carry out the project under such a challenging environment, overall borrower performance is rated MS. 6. Lessons Learned 63. The findings from this project provide lessons of broader significance and recommendations. Those considered more relevant are listed below. For Project Design 64. The design, implementation and evaluation of innovative education program must account for the dynamism of risks but also potential rewards. SEDGAP had ambitious objectives. It innovatively addressed the need of girls’ education in Yemen, and the constraints especially in rural areas. Although its focus was on education of girls in rural areas, it did not isolate boys. It considered “gender” through an inclusive and integrated approach. It supplied education services, but also recognized that gender constraints are rooted in socio-cultural norms. It built alliances across national and subnational governments, and with development partners. Identified and unexpected risks actualized. Disbursement was slow during the first two years of implementation, and donor harmonization did not hold as initially envisaged. In addition two acute periods of armed conflict occurred. In spite of these obstacles, however, the innovative program achieved its objectives and many were surpassed. The lesson learned is that innovation brings both risks and rewards in a dynamic way. Project designers, implementers and evaluators must account for this dynamism and judge the success of a program by its final outcomes, and not by the low and high implementation periods, or the number of risks actualized or overcome. 65. Preparing simplified program designs is possible to integrate access, quality and gender equity goals. SEDGAP balanced a complex set of interventions with strategies for simplified implementation to achieve quality learning and behavioral changes (to support education for girls in rural areas). Such complex set of objectives require both supply (access and quality inputs: curricular reforms, teacher training, pedagogical materials) and demand-side interventions (incentives, school-community grants, participatory processes). SEDGAP shows that a relevant project design must start from a serious analysis of the theories of change leading to the desired outcomes. Having identified the interventions to achieve desired outcomes, simplifying their delivery given capacity and context is crucial. In the example of SEDGAP, the quality and equity PDOs required both supply and demand interventions for their achievement, and designers found a viable way to provide them. 66. Components that support policy dialogue can provide a platform for longer term education strategies and for on-time adaptation to changing contexts. The alignment of SEDGAP to the NGSES policies was critical. Policy alignment of education projects not only supports a collective development vision, but it also benefits from broader sector guidance and full country ownership. The design of SEDGAP strategically included a “policy studies” component (original component #3) to support such on- going policy dialogue for evidence-based secondary education reforms. Although this component was cancelled during the 2012 restructuring, it could have helped as well in a dialogue on how to respond to new needs posed by the 2011 and 2015 crises. A derived recommendations from this experience, for education programs in contexts of crises, is to keep a balance between the immediate needs and the long- term strategic processes. This requires an on-going policy dialogue. 19 For Project Operations 67. Project implementation units integrated within their line ministries avoid silos and promote MOE leadership and stakeholders’ coordination. Institutionalized implementing units such as PAU and PWP upheld the ownership and leadership of ministerial authorities. Although PAU and PWP are not physically or legally housed in the MoE, yet both entities have maintained close relationship with MoE and played pivotal role in liaison with other DPs and LEGs. PAU in particular, has a long history and deep experience of working together with MoE and was able to address implementation issues swiftly on behalf of the MoE. Such institutionalized implementation units also support coordination among multiple stakeholders (national, sub-national and international) through formal arrangements—such as preparation of action plans, monitoring reports, cooperation protocols, and signing of MOUs. Institutionalized rather than parallel project implementation units should be promoted for development projects. 68. Procurement proficiency prior to project launch can contribute to mitigate delays traditionally experienced during the initial months of project implementation. The preparation of new projects in Yemen can benefit from standardized procurement introduction workshops prior to project launch. This propaedeutic training will support the staff of country partners to understand and operationalize the procurement procedures included within the respective credit or loan agreements. Formal procurement procedures training for officials, prior to project implementation, will mitigate the risks of any lengthy and unnecessary delays after project launch. 69. Donor harmonization and pooling funding also require feasibility assessments. The implementation of SEDGAP shows the challenges in the harmonization of development partners within one sector wide program. The ambitious goal of shared leadership across DPs to support the technical, operational, and fiduciary supervision needs of SEDGAP was not easily achieved. The complex and rigid pooled financing mechanism often was the source of certain delays in project implementation due to its time-consuming processes of agreeing on IFRs as well as pro-rata payment transactions. The most relevant approach to donor harmonization should be defined based on each country context and the existing DPs relations and capacities. When pooled funding and rotating leadership is not possible, other alternatives such as in-kind support and parallel financing should be considered. 70. Third Party Monitoring (TPM) is a relevant approach to support on-going supervision of projects in both stable and unstable contexts. The objective of the Yemen Monitoring Program contract with MSI was to provide data to verify the technical, social and environmental safeguards of contracts and their implementation mostly civil works activities. In the final round of TPM, the vendor managed to include assessment of CCT by conducting beneficiary interviews, which provided useful set of information in evaluation of the CCT. Overall, the TPM turned out to be a useful mechanisms especially in insecure contexts where Bank staff cannot be mobilized for field-based supervision. For Gender-Equity and Gender-Sensitive Projects 71. Education policies and programming can commit to gender-related objectives, interventions and resources. SEDGAP has shown that education and gender-equity goals can be achieved. This requires attention to both educational investments (school infrastructure, teachers, and training) but also to interventions that support family and community behaviors that support female education. Specifically, SEDGAP supported the participation of parents in school management committees, entrusting parents with supportive roles in secondary schools for adolescent girls (selecting and hiring female teachers, managing transportation grants, etc.) CCTs provided an additional incentive, especially for poor families, helping to defray household costs for education of girls. Finally, female teachers provided a positive role model and, since they were hired by the school and community, a trusted adult. These integrated interventions are particularly important in community cultures where conservative norms have traditionally being resistant to secondary and tertiary education for women. Education policies must first commit to these dual objective 20 (education-gender) and then define relevant programs, which as SEDGAP shows, require investments in both education services as well as family and community support. 72. Gender-sensitive intervention during crises are important because male and female students may be affected in different ways. SEDGAP has shown that gender-based education investments can better address the differential challenges boys and girls face. These gender differences are not fully understood, especially in the conflict-affected context of Yemen. For girls issues related to safety, need of female teachers as mentors, and confronting practices such as early marriage are critical. For boys, forced recruitment, violence extremisms and the need to support the economic needs of their families may be obstacles in conflict-affected contexts. SEDGAP findings also suggest that more flexible and alternative education services are needed in crises prone contexts, especially bringing together families, schools and communities to support both education delivery and protection of learners. DPs, including the World Bank, need to develop relevant emergency education approaches to continue to provide relevant and gender- sensitive education support during crises and through the transition to on-going development. Some suggestions are offered in Annex 10. 5 For Education Response to Emergencies/Crises 73. Community participation and school-community alliances are crucial during crises and stable times. SEDGAP’s demand based interventions valued community participation in education. Community-school alliances were crucial to the goals of gender equity, which were achieved by the project. With more complex operations, including education response during crises, community-school alliances for decision making and school management is crucial for services that seek to provide protection, wellbeing support and learning for students in at-risk situations. This is important for the following reasons: (i) communities are the first respondents at times of emergency and better place to protect children and youth; (ii) they can provide insights on the feasibility of implementation arrangements; (iii) communities that understand the full package of project services, can support on-time monitoring of their delivery; (iv) communities informed on the details of project design are better able to manage those activities under their responsibility; and (iv) knowledge and participation, in general, contribute to ownership and collective commitment toward project outcomes. Future education programs, including those for emergency response, should build on the community involvement and school-based management efforts to-date. 74. Any consideration of suspension of disbursements by donors during a crisis needs a proactive Emergency Implementation Plans to sustain operations. The decision to suspend credit/grant disbursements should be preceded by the preparation of alternative and transition plans for those projects to be affected. This would require not only the proactivity of task teams, but also the guidelines and support of broader Bank systems. During the 2011 crisis, SEDGAP was able to put together an emergency plan that included identification of essential activities, harmonization with DPs that remained in the country, and negotiation with active contractors. Such a plan should be a requirement for Bank-financed projects prior to suspensions. 75. Education in crisis situations are no longer rare and the World Bank needs relevant education sector approaches to respond during such times. As emergency situations become chronic, the boundaries between humanitarian and development aid have blurred. For example, Yemen 2011 crisis ignited again in 2015 with a more forceful armed conflict and negative impact on the education sector. Development organizations such as the World Bank increasingly are called to support education systems and education communities in the midst of crises, for which more standardized and institutional approaches will be needed. Annex 10 provides some lessons learned of such approaches, including (i) alternative 5 The following source presents additional advice for gender-sensitive education assessments, policies and programs in contexts of adversity such as conflict and violence: Reyes, J., Kelcey, J. and Diaz-Varela, A. (2013). Transformative Resilience Guide: Gender, Violence and Education. 21 education services, when using traditional schools is not possible; (ii) protection of students, teachers and other education actors—both in terms of their physical and socio-emotional wellbeing; (iii) safeguarding fallbacks in education quality, planning for relevant content delivery and assessments to students in the midst of crisis; and (iv) reconstruction and on-going education policy and investment developments. Disengagement should not be an option, but rather identifying relevant ways for development institutions, such as the World Bank, to support the education sector in times of crisis. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 76. The MOE and its PAU have submitted an implementation review which is included in Annex 5. In addition, MOE, PAU and PWP staff interviews contributed to the understanding of project implementation and lessons learned. These have been incorporated into this report. (b) Co-financiers 77. The development partners and SEDGAP stakeholders that provided feedback for this ICR noted the following lessons learned regarding aid harmonization and shared DP leadership: 78. DP leadership rotation for SEDGAP was constrained by limited experience in harmonization. The project started under a strong donor harmonization and alignment framework; however, not all donors had the system infrastructure, tools and teams to support the technical and/or fiduciary needs of the project. The expectation that different donors would rotate leading joint supervision missions was not possible in practice. In general, finding an effective governance model (leadership, communication, collective decision making, etc.) to support donor harmonization became a complex task. Through the initial implementation phase, DPs conducted two joint supervision missions (November 2009 and October 2010). DP changing aid priorities towards Yemen impacted pledged financial commitments to SEDGAP, reducing over 50% of the original project planned financing. This had a major negative impact on the project. 79. The support of DPs remaining in Yemen during the Emergency situation, such as from KfW/GiZ, was crucial to SEDGAP coping with the 2015 crisis. While it was understandable that the 2015 political crisis and insecurity prompted an evacuation of DP staff and suspension of disbursements to Yemen, it is important to note the crucial role played by KfW/GiZ which remained in Yemen. This DP provided a crucial technical and financial bridge during the emergency. Such a role may point to the increasing need for development agencies to have more relevant approaches to supporting partners during emergencies, including and especially in the education sector. (c) Other partners and stakeholders 80. No comments received. 22 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Component 1:Upgrading of school 33.65 31.93 95% facilities Component 2: Enhancing the quality 14.10 11.64 83% of service delivery Component 3: Project management 2.43 2.19 90% and dropped activities Total Baseline Cost 50.18 45.76 91% Physical Contingencies 0.04 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 50.22 45.76 91% Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.00 0.00 .00 Total Financing Required 50.22 45.76 91% (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing Appraisal (USD millions) (USD millions) Borrower 8.70 5.60 64% UK: British Department for International Parallel 2.40 2.33 97% Development (DFID) Financing GERMANY: German Technical Parallel 3.00 3.00 100% Assistance Corporation (GTZ) Financing International Development Association 20.00 19.12 96% (IDA) GERMANY: KREDITANSTALT FUR Parallel 9.50 9.09 96% WIEDERAUFBAU (KFW) Financing NETHERLANDS: Min. of Foreign Co-financing 6.60 6.62 100% Affairs / Min. of Dev. Coop. 23 Annex 2. PDO Impact and Outputs by Component PDO Indicators 1. SEDGAP intended to measure its PDO through four indicators of improved gender equity, learning quality, and efficiency of secondary education: a gender parity index, retention rates for boys in grades 10- 12, retention rates for girls in grades 10-12, and improved learning outcomes in mathematics. The baselines for these indicators were collected across the original nine governorates which were the direct project beneficiaries. However, the scope of the project was reduced to five governorates when the project scaled down due to the 2011 political crisis and resulting insecurity, and withdrawal of financing from two of the four major development partners for SEDGAP (the Netherlands and DFID). Therefore, the project impact is assessed based on the baselines and targets for the five governorates targeted after the 2011 crisis and project restructure of 2012: Abyan, Hadramout, Ibb, Lahej, and Taiz. 2. As show in table 2.1, based on the five beneficiary governorates, the project surpassed two of its four targets. The actual outcomes for each PDO indicator are: • The Gender Parity Index baseline increased from 0.43 to 0.63, above the expected target of 0.60. • The grade 10-12 male students’ retention rate increased from a baseline of 73% to 78.7, slightly below the expected target of 81% • The grade 10-12 female students’ retention rate increased from a baseline of 78.3% to 84.5%, above the expected target of 84.3%. • The mathematics assessment scores increased from a baseline of 19.3 in 2009 to 31.2 surpassing the 24% expected mid-target at this time. However, at the end of the project, in 2015, the end-line survey on the learning assessment was not conducted due to the on-going armed conflict. The expected end-of-project target was 40 points. 3. Although all four PDO indicators show a positive trend with both the Gender Parity Index and Female Students Ratios showing above expected results, as a whole, the PDO indicators show mixed results. It is particularly important to further assess the retention rates of male students, which actually showed a decline from a rate of 79.8% in 2013 to a final measured rate, for this project, of 78.7% in 2014. This decline may infer that boys in the participating governorates may have been more at risk of leaving school in times of conflict. This is a hypothesis to be further explored. When the security situation permits, it would be important to explore the gender-impact of armed conflict in Yemen. This is important since improvements in the Gender Parity Index are sensitive to both increased access and retention of girls, increased access and retention of boys and girls, or increased access and retention of girls and decline for boys. Intermediate Outputs by Component 4. As for the intermediate outcome indicators, the project met or surpassed its indicators for component 1: school expansion, grants to students, hiring of female teachers and building of laboratories/ libraries. However, the achievement indicators for the quality component fell slight short of expected targets, for example only 14,349 teachers were trained from the 17,267 targeted, and the pedagogy certification training was not scaled up after it was designed (only the training of trainers was conducted). The classroom observation study is not yet complete (and is being supported by KfW/GiZ). 5. In May 2014—to more accurately assess project progress and achievements—the targets for the outcome indicators in components 1 and 2 were revised to reflect SEDGAP activities in only five remaining governorates. The end-of-project targets reflect these revisions: 24 6. Component 1: Improving equity and reducing gender gaps through demand- and supply-side interventions. This component upgraded and furnished school facilities, including extensions and rehabilitations; financed the provision of learning equipment and learning materials appropriate to the existing curriculum and technical assistance to develop the Operations Manual for the school community grant scheme; and provided, managed and supervised community school grants and CCTs to girls in grades 4-12 and to boys in grades 10-12. It also financed eligible contractual female staff in co-educational schools. Its achievements were monitored by the following indicators and achieved targets: • Just Below Expected Target: 43 schools received civil works, out of 44 targeted; • Surpassed Expected Target: 8,141 students benefited from project provided grants, out of 5,200 targeted; • Met Expected Target: 89 female teachers financed out of 89 targeted. 7. Component 2: Enhancing the Quality of Service Delivery. This component was simplified to focus only on provision of learning equipment and material to selected schools; and capacity building of teachers, principals and supervisors in selected schools to improve teaching and learning practices in the classroom. Its achievements were monitored by the following indicators and achieved targets. • Met Expected Target. 50 schools were equipped with science laboratories of 50 schools targeted; • Met Expected Target. 50 schools were equipped with computer laboratories of 50 schools targeted; • Met Expected Target: 50 schools were equipped with libraries of 50 schools targeted; • Below Expected Target: 14,349 teachers were trained in subject-based modules out of 17,267 targeted • Well Below Expected Target. 0 teachers were trained and certified in pedagogy from 1,500 targeted • Well Below Expected Target. 0 schools were assessed on performance of teachers from a target of 50 schools and expected 70% score. 8. Although the project had mixed results regarding its expected outcomes by component, overall, the results are impressive given the context of crises in which it was implemented. Table 2.2 shows the impressive revamping of the project in the post-2011 crisis period. 25 TABLE 2.1 SEDGAP CUMULATIVE TARGETS BY PDO AND INTERMEDIATE OUTCOME INDICATORS Project Development Objective (PDO): The objective of the project is to support the Government of Yemen’s program for improving gender equity, quality, and efficiency of secondary education in selected districts with a particular focus on girls in rural areas. Revised Project Development Objective: No change C=Continue Cumulative Target Values** R=Revised Actu Data Responsibili Core PDO Level Results al Frequenc Source/ Unit of Measure YR 1 YR 2 YR3 YR4 YR5 YR6 ty for Data Indicators* N=New Base y Methodolog 2009 FY 2010 FY 2011 FY 2012 FY 2013 FY 2014 FY Collection line y P Indicator One: R GPI based on the 1 Target calculation x 0.43 n/a 0.45 0.50 0.52 0.59 0.60 Annual MOE AES MOE TO Gender parity index methods of PAD in the intervention page 30 governorates 2006/2007 Actual AES data incomplete 0.45 0.50 0.52 0.59 0.63 (District level missing) P Indicator Two: R % 73.0 n/a 73.8 74.4 78.8 79.8 81.0 Annual MOE AES MOE TO 2 Target G10 to G12 male 2006/2007 retention rate in the Actual AES data intervention districts incomplete 73.8 74.4 78.8 79.8 78.7 (District level missing) P Indicator Three: R % 3 Target 78.3 n/a n/a n/a n/a n/a 84.3 Annual MOE AES MOE TO G10 to G12 female 2006/2007 retention rate in the Actual AES data intervention districts incomplete 82.2 79.6 80.6 83.3 84.5 (District level missing) P Indicator Four: C % 4 Average score in GIZ GIZ in GIZ 19.3 Mathematics and Target n/a 27.6% n/a n/a 40% Independe collaboration Independent % Science Test for G11 nt Study with MOE Study students 6 Independent Study is Actual n/a n/a n/a 31.17% currently funded by KfW and it is in 6Standard test conducted through GIZ survey on a sample of 24 schools in 3 governorates: Ibb, Lahej and Taiz. * Three handed over extended secondary schools have minor remarks that do not affect their operation. 26 advance stage. Results will be reported as soon as the study is completed by ERDC. INTERMEDIATE RESULTS Intermediate Result (Component One): C=Continue Cumulative Target Values** R=Revised Data Responsibili Frequenc Source/ Actu ty for Data N=New y Methodolog Collection Core Intermediate al y Unit of Measure Results Indicators Basel ine YR 1 YR 2 YR3 YR4 YR5 YR6 2009 FY 2010 FY 2011 FY 2012 FY 2013 FY 2014 FY 1. Civil works programs C Number Progress 1 (including civil Target 0 0 44 44 44 44 Annual PWP/ PAU Report works, furniture, and 19 43 schools were equipment) schools handed over. One completed in 44 Actual handed school was cancelled schools over due to site problems & at the beginning of 0 0 0 0 Another the project. * set of8 schools under delivery 1. 5,200 students C Number 0 Progress 2 attending the project Target 0 1,000 3,000 4,100 5,000 5,200 Annual PAU Report schools using the transportation scheme provided by Actual the SCG 0 0 4,969 6,189 7,570 8,141 1. 89 female staff C Number 0 General Progress 3 financed through the Target 0 24 60 80 85 89 Annual Education Report SCG Sector/ PAU Actual 0 0 88 89 89 89 Revised Intermediate Result (Component Two): 27 C=Continue Cumulative Target Values** Data R=Revised Source/ Responsibilit Intermediate Results Unit of Actual Frequen YR 1 YR 2 YR3 YR4 YR5 YR6 y for Data Core Indicators Measure Baseline cy Methodolo Collection 2009 FY 2010 FY 2011 FY 2012 FY 2013 FY 2014 FY gy N=New 2.1 No of schools equipped C Progress Training Number 0 0 10 20 30 40 50 Annual with Science Labs Target Report Sector/ PAU 50 schools Actual equipped with Physics labs, Biology labs, Chemistry Tender Under equipment and 0 0 prepared, Delivery materials, audio evaluated visual. Math and Geography posters and maps were cancelled due to project suspension. 2.2 No of schools equipped C Number 0 General Progress with Computer Labs Target 0 10 20 30 40 50 Annual Education Report Sector/ PAU Tender Under 50 Schools Actual documents advertise equipped with prepared ment. Computer Labs. 2.3 No of schools equipped C Number 0 General Progress with Libraries Target 0 10 20 30 40 50 Annual Education Report Sector/ PAU 31 50 Schools Actual delivered equipped with & Libraries. 6 19 schools under advertise ment 2.4 Number of teachers C Number Curriculum trained in subject 0 250 1,100 5,760 11,500 17,267 17,267 and Target Inspection 28 modules in 5 Sector and governorates COME Training program Actual kits were developed, printed and master trainers and trainers were trained. The scale- up program was suspended due to 1,100 1,450 1,450 14,349 unavailability of government contribution funds. As soon as these funds are available, the training will be conducted and the results will be reported. C=Continue Cumulative Target Values** Data Source/ Responsibilit Intermediate Results R=Revised Unit of Actual YR 1 YR 2 YR3 YR4 YR5 YR6 Frequen y for Data Core Indicators Measure Baseline cy Methodolo Collection N=New 2009 FY 2010 FY 2011 FY 2012 FY 2013 FY 2014 FY gy 2.5 C Number 0 0 0 500 1,000 1,500 1,500 MOE Target Annually Decree Curriculum Number of teachers from Sector project district schools trained on pedagogy 0 Training program training certification Actual kits were 0 0 NOL request developed, will be sent printed and to IDA on master trainers April and and trainers were activity will trained. The be scale-up program 0 completed was suspended this due to academic unavailability of year for 1500 government teachers contribution funds. As soon as these funds are available, the training will be 29 conducted and the results will be reported. 2.6 Average performance of C % 54.8 60% n/a n/a n/a n/a 70% Curriculum teachers from the 50 and 2nd, 5th Independen schools in classroom Target Inspection year t Study observation test. Sector and COME n/a n/a n/a n/a Independent Actual Study is currently funded by KfW and it is in advance stage. 69% Results will be reported as soon as the study is completed by ERDC. The 5 intervention governorates are Abyan, Hadramout (Coast and Valley), Ibb, Lahej, and Taiz. Formula for calculating PDO indicators: P.1:= Number of female new intake/ male new intake in project districts; P.2: = number of non-repeaters in G12 as share of G10 students 2 years before (methodology defined on the SEDGAP PAD page30). 30 Annex 3. Economic and Financial Analysis (including assumptions in the analysis) 1. The original cost-benefit analysis for SEDGAP was updated to assess how costs and returns assumptions have hold, and to provide an updated Economic Internal Rate of Return (EIRR) for the restructured project. The scope and scale of the project was reduced from nine to five governorates. Actual data was used to analyze the economic analysis and the final project had much lower costs and its benefits still far exceed these costs. The updated EIRR is 17.0%, compared to the initial appraisal EIRR of 13.7%. The updated actual costs and benefits indicate the economic viability of the completed project. 2. Since the project was scale down to about 50% of the original planned investments, one leading factor in the improved EIRR is the reduction of input costs. Yet retention of both boys and girls improved in the targeted governorates. Retention of boys improved by 5.7 percentage points and of girls by 6.2 percentage points, leading to expected improvement in completion rates. The updated review of benefits and cost is presented next. Benefits 3. The updated benefits stream is still quantified by estimating: (i) reduced supply side/input cost of producing secondary education graduates, and (ii) lower opportunity cost incurred by students due to fewer years invested in completing secondary education. This analysis uses the same methodology retained for appraisal adjusting the assumptions to reflect the reduced scope of the restructuring. Although the secondary education cycle in Yemen lasts three years, due to high repetition, dropout rates and low pass rates in the secondary school examination results, the number of years invested per secondary graduate is substantially higher. Due to the documented PDO targets reached in the targeted governorates, the years invested per student are expected to improve. 4. The restructured project maintained most of the supply- and demand-side interventions, as well as the quality improvement measures initially proposed for SEDGAP. The theory of change of these interventions pointed to the need to increase secondary education services in rural areas, but with incentives for families’ and communities’ to change their traditional views of education for girls. These included school grants to hire female teachers and for transportation, as well as CCTs benefiting more than 14,000 female students. Indeed, the effectiveness of combining supply and demand-based interventions led to improved retention (reduced repetition and dropout rates). Math scores also improved in the remaining five governorates from a baseline of 19.3 to 31.2 during the mid-term evaluation (a final evaluation in 2015 was not possible due to the on-going armed conflict). Improved retention and student learning performance, thereby, it is expected to reduce the number of years required to produce secondary education graduates. 5. At baseline, the average number of years to complete secondary education were estimated to be 4.44 for boys and 4.12 for girls in project districts. By achieving the PDO targets, the years invested per student are expected to improve to 4.2 (boys) and 3.9 (girls) in target districts by the end of the 6 year implementation period of the project, and to 4.1 and 3.8 respectively by the end of 20th year since project commencement. Retention improved from 73.0% to 78.7% for boys and from 78.3% to 84.5% for girls in program districts and the pass rate was noted as 82.8%, based on national average. The difference in the retention of boys and girls needs to be explored further, especially in how conflict may pressure boys and girls in different ways to dropout from school. 31 6. The annual per student cost incurred by the GoY on secondary education is estimated as YR70,371 (US$368) in 2005/06.5 Assuming that this cost remains unchanged, the savings generated from reduced number of years per graduate are estimated at US$59 million over 20 years as indicated in Table 1 (or present value equivalent of US$18.1 million). Students’ foregone earnings (annually averaging US$1,007 for boys and US$506 for girls) will also be reduced by the reduction in the number of years invested per graduate. This will generate savings of US$128.1 million over 20 years as indicated in Table 1 (or present value of US$39.7 million). Costs 7. The cost stream is estimated on the basis of (A) project investment costs, (B) recurrent costs incurred from the project, which include: (i) maintenance costs for civil works, furniture, equipment, and learning materials; (ii) continuation of female staff recruited through the project; and (iii) continued incentives through school community grants; and (C) foregone earnings of newly enrolled students. 8. Given the restructure of SEDGAP in 2012, the estimated project cost has been nearly halved from an initial cost of US$103.4 million to US$47.2 million. Of this new cost, IDA proposed to invest US$20 million, the DPs proposed to invest US$18.5 million, and the GoY proposed to invest US$8.7 million. The project implementation rate remained 6 years between 2009 and 2014. In addition to the project investment, recurrent costs are calculated as part of the project cost stream. The approximate average annual recurrent cost resulting from the project during the next 14 years after project end (until year 2020) is US$3.3 million per year (year 7 to year 20). See table 3.1, below. EIRR Simulation Result 9. Table 1 shows the summary of both cost and benefit streams. The EIRR of the project will be 17.0%, and the Net Present Value will be US$11.7 million using a discount rate of 10%. 32 Table 3.1: Summary of Cost and Benefit Streams Summary of Cost and Benefit Streams and EIRR Years since project started (real USD '000) Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 20 Program NPV at 10% Total discount rate BENEFIT STREAM Total savings generated 86 177 274 936 1,631 2,385 2,585 5,133 58,977 18,123 Total saved foregone earning for all graduates 202 410 631 2,137 3,698 5,368 5,774 10,897 12,174 39,748 Total Benefits 288 587 905 3,073 5,329 7,753 8,359 16,030 187,150 57,871 COST STREAM Project Investment Cost 23 1,649 7,582 23,821 12,241 1867 47,209 29,107 Recurrent Cost 731 731 731 3,299 3,568 50,537 15,543 Students' opportunity cost 3 3 32 242 259 473 266 163 3,702 1,523 Total Costs 3 26 1,681 8,555 24,811 13,445 5,432 3,731 101,447 46,173 Net Benefits 285 561 (776) (5,482) (19,482) (5,692) 2,927 12,299 85,704 11,698 IRR 17.0% NPV at discount rate of 10% 11,698 Assumptions Annual per student cost is USD 368 (2004) Annual maintenance cost is calculated as: (1) for civil works - 0.5% during first 10 years and 1.5% during next 10 years of total value of the building; (2) furniture - 5% of installment cost; (3) 10% of equipments; (4) 10% of learning materials Salary includes that of for female teachers hired at intervention schools (assuming $1200 per year) School grants continue in transportation allowance and cash transfer (approx. 5,000 students per year at secondary level) Total saved unit cost and foregone earning for all graduates are calculated based on the number of years invested per graduate Benefits are calculated separately for program schools (Component 1 target schools) and other schools in the program districts (Comp. 2 only schools) 33 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Ayesha Vawda Education Specialist MNSHD TTL Abdulrahman Al-Sharjabi Operations Officer MNSHD Operational Support Ghada Younes Country Lawyer LEGAM Legal Asma Al-Hanshali Program Assistant MNCYE ACS Support Financial Moad Al-Rubaidi Financial Specialist MNAFM Management Mikael Mengesha Sr. Procurement Specialist MNAPR Procurement Supervision/ICR Ayesha Vawda Education Specialist MNSHD 1st TTL Abdulrahman Al-Sharjabi Consultant 2nd TTL Mahmoud Gamal El Din Sr. Operations Officer MNSHD 3rd TTL Tomomi Miyajima Education Specialist MNSHD 4th TTL Danielle Malek / Nikolai Country Lawyer LEGAM Legal Soubbotin Asma Al-Hanshali Program Assistant MNCYE ACS Support Financial Moad Al-Rubaidi Sr. Financial Specialist MNAFM Management Shivendra Kumar Consultant MNAPR Procurement Samira Al-Harithi Procurement Analyst MNAPC Procurement Christina D. Wright Operations Officer MNSHD Operations Khalid Moheyddeen Consultant MNDYE M&E (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) USD Thousands (including Stage of Project Cycle No. of staff weeks travel and consultant costs) Lending FY05 6 19 FY06 11 49 FY07 15 54 FY08 45 229 Total: 77 351 Supervision/ICR 34 FY08 1 6 FY09 14 60 FY10 8 31 FY11 23 104 FY12 20 109 FY13 40 185 FY14 50 179 FY15 70 239 FY16 2 17 Total: 228 930 35 Annex 5. Beneficiary Survey Results N/A 36 Annex 6. Stakeholder Workshop Report and Results N/A 37 Annex 7. Summary of Borrower's ICR The MOE and PAU conducted their own ICR for SEDGAP, with the purpose of reviewing and assessing the main operations undertaken by the project, evaluating project implementation, impact and execution from the beneficiaries' point of view. This annex provides a summary of their findings, as per their report dated August 2015. Main results: • SEDGAP has a strong and clear impact on supporting the GOY’s program for improving gender equity, quality, and efficiency of secondary education in selected districts with a particular focus on girls’ education in rural areas. • Implementation arrangements were appropriate; having the same unit (PAU) administering BEDP2 and SEDGAP helped in easing the implementation of the two projects. • Both the intermediate and the PDO level indicators have been almost achieved. This leads to an overall conclusion that the PDO has been achieved. • Both the GoY and SEDGAP donors were committed to SEDGAP and its objectives, despite some very small exceptions; their performance was satisfactory and resulted in overcoming the implementation obstacles of the project. • SEDGAP built 934 rooms (including classrooms, administration rooms and different services and facilities) in 43 targeted schools; each school has been provided with basic furniture and accessories. • Fifty targeted schools received life skills sources for domestic economy and handicrafts such as sewing machines, cooking machines and refrigerators in addition to the tools and equipment needed for drawing, fine arts, sports, and musical instruments. • About 600 schools in the 5 targeted governorates received sources of teaching and learning. • There was a fiscal deficit of an estimated US$1 million in the civil works allocations as reported by PWP resulting from both currency exchange differences, and non-payment of the government's contribution. As a result, PWP was not able to pay the remaining dues to contractors and suppliers; this is an unresolved issue to date. There was a similar deficit in relation to important training activities that were suspended due to unavailability of funds. • The school community incentives (transportation incentive, CCT and FTAs) helped support the relationship between the school and the community; consequently student enrolment increased, particularly for girls. • SEDGAP training programs have been implemented with high level of professionalism and have an impact on improving the skills of the targeted trainees. • There were slight differences, but not statistically significant, in favor of the performance of science and math teachers in the 50 targeted schools with integrated interventions for other schools in the same governorate. The presence of statistically significant differences were in favor of teachers who were trained in SEDGAP governorates in comparison to the teachers who did not take training from other governorates. 38 Following are the main conclusions and lessons learned presented in four themes: 1- Conclusions and lessons' learned in relation to project in general • Active participation of MoE sectors and offices at all levels in SEDGAP activities in relation to the administrative arrangements for the project implementation contributed significantly to the success of the project activities. • SEDGAP created positive school environment in the targeted schools that made them model schools. This happened through developing school facilities in an integrated way and providing training to school management, teachers and supervisors as well as providing a mechanism to provide requirements for the implementation of these programs. The project also provided schools with learning and teaching resources as well as stationery and operational budget that contributed to the success of the project. • The project contributed to improving the process of teaching and learning through a range of various and integrated interventions. • There was noticeable delay in getting the TORs by MoE sectors and on the reports of accepting, commenting or rejecting consultant works; the later situation caused problems on paying some consultant payments. 2- Upgrading and furnishing school facilities • Limiting civil work to 5 governorates and 43 schools helped to easily achieve this intervention's objective; but, there is a real need for such interventions in all governorates. • The school environment has become conducive and supportive of teaching and learning in targeted schools because of the development of school facilities in an integrated manner, and providing schools with materials and resources for teaching and learning. • There was noticeable delay in getting extended schools' building completed and furnished; so that the use of the schools had to be delayed, particularly science and computer labs. The main reasons of this delay are related to the country unusual situation of unrests, government fiscal deficits, the weakness of the soil in some schools, which necessitated testing sites and re-adjust the designs, and the lack of commitment by some contractors in the completion deadlines. These targets met but with considerable delays through implementation. • Most MoE engineers could not get involved in civil work supervision and follow up due to unavailability of funds allocated for their transportation. 3- Providing school-community grants • The participatory process between the community/family and the school in addition to the interaction of parent councils increased community awareness of the importance of education and increased access to secondary education (particularly girls) and lowered dropout rates. • Female teacher assistants (FTAs) encouraged and motivated families to keep their daughters in the co-educational schools, where most of the students, school administration and teachers are male; they assisted in providing security factors to families to keep and sustain their daughters in the co- educational schools, and eliminated fear and embarrassment to keep them in the schools. However, there was a lack of taking the necessary arrangements to ensure the sustainability of the FTAs after the project is closed as it was adopted by the Basic Education Development Project (BEDP) for rural female teachers. • Life skills manual developed in a good manner, but most schools need specialists in this field. 39 4- Capacity building to improve teaching and learning practices • SEDGAP developed training methodologies and mechanisms according to modern methods that included proper preparation for implementation and evaluation tools for teachers before and after training. The direct training was supplemented with Technical Support through follow-up field visits for teachers and providing immediate feedback after each visit. • Master trainers and supervisors were a cornerstone of the success of the training activities in SEDGAP; they were selected objectively and carefully. They have been trained on various training materials for programs target groups; and represent elite scientific, practical and technical resources that can benefit future training programs. • Strengthening the technical role of the school principals via training programs enabled them to make classroom visits for teachers, observe their performance and provide immediate feedback after each visit to develop their performance. • Training of teachers in the school without need to leave their school, (so-called internal training) did not affect the course of the educational process. • Professional development meetings planned within follow-up visits positively contributed in enhancing teacher classroom practices. Supervision visits played a major role in assessing and supporting the teachers and students performance. • Good lesson planning, using teaching techniques that make the student the center of teaching and learning, implementing various activities and doing homework have a great role in improving students' attainment. • Acquiring the concepts of active learning and their implementation besides using technology increased student learning. • Encouraging the students' motivation increased their self-confidence and belief that learning is achieved with persistence and practice; so as to be able to present their work confidently. • Low efficiency of teachers and secretaries of laboratories in the use of some modern equipment that was provided to the targeted schools. Following are general recommendations that can help in improving teaching and learning and in any future projects that will be presented in four groups. 1. Recommendations in relation to the project in general • Additional projects are needed to be designed for targeting more governorates, districts and schools. • In future projects, implementation plans have to observe time milestones carefully; procurement of school furniture plans should ensure that school furniture is delivered as soon as the school is completed. • There is a need to provide the rest of the government contribution to cover the shortfall in civil works costs that have already been implemented and training activities that have been prepared for their implementation, but not implemented due to the inability of the government to contribution. • Stop leak of teachers from teaching to technical and administrative works via improving the standard of living and professional teacher status; so as to have the teaching profession become an attractive profession for divergent initiators. • Consultants should be assigned to develop TORs and participate in the analysis of bid offers. Specific periods should be outlined for MoE sectors to accept Consultants reports, provided comments/ feedback or reject the reports. • There is a need to improve living situation of the PAU employees so as to keep the trained staff from leaving PAU to other jobs. 40 2. Upgrading and furnishing school facilities • More attention should be paid to selecting appropriate school sites and contractors for implementing school buildings in accordance to clear criteria. • Funds should be allocated for the engineers' transportation of the MOE project sector, Governorates Education Offices (GEOs) and District Education Offices (DEOs); so as to be involved in civil work supervision and follow up to help in overcoming and solving any obstacles related to the community or/and existing school. • Strict adherence to the implementation planned activities for civil works in future projects, so that the supply of furniture reaches schools immediately after completing the buildings. • Provide facilities for sports such as basketball and table tennis as an outlet for students to practice some extra-curricular activities. 3. Providing school-community grants • Get benefit from lessons learned of School Community Grants (SCGs) for the purpose of development, expansion and sustainability of the pilot, especially female staff and transportation incentives. • Employ SCGs in other districts and governorates with increasing the number of schools in general. • Find a solution to recruit female staff after the project is closed, or include them within the Rural Female Teacher Scheme under BEDPII. 4. Capacity building to improve teaching and learning practices • Expand teacher training to include teachers of other subjects who are not trained in this project via following the mechanisms and methods that have been developed and used in this project with a focus on training to concentrate on 'student-centered learning' and active learning. • Continue professional development meetings for nearby schools to exchange experiences. • Make use of the training methods and techniques used within SEDGAP in any future project. • Train trainers on training skills. • Utilize the SBS model in any future project interventions; so as to maximize the benefits of the project interventions at the school level and contribute to raising the quality of Education. 41 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders N/A 42 Annex 9. List of Supporting Documents Brookings (January 2013). Improving the Quality of Basic Education for the Future Youth of Yemen Post Arab Spring. Washington, DC: Global Economy and Development at Brookings. Government of Yemen (PAU and PWP) SEDGAP Progress reports Management Systems International (MIS), World Bank Yemen Monitoring Program. Third Party Monitoring Reports I, II, and III MDTF Financing Agreements with DFID and Kingdom of Netherlands (EkN) Memorandum of Suspension of Disbursements of All IDA Grants and Credits of March 11, 2015/Yemen Suspension of Disbursements Guidance Note, March 11, 2015 MoU with Development Partners about pooled financing and agreement on the lead DP and mission schemes Project Appraisal Document for the Yemen Secondary Education and Girls Access Project IDA Financing Agreement for the Yemen Secondary Education and Girls Access Project SEDGAP Annual Progress Report 2014, Education Development Project Administration Unit, Monitoring and Evaluation Department, Ministry of Education of Yemen SEDGAP Mid-term-review reports SEDGAP Restructuring papers: April 2012, May 11, 2014 and October 31, 2014 UNICEF (February 2007). Accelerating Girl’s Education in Yemen: Rethinking Policies in Teacher’s Recruitment and School Distribution. New York City: United Nations Children’s Fund. World Bank, Implementation Status and Results (ISR) No. 1 to 14 (2008-2015). World Bank and Government of Yemen, Aide Memoires (2006-20014) 43 Annex 10. Education Impact of the Political Crisis and Armed Conflict in Yemen and Education in Emergencies Lessons Learned 1. The 2011 and 2015 political crises and violence experienced in Yemen had a significant detrimental impact on the education sector, from suspension of education services to physical harm and trauma in students and education staff. As part of this ICR, the authors documented feedback provided by Yemeni stakeholders regarding the impact of the crisis on education, in order to provide some recommendations on the type of education support needed in such a context of emergency. This annex summarizes these initial findings. Education Impact of the Yemen Political Crisis and Armed Conflict 2. During the 2011 crisis, but especially during the ongoing armed conflict that started in January 2015, education services have been suspended in many areas of the country. Feedback from stakeholders in the Yemen shared the following detrimental conditions of the education system: (i) destroyed schools and schools occupied by armed groups; (ii) limited protection for students and other education actors in the most affected communities; (iii) suspended education services (no mechanisms for on-going and flexible service provision); (iv) lack of mechanisms to evaluate learning during the year and provide accreditation to students; and (v) limited resources for staff salaries and other administrative support. Due to the ongoing political and insecurity crises in Yemen, previously gained education improvements are being reversed. In addition, the education sector has to be ever more mindful of the security of students, teachers and other education personnel. Present Emergency Education Needs 3. The 2015 armed conflict in Yemen has affected previous gains in access and learning. Street bombings, rationed fuel and electricity, and escalated armed conflict has prevented more than 1.8 million students (62% of enrollments) to attend the second semester period of the 2014/15 school year. Except for 9th and 12th grade students who took their final exam late in August 2014, all other grades in affected areas were not able to conduct learning assessments. In lieu, a ministerial decree allowed the use of the students’ grades during the first semester of 2014 as their end-of-year results. Schools and other education infrastructure (libraries, laboratories, etc.) have been destroyed, and in general advances in student learning have suffered setbacks. Especially there is no detail information on how adolescents and youth are being affected by the armed conflict, although some reports point to forced recruitment for boys and increased gender-based violence affecting girls (see for example UN Report of the Secretary-General to the Security Council, A/69/926-S/2015/409, of 5 June 2015). 4. Given the above scenarios, stakeholders interviewed for this ICR mentioned the type of donor support urgently needed today. Education emergency needs include: (i) conducting fast assessment of the education sector needs in the midst of this crisis; (ii) funds to support education in emergency activities; and (iii) technical assistance to plan and support activities to transition the education sector from a state of emergency to on-going development once the crisis subsides. The table below lists the type of education risks noted by Yemeni education actors and examples of specific education needs manifested in the country today. 44 Type of Education Emergency Risks Specific Education Emergency Needs Manifested in Noted By Yemeni Stakeholders Yemen in 2015 Consulted Access and Completion Detrimental • 62% of students unable to attend schools and Impact complete the annual curriculum • Future compounded access and dropout effects • Lack of alternative and flexible education delivery mechanisms Compounded Risks Affecting the Most • Poor, rural and traditional excluded most exposed Vulnerable • Increasing students and teachers’ safety concerns • Students afraid to walk to and from school • Schools destroyed and occupied by Internal Destruction of Physical Infrastructure Displaced Population (IDPs) and armed groups (old and new investments) • Destruction of education materials, laboratories and school furniture Quality Learning Set Back • No adjustment to curricular content and delivery • No alternative learning assessments for student progression • Trauma and psychological impact across education actors Lessons Learned on Education in Emergencies Response 5. International education development response in emergencies must address new education challenges, in addition to the traditional access and learning needs. The experience today of education services in contexts of emergency points to various ways that development partners can help education communities in distress. These include flexible education services, community and school alliances, protective measures for students and education staff, and even negotiations and approaches to protect schools from attacks. Emergency education support is not only needed during acute or humanitarian crisis, but also in the longer term. Conflict, environmental crisis, and other forms of emergencies have become chronic and their underlying sources and mechanisms continue over time. For example in Yemen, although this ICR notes two different crises – in 2011 and 2015—the reality may be that the underlying sources of conflict could have existed and continued all along. Given the longer-term effect of crises today, relevant education development response is needed before, during and after such contexts of adversity. 6. The case of Yemen clearly exemplifies how crisis situations, such as political conflict and insecurity, directly affect the education sector in the longer term. Feedback provided on the education impact of the 2011 and 2015 crises pointed to disruption in education access and completion across all education levels. Those most affected were students in already vulnerable situations, especially in rural areas where conflict intensified. Just as crises affects education settings, education services can support mitigate some of the risks noted in Yemen. For example, alternative and flexible education services can support student learning, protection and wellbeing during both stable and emergency times. Community- school alliances, as well, are supportive mechanisms for access, retention, and learning, but also for protection, socio-emotional wellbeing and community cohesion. In contexts prone to crises, development actors must support not only quality but relevant education services. The table below provides examples of relevant Education in Emergencies responses that can be supported by development partners working in this and other crisis situations. 45 Education in Emergencies (EiE) Lessons Learned EiE Needs Identified by Examples of EiE Relevant Response s Yemeni Stakeholder Disrupted education • Alternative and non-formal education services (home based, media delivery based) for students unable to attend school and for host and IDP communities • Increased access capacity in schools in safe areas, for example through additional shifts • Allow displaced teachers to teach in schools in host communities or in alternative/non-formal programs • Alliance with community and households to support education services and safety areas Loss of school year and no • Relevant and alternative education strategies to complete school year alternative mechanisms to include non-formal education services for IDPs, distance and home assess student learning based education, NGO-supported service delivery, community-based studies and mobile classrooms, etc. • Alternative learning assessment strategies, for example through assessment of reduced or accelerated content, credit for non-formal learning, flexible assessment schedules Destroyed or occupied • Safe and temporary school facilities, rapid assessments for infrastructure reconstruction, negotiate schools as free zones and rules of no engagement in education facilities Socio-emotional trauma • Training of adults (teachers, parents, and community members) on affecting education actors strategies to transmit a sense of normalcy, security and care to (students, teachers, students administrators and • Socio-emotional programs in school setting through social, artistic families) and physical activities (sports and exercise) • Counseling services for educators and parents Physical safety and • Provision of basic needs, safety shelter for displaced communities, protection security and protective support • Support to government negotiations for safe schools with armed groups (mediated by UN and other trustable partners) Post-Crisis support • Relevant education services to mitigate long-term effects of crisis through school-based practices for social harmonization, reconciliation, values, positive relations, representation of minorities, non-violent discipline, etc. Community-School • Build and strengthen alliances between communities and schools to Alliances provide safety, access for out-of-school children and youth, relevant learning, indigenous and cultural practices, shared accountability, etc. • Public, private and non-governmental institutions and services provide school-community support before, during and after crises. 46 Annex 11: Timeline of World Bank Supervision and Issues Addressed The review for this ICR was based on detailed analysis of the project supervision documents prepared and archived since 2008 for SEDGAP. It clearly shows that the implementation of SEDGAP, over its six years of operation, went through three distinct phases: (i) a slow start up in 2009-10; (ii) the 2011 security and political crisis and 2012 project restructuring; and (iii) the 2013-2015 accelerated progress. In addition, in 2015, the project closed in the midst of renewed armed conflict in Yemen. Coding of the supervision reports also noted that crucial issues over the life of the project included governance, component execution, and progress monitoring issues, all within various periods of acute political and security crises. The following tables present the data used to reconstruct this project timeline and issues of concern. 47 1st Phase of Implementation: Delayed Effectiveness and Slow Project Start Up (2008-2010) First Phase of Implementation: 2009-2010 Issues to Explore 2008 (ISR #2) 2009 (ISR #3 and #4) 2010 (ISR #5 and #6) GOVERNANCE ISSUES Government Collective • Client Ownership first steps Coordination • National 2nd Edu Workshop Planned Vision and Ownership • workshops held and to be ongoing 9 governorates • Limited Collective Vision For Secondary Education Reform signed participation protocol Goals • Government ownership workshops in Aden and • Need for Stakeholders Coordination Mukallah • Proposed TA to help build inter-institutional coordination • National secondary Ed workshop proposed due to political sensitivity of reforms Project Management • Technical Assistance for PAU Capacity • Consulting firm to support school civil works to be hired • Limited counterpart funds and resources • No secondary Level Data for Monitoring and indicators • Procurement of 49 school advanced but with delays • Project management downgraded to moderately satisfactory due to Govt. limited funding Donor Coordination & • KFW and GTZ parallel co- • DP coordination and rotation of leading role NL lacks • The Netherlands embassy will be contact DP Harmonization financing Clarity of TF Mgmt. by World Bank NL commits 14.8 • Multi-Donor Mission takes place • NL Trust Fund co-finance million • DFID Informs of Fully Withdrawal from Project by March administered by World • KFW expected to take leading role after 1 year Delay 2011 Bank in DFID Funds Commitment Initial Donor Tensions on • Netherlands to Reduce Financing by 50% • DPs and GoY MOUs Taking Leadership • Germany – GTZ to support modules for science and math, drafted and teacher and staff training • DFID Administration and KFW funds • Donor Partners Legal Confirmation World Bank Support • Bank continues to lead fiduciary reviews (procurement and FM) COMPONENT IMPLEMENTATION ISSUES Supply-Based Access • Noting delays in infrastructure designs • Field Visits to Communities with School Construction Support (School Building • 18 Secondary School Designs Slow and Materials) • School designs still delayed • Claim that Government Introduced many changes to school designs • Inventory of School furniture and equipment to start Quality Teaching and • Secondary School Curriculum Politically Sensitive • Planning assessment interventions Learning • Noting delays in curriculum framework design • Science and Mathematics modules to be developed • Education Policy Studies delayed • Teacher Training to be planned PROGRESS AND OUTCOMES ISSUES 48 Implementation Progress • Credit Effectiveness • Project Launch (Feb 2009) • Implementation Delays Reached • Slow 1st Year of the Project 166,000 Disbursement • Restructuring discussions begin • Sequence of Interventions: ISRs have not mentioned demand based interventions (School Based Management, Grants, CCTs, etc.) Outcomes • Baseline of results framework to be filled • Project M&E Readiness and TA • Propose o UNESCO IIEP for capacity building • 2nd Phase of Implementation: 2011 Political and Security Crisis Second Implementation Phase: 2011 Political and Security Crisis Issues to Explore 2011 (ISR 7) 1st Semester of 2012 (ISR 8) Context of Adversity 2011 Political and Security Crisis • 2011 Crisis affects PAU staff • Voluntary Evacuation of Bank Staff Governance Issues Government Collective Vision and • Collective Vision and Dialogue Problems Accentuated • Collective secondary education vision workshop in Ownership • Education Reform and Innovation Workshop postponed agenda • Coordination problems noted again • Education reform and innovation dropped • Lack of Ownership as ongoing complaint Project Management Capacity • Contracting Delays • Disbursements suspension affects implementation • Government financing delays • Lack of resources and financial planning as on- • Implementation Delays going complaint • No Data for Sector Indicators • Project director leaves noting stressful job • PAU reports 2011 delays • Project management with moderate functioning • Capacity short term TA dropped (due to cash flow problem) Donor Coordination & Harmonization • DFID leaves support • Donor Financing Reduced (Dutch and DFID) • Germany, GIZ and KFW support for quality component • Bank and Dutch Trust Fund (no earmarked funds: • Netherlands re-confirms fund cancellation flexible) • ISR inconsistency: DFID and Netherland mentioned reduced • KFW parallel financing support before 2011 crisis (why?) World Bank Support • Contradiction - Procurement and Account Hold and • Suspension noted again Implementation Delays • Management concur with restructuring • Hold Replenishments to Designated Account and cash flow • Supervision Moved to Amman problems • Suspension of disbursements lifted • Procurement NOB Caution by Bank • Third Party Monitoring Proposed (MIS hired) • Official suspension of disbursements 49 Component Implementation Issues Supply-Based Access Support (School • School Contracts Signed • School Designs approval Building and Materials) • School Field Visit _ Infrastructure (no use of septic tanks by • School rehabilitation contracts progress community) • Demand Incentives Start • School Community grants delayed Demand Side Interventions (ICTs, • Transportation Incentives Noted Again School Grants) Quality Teaching and Learning • Assessment action plan by MOE • Quality component delays noted with similar • Quality (teaching and assessment) delayed reasons as before • Teaching and Learning Materials Delayed • Previous plan for teaching and Learning materials noted again • Propose again training, assessment acceleration Component 3 (Policies Studies) Studies continue on hold • Studies delayed due to restructuring Progress and Outcomes Issues Implementation Progress • Project Monitoring Limited Data • Project restructuring finally launched • Tensions against restructuring • Scope of project reduced (5 governorates and 50 schools) Outcomes • Progress before crisis noted in ISR • Retention going down_ concern 50 3rd Phase of Implementation: Revamping Project Progress (June 2012-May 2015) Third Phase of Implementation: May 2012-May 2015 Issues to Explore 2012- 2013 (ISR 9 and 10) 2014 (ISR #11, 12) 2014-2015 (ISR #13 and 14) CONTEXT OF ADVERSITY 2011 Political • Political crisis impacts policy discussions • Acute political and security crisis and Security and activities • Bank lessons learned of education in conflict Crisis • Political Unrest Impact on Donor • Bank staff evacuated to Cairo Commitment • Destroyed MOE documents during crisis • Government Contributions Limited During Crisis • Houthi attacks in Sana’a • Lack of water, electricity, food, water during crisis • Missions suspended January 2015 • Saudi airstrikes in Yemen GOVERNANCE ISSUES Government • Lack of coordination repeated, yet Collective Vision visioning workshop dropped and Ownership • Ownership re-noted as a problem, yet workshops cancelled Project • Project management coping to recover • High Staff Turnover • US$ 2.2 million Govt. counterpart financing not yet Management post-crisis • PAU Staff Salary Dissatisfaction provided by end of 2014 Capacity • Financial administration delays while • Limited counterpart funds: • Procurement progress in spite of difficult strengthening PAU circumstances • International Procurement TA Support • Direct payment requests submitted to accelerate • PAU Staff Salaries Increments Request implementation • Rotation of PAU directors • Third Party Monitoring Feedback available Donor • EKN and DFID noted their Coordination & reduction/withdrawal in 2010, which ISRs Harmonization noted as a result of the 2011 crisis • IDA and reduced Netherland Trust Fund are reallocated across reduced project scale World Bank • Suspension lifted • Bank-PAU weekly audio conferences Support • Disbursement method (statement of • World Bank disbursement suspended again in 2015 expenses) lack of understanding • World Bank Supervision Resources re- committed and TTL Change in 2013 51 COMPONENT IMPLEMENTATION ISSUES Supply-Based • Advance of School Construction and • Civil Works on Track: Last two Access Support Rehabilitation remaining schools to be delivered (School Building • Note infrastructure progress, but no • School use before completion (high and Materials) progress on quality inputs demand) • School contracts re-initiated after • Environmental (soil) issues at school suspension • Progress in Contracting Rural Female Teachers Demand Side • Transportation incentive (double check if • Community Incentives Satisfaction • Achieved school based management and Interventions double reporting) • Progress of Community Grants, CCTs, accountability (ICTs, Transportation Grants • Capacity building for parental councils School Grants) • Demand based girls enrollment and retention • CCTs provided for girls in grades 4-12 grade girls • CCTs provided for boys in grades 10-12 • Lessons learned on demand based community programs • Satisfaction on Community-based interventions Quality Teaching • Quality component delays complains • Repeated Assessment • Training for 3000 teachers suspended (no and Learning repeated, no advances on prior Recommendations counterpart funds) agreements on material, training and • Training Progress and On Track by assessment end of 2014 • Request for student assessment plan and GIZ baseline and Midterm surveys for learning assessment • Teacher Training still slow Component 3 • Component 3 -studies-dropped (Policies Studies) PROGRESS AND OUTCOMES ISSUES Implementation • Project restructured (April 2012) • Advances in all components • 95% progress achieved Progress • After negative impact of Disbursement • Results Framework Amendment • Cancelled Interventions suspension, Implementation Progress • Disbursement from 5% (6_2013) to • Project will close May 31 2015 Improving 58% (5_2014) • Results framework to be updated • Midterm review postponed to Nov 2013 • Mid-Term Review Completed • Education Research and Development Center • Project focused and scope reduced • Indicators not adjusted before (9 to 5 (ERDC) for PDO indicator 4 and outcome indicator governorates) 2.6, and teacher training for outcome indicator • Level II restructuring (amendment of 2.5.have been aborted due to the suspension of PDO Indicators to match baselines disbursement and lack of GoY financing. for only 5 governorates) • 2015 World Bank Suspension of Disbursement for • Project Progress from 58% to 79% Overall Portfolio and automatic MU implementation (Nov 2014) Disbursement rating for all projects 52 Outcomes • 44 schools remaining at 35% progress Male retention dropped – 79.8 (2013) % • Missing data to be updated for PDO indicators • Gender Parity Index Progress to 78.7% (2014): possible armed conflict • 2014-2015 Annual Education Survey Data for PDO 1, • Girls GER from 44% to 61% impact 2 and 3 • • 89 female teachers in rural areas • Moderately Satisfactory PDO rating • Anecdotal evidence collected by PAU and MoE team strongly suggest that the project significantly contributed to encourage girls schooling in disadvantaged and rural areas and awareness towards school based management and enhanced accountability at community level has been achieved. • Reported positive perception towards girls education 53