Document of The World Bank Report No: ICR00003246 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-42030) ON A CREDIT IN THE AMOUNT OF SDR 15.1 MILLION (US$ 22 MILLION EQUIVALENT) TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR A BALOCHISTAN EDUCATION SUPPORT PROJECT January 30, 2015 Education Global Practice Pakistan Country Management Unit South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2014) Currency Unit = Pakistani Rupee (PKR) US$ 1.00 = PKR 102.270 FISCAL YEAR July 1 – June 30 ABBREVIATIONS AND ACRONYMS BEF Balochistan Education Foundation BEMIS Balochistan Education Management Information System BESP Balochistan Education Support Project BoD Board of Directors BPEP Balochistan Primary Education Project CAS Country Assistance Strategy CBO Community Based Organization CIPs Community Schools Implementation Partners CPS Country Partnership Strategy DoE Department of Education EGMA Early Grade Mathematics Assessment EGRA Early Grade Reading Assessment EMF Environmental Management Framework FATA Federally Administered Tribal Areas GER Gross Enrolment Rate GoB Government of Balochistan GoP Government of Pakistan IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IDA International Development Association IECs Individual Engineer Consultants IPs Implementation Partners (includes CIPs, PIPs and TIPs) IR Intermediate Result KPK Khyber Pakhtunkhwa M&E Monitoring and Evaluation MD Managing Director MIS Management Information System MTR Mid-term Review NER Net Enrolment Rate NGO Non-Governmental Organization PDO Project Development Objectives ii PEC Parent Education Committee PGEB Promoting Girls Education in Balochistan PIPs Private School Implementation Partners PITE Provincial Institute for Teacher Education PPAF Pakistan Poverty Alleviation Fund PRSP Poverty Reduction Strategy Paper QALP Quality Assessment of Lending Portfolio QER Quality Enhancement Review RCC Roller-Compacted Concrete SDR Special Drawing Rights SED Secondary Education Department SIL Sector Investment Loan STR Student Teacher Ratio TIPs Technical Assistance Implementation Partners TPV Third Party Validation Vice President: Annette Dixon Country Director: Rachid Benmessaoud Practice Manager: Halil Dundar Project Team Leader: Huma Ali Waheed ICR Team Leader: Ayesha Khan iii PAKISTAN Balochistan Education Sector Project Table of Contents DATA SHEET ............................................................................................................................... vi A. Basic Information ...................................................................................................................... vi B. Key Dates .................................................................................................................................. vi C. Ratings Summary ...................................................................................................................... vi D. Sector and Theme Codes.......................................................................................................... vii E. Bank Staff ................................................................................................................................. vii F. Results Framework Analysis ................................................................................................... viii G. Ratings of Project Performance in ISRs ................................................................................... xi H. Restructuring (if any) ............................................................................................................... xii I. Disbursement Profile ............................................................................................................... xiii 1. Project Context, Development Objectives and Design ....................................................... 1 1.1 Context at Appraisal ............................................................................................................1 1.2 Original Project Development Objectives (PDO) and Key Indicators ................................1 1.3 Revised PDO and Key Indicators and Reasons/justifications .............................................2 1.4 Main Beneficiaries ...............................................................................................................2 1.5 Original Components ...........................................................................................................2 1.6 Revised Components ...........................................................................................................4 1.7 Other significant changes .....................................................................................................4 2 Key Factors Affecting Implementation and Outcomes ...................................................... 5 2.1 Project Preparation, Design and Quality at Entry ................................................................5 2.2 Implementation ....................................................................................................................6 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization .....................8 2.4 Safeguard and Fiduciary Compliance ................................................................................10 2.5 Post-completion Operation/Next Phase .............................................................................12 3 Assessment of Outcomes .................................................................................................. 13 3.1 Relevance of Objectives, Design and Implementation ......................................................13 3.2 Achievement of Project Development Objectives .............................................................13 3.3 Efficiency ...........................................................................................................................17 3.4 Justification of Overall Outcome Rating ...........................................................................17 3.5 Overarching Themes, Other Outcomes and Impacts .........................................................18 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ..................19 iv 4 Assessment of Risk to Development Outcome ................................................................. 19 5 Assessment of Bank and Borrower Performance ............................................................. 19 5.1 Bank Performance ..............................................................................................................19 5.2 Borrower Performance .......................................................................................................21 6 Lessons Learned................................................................................................................ 22 7 Comments on Issues Raised by Borrower/Implementing Agencies/Partners................... 23 Annex 1. Project Costs and Financing .......................................................................................... 24 Annex 2. Outputs by Component.................................................................................................. 25 Annex 3. Economic and Financial Analysis ................................................................................. 29 Annex 4. Bank Lending and Implementation Support/Supervision Processes............................. 35 Annex 5. Summary of Borrower's ICR and/or Comments on Draft ICR ..................................... 37 Annex 6. List of Supporting Documents ...................................................................................... 46 MAP .............................................................................................................................................. 47 v DATA SHEET A. Basic Information Pakistan: Balochistan Country: Pakistan Project Name: Education Support Project - BESP Project ID: P094086 L/C/TF Number(s): IDA-42030,TF-91827 ICR Date: 12/29/2014 ICR Type: Core ICR ISLAMIC REPUBLIC Lending Instrument: SIL Borrower: OF PAKISTAN Original Total XDR 15.10M Disbursed Amount: XDR 15.07M Commitment: Revised Amount: XDR 15.10M Environmental Category: B Implementing Agencies: Balochistan Education Foundation Planning and Development Balochistan Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 01/06/2005 Effectiveness: 08/09/2006 08/09/2006 07/15/2010 Appraisal: 04/03/2006 Restructuring(s): 07/20/2012 05/27/2014 Approval: 06/22/2006 Mid-term Review: 10/31/2009 03/15/2010 Closing: 01/31/2011 07/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: High Bank Performance: Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Satisfactory Government: Unsatisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: vi Overall Bank Overall Borrower Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision No None (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Other social services 21 21 Primary education 79 79 Theme Code (as % of total Bank financing) Education for all 33 33 Gender 17 17 Other Private Sector Development 17 17 Participation and civic engagement 17 17 Rural services and infrastructure 16 16 E. Bank Staff Positions At ICR At Approval Vice President: Annette Dixon Praful C. Patel Country Director: Rachid Benmessaoud John W. Wall Practice Manager/Manager: Halil Dundar Michelle Riboud Project Team Leader: Huma Ali Waheed Naveed Hassan Naqvi ICR Team Leader: Ayesha Khan ICR Primary Author: Ayesha Khan Aazar Wali Bhandara vii F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objective of the Project is to promote public-private and community partnerships to improve access to quality primary education, in particular for girls. Revised Project Development Objectives (as approved by original approving authority) The PDO was not revised. (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years An additional 35,000 students in primary schools established through public-private Indicator 1 : partnerships, by end of project period 707,000 students in primary Value schools of which 45000 quantitative or 35,000+ 48,354 students study in the non- Qualitative) government/private sector Date achieved 06/01/2005 01/31/2011 06/30/2014 This target was exceeded by 38%. At Project closing, enrollment in community schools Comments was 25,973. Enrolment in private schools supported under Component 2 of the Project (incl. % was 22,381 at the time activities were concluded. Enrollment in project schools at achievement) Project Completion (48,354) is the sum of enrollment in community and private schools. Indicator 2 : Average student attendance rate not less than 70% Value 85% community 84% in community schools; quantitative or 70% schools; 86% private 91% in private schools Qualitative) schools Date achieved 05/29/2008 01/31/2011 06/30/2014 The baseline reflects performance at the end of the first year of operation of schools Comments established under the Project. This performance was maintained throughout project (incl. % implementation. At appraisal, a target of at least 70% was set in view of average achievement) attendance rates at public schools. Indicator 3 : Average student grade completion rate no less than 70% Value 74% community quantitative or 70% 70% schools; 85% private Qualitative) schools Date achieved 05/19/2008 01/31/2011 06/30/2014 Comments This target was exceeded. Average student grade completion rate was maintained above (incl. % 70% in both community and private schools throughout the life of the Project. achievement) Indicator 4 : Average teacher attendance (net of leave entitlement) rate not less than 90% Value 95% community quantitative or 95% 90% schools; 94% private Qualitative) schools Date achieved 05/29/2008 01/31/2011 06/30/2014 viii Comments The baseline was set at the end of the first year of operation of supported schools. (incl. % Average teacher attendance was maintained above 90% in both community and private achievement) schools throughout the life of the Project. Average annual increase of 3% and 5% in learning achievement in language and Indicator 5 : mathematics, respectively, in project-supported schools Value Mean score of 13.75 for Increase in Math of Math>5% quantitative or Mathematics and 19.36 for 9.6%; Language Language>3% Qualitative) Language 16.17% Date achieved 01/01/2009 01/31/2011 06/30/2009 Comments This target was exceeded over the 2008/09 period. However, given that only a baseline (incl. % and one subsequent round of assessments were conducted, there are insufficient data achievement) available to assess this indicator. Indicator 6 : At least 40% enrollment in community schools is of girls PSLM 2004-05 Data Value 42% community Pakistan primary GER: M/F quantitative or 40% schools; 34% private (94/77) Balochistan primary Qualitative) schools GER: M/F (83/49) Date achieved 10/04/2006 01/31/2011 06/30/2014 Comments This target was met for community schools, where female enrollment was maintained (incl. % above 40%. However, this target was not met for private schools, where it remained achievement) approx. 35% on average. Indicator 7 : Direct project beneficiaries (core indicator) 752,000 primary school Value students (707,000 in govt. quantitative or 19,500 48,354 schools and 45,000 in Qualitative) private/non-govt. schools) Date achieved 06/01/2005 07/31/2014 06/30/2014 Comments This indicator was added as a core indicator in 2011 (Implementation Status Report 12). (incl. % Please see Indicator 1. achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years By Mid Term Review (MTR): 10,500 students in 350 community schools, at least 40% Indicator 1 : of whom are girl students. 10,500 students in 707,000 students in primary 26,839 students in Value 350 community schools of which 45,000 643 community (quantitative schools, at least students study in the non- schools, 47.5% of or Qualitative) 40% of whom are government/private sector whom are girls. girl students. Date achieved 06/01/2005 12/01/2008 03/31/2009 Comments By MTR, these targets had been surpassed. The enrollment target had been exceeded by (incl. % 155% and the community school target by 83.7%. Female student enrollment was 7.5% achievement) above the target. ix Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years By Project End: Enrollment maintained or increased above 19,500 students in 650 Indicator 2 : community schools established in the first three years of the Project. At least 40% of the students are girls. Enrollment maintained or increased above At the end of the 707,000 students in primary 19,500 students in Project, enrollment in Value schools of which 45,000 650 community community schools (quantitative students study in the non- schools established was 25,973 in 633 or Qualitative) government/private sector in the first three community schools years of the Project. (42% girls). At least 40% of the students are girls. Date achieved 06/01/2005 01/31/2011 06/30/2014 Comments At Project End, the enrollment target was exceeded by 33.2%. Enrollment in CSs was (incl. % above the target of 19,500 throughout project life, as was the proportion of female achievement) enrollment. From the 649 CSs established under the Project, 633 remained operational. By MTR: 200 new schools will be supported with total new enrollment of 10,000 Indicator 3 : students 200 new schools Value will be supported 18,631 students in (quantitative 0 with total new 209 supported Private or Qualitative) enrollment of Schools. 10,000 students Date achieved 10/01/2006 12/01/2008 03/31/2009 Comments By MTR, 209 new private schools were supported by the Project, with the enrollment (incl. % target exceeded by 86.3%. achievement) By Project End: 300 new schools supported with enrollment maintained at or increased Indicator 4 : above 15,500 300 new schools supported with 197 new schools Value enrollment supported with an (quantitative 0 maintained at or enrollment of 22,381 or Qualitative) increased above students 15,500 Date achieved 10/01/2006 01/31/2011 Comments The number of schools to be supported was decreased to 200 at MTR. Despite this, (incl. % enrollment was maintained well above the target until support to this component was achievement) concluded. By MTR: 1,050 teachers trained, members of 350 PECs trained, at least 50% of BEF's Indicator 5 : professional staff and Project-related IP staff trained. Value 1,050 teachers 1000 teachers (quantitative 0 trained, members of trained; members of or Qualitative) 350 PECs trained, at 643 PECs, and all x Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years least 50% of BEF's professional BEF professional staff staff trained and Project-related IP staff trained. Date achieved 10/01/2006 12/01/2008 03/31/2009 Comments This target is considered met. There is only a marginal shortfall in the number of (incl. % teachers trained, and the targets for PEC member- and BEF staff- training were achievement) exceeded. By Project End: 2,100 teachers trained, members of 650 PECs trained, all BEF Indicator 6 : professional staff, and 80% of Project-related IP staff trained. 1,100 teachers were 2,100 teachers trained. 764 teachers trained, members of were trained in CSs Value 650 PECs trained, and 336 in private (quantitative 0 all BEF professional schools. 3,245 or Qualitative) staff, and 80% of members of 649 Project-related IP PECs were trained. staff trained. 100% of BEF and IP staff was trained. Date achieved 01/31/2011 Comments This target is considered met. The appraisal estimate target of 2,100 teachers to be (incl. % trained was high as it anticipated a larger number of teachers per school. achievement) BEF has successfully enhanced its capacity and that of its IPs to expand its operations Indicator 7 : into low enrollment areas and is in a position to scale up its operations in follow-up project. Value (quantitative Partially achieved or Qualitative) Date achieved 01/31/2011 Comments BEF supported schools in all 31 districts of Balochistan through the Project, and (incl. % achieved its capacity building objectives. Please see Section 2.5 for a description of achievement) follow-on support. G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/05/2006 Satisfactory Satisfactory 0.07 2 04/27/2007 Satisfactory Satisfactory 1.97 3 06/19/2007 Satisfactory Satisfactory 1.97 4 12/17/2007 Satisfactory Satisfactory 2.66 5 06/18/2008 Satisfactory Satisfactory 4.15 6 12/11/2008 Satisfactory Satisfactory 5.43 xi Date ISR Actual Disbursements No. DO IP Archived (USD millions) 7 06/18/2009 Satisfactory Moderately Satisfactory 6.24 8 12/09/2009 Satisfactory Satisfactory 8.50 9 06/17/2010 Satisfactory Satisfactory 10.21 10 01/03/2011 Satisfactory Satisfactory 11.34 11 07/07/2011 Satisfactory Moderately Satisfactory 11.34 12 02/11/2012 Moderately Satisfactory Moderately Satisfactory 14.82 13 08/20/2012 Moderately Satisfactory Moderately Satisfactory 14.82 14 03/21/2013 Moderately Satisfactory Moderately Satisfactory 15.22 15 10/13/2013 Moderately Satisfactory Moderately Satisfactory 18.59 16 04/16/2014 Moderately Satisfactory Moderately Satisfactory 20.46 17 07/30/2014 Moderately Satisfactory Moderately Satisfactory 21.58 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions Extension of Project closing date from January 31, 2011 to July 31, 07/15/2010 S S 10.21 2012 to allow completion of construction activities under Component 1 of the Project. (i) Extension of project closing date to July 31, 2014 to allow for completion of construction of functional community schools under Component 1 and (ii) Reallocation of unallocated expenditure category to finance 07/20/2012 MS MS 14.82 Community Implementation Partner contracts for extension period, to meet BEF equipment and operational expenses during extension period, and to provide additional resources for school construction. Reallocation of funds originally allocated for the construction of a BEF office to cover expenses for office equipment for BEF staff 05/27/2014 MS MS 20.46 (including monitoring equipment), a refresher teacher training course for community school teachers, and overdrawn Category 4 xii ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions expenditure on support to construction of community schools. I. Disbursement Profile xiii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1.1.1. At the time of preparation of the Balochistan Education Support Project (BESP), Pakistan was the second most populous country in the South Asia region with a population of over 150 million. Economic growth had increased from an average of 3.3% during 1997- 2002 to an average of nearly 6% during 2002-05. The 2005 earthquake in the north of Pakistan had no direct impact on the province of Balochistan and was expected to have a marginal impact on economic growth. 1.1.2. Balochistan's economy historically has differed from the rest of Pakistan. It is the poorest of the country's four provinces with standards of living and social indicators lagging substantially behind the rest of Pakistan. With 43% of the total landmass of the entire country, the province has only 5% of the country's population. This makes service delivery and infrastructure development challenging. A large proportion of the population lives in small and dispersed rural settlements with only 24% of the population living in urban areas. 1.1.3. At the time of appraisal, education attainment in Balochistan reflected the province's overall low development indicators. Literacy levels were at 37%, lagging far behind those of the other provinces and the national average of 53%. Low and declining enrollment, large gender disparities, poor quality of education, lack of continued community focus in community support programs, limited private sector participation, weak public sector capacity and limited financial resources were identified as issues affecting education sector development in the province at the time of appraisal. 1.1.4. Rationale for Bank Involvement: The Bank had been supporting education programs in other provinces and recognized that Balochistan had the greatest need, requiring both financial and technical support to enable it to test different service delivery models. At the same time, recognizing the challenges faced by the public sector in delivering quality education at the primary level, the GoB was committed to testing different models of public- private partnerships for primary education delivery, which, if successful, could be scaled up. It designed the Balochistan Education Support Project (BESP) to test an alternate delivery mechanism and approached the Bank for financial and technical assistance given its experience of supporting the Balochistan Primary Education Project (BPEP). 1.1.5. The Project objectives were aligned with the 2006 Country Assistance Strategy (CAS) and provincial priorities as articulated in the Balochistan Poverty Reduction Strategy Paper 2003 (PRSP 2003). Specifically, the Project was to contribute to improvements in education delivery, a key objective of the third pillar of the CAS, i.e., improving lives and protecting the vulnerable. The Balochistan PRSP emphasized the need to increase focus on improving human development, one of its five pillars. The Project also contributed to Pakistan’s long- term objective of achieving its Millennium Development Goals (MDGs) by increasing net primary enrollment rates and reducing gender disparity in Balochistan via successful public- private and community partnership delivery models. 1.2 Original Project Development Objectives (PDO) and Key Indicators 1.2.1 The development objective of the Project was to promote public-private and community partnerships to improve access to quality primary education, in particular for girls. 1 1.2.2 Success in achieving the PDO was to be measured using the following five PDO level outcome indicators and targets (PAD page 4): a) Total enrollment of students (Target: enrollment increased by 35,000, with at least 40% girls); b) Average annual student attendance (Target: average attendance not less than 70%); c) Average grade completion rate (Target: average completion rate not less than 70%); d) Average annual teacher attendance (Target: average annual teacher attendance, net of leave entitlement, not less than 90%); and e) Average annual increase in learning achievement in Language and Mathematics (Target: an annual increase of 3% and 5% respectively). 1.3 Revised PDO and Key Indicators and Reasons/justifications 1.3.1 The PDO and the PDO level key indicators were not revised. 1.4 Main Beneficiaries 1.4.1 The primary beneficiaries under the Project were students in the private and community schools funded by the Project. Secondary beneficiaries included the Balochistan Education Foundation (BEF), parents, teachers, non-governmental organizations (NGOs), private sector partners, public and private training institutes and local construction contractors. 1.5 Original Components 1.5.1 The Project supported education service delivery models through partnerships with NGOs and low-cost private sector education providers. The Project had three components which were to be implemented under partnership arrangements between BEF and three distinct types of Implementation Partners (IPs). These included: (i) Community Schools Implementation Partners (CIPs); (ii) Private School Implementation Partners (PIPs); and (iii) Technical Assistance Implementation Partners (TIPs). US$1.8 million (9% of the project cost) was left unallocated at the start of the Project to cover contingencies and the expansion of successful interventions. The components and partner roles are summarized below. 1.5.2 Component 1: Establishment of New Community Schools in Rural Areas (US$ 13.9 million): This component was to provide access to quality primary education to school-aged children through the establishment of new community schools in rural areas where the community was able to enroll at least 20 students in a school and where there were no girls’ schools within a radius of two kilometers. These schools had to provide formal primary education with assistance from eligible partner NGOs, known as CIPs. BEF was responsible for monitoring performance of these schools and CIPs and their compliance with the BEF’s Operations Manual, and for arranging third party assessments and evaluations of the schools established under this component. 2 Table 1: Responsibilities of Key Entities under Component 1 Community Schools Implementation Partners Parent Education Committees (PECs) (CIPs) i. Mobilize communities and ensure their viii. Provision of premises for school; participation in the selection of PECs; ix. identification and employment of local ii. help PECs get registered with GoB; teachers; iii. arrange training of PEC members in x. opening and maintaining school accounts in management and finance; either a bank or a post office; iv. assist PECs in starting and operating their xi. maintaining financial and meeting records; community school; xii. ensuring proper maintenance and physical v. arrange for on-site classroom and teacher condition of the school premises, support; xiii. Oversight of student enrollment and teacher vi. establish baseline of the school, PEC and attendance; community; and xiv. Maintenance of active community vii. monitor enrollment, teacher and student participation; and attendance, community participation, quality xv. monitoring school performance. of learning and physical condition of the school. 1.5.3 The PECs, selected by beneficiary communities, were responsible for managing and supervising community schools. They were also to pay teacher salaries and other recurrent costs of schools through funding provided by BEF. The Project was to fund recurrent costs for the first two years of each school’s operation through BEF at the recommendation of CIPs and following verification of agreed milestones. To ensure the sustainability of community schools, GoB committed to fund recurrent costs of schools from the third year of each school’s operation through BEF. These schools were to continue operating under the management of the PECs after project closure. 1.5.4 Eligible community schools under the Project which successfully maintained enrollment for two years and whose community donated land by formal transfer of ownership by mutation to the respective PEC were to be provided with appropriate funding from the Project for a new permanent school building in accordance with existing and projected enrollments. Construction had to be undertaken by the community according to an earthquake resistant design approved by the Bank and with technical assistance arranged through the Project. 1.5.5 Component 2: Support to New Private Schools (US$2.1million): This component focused on promoting access to low-fee quality private primary education in semi-urban and urban areas through support to private school operators, known as PIPs, to establish new private schools along the fellowship model tested successfully under BPEP. Each PIP was to be given a per student subsidy for up to four years according to rates determined by BEF in consultation with PIPs. PIPs were to receive annual subsidies per student for facilities and material costs, and a monthly subsidy linked to student enrollment and attendance. New private schools were to be established (i) if there were at least 50 out of school children (age 4-9) in a locality; and (ii) if there was no government primary school or any other school for girls in a radius of one kilometer. An additional school in the same locality could only be established if it were justified based on growth in the number of students or overcrowding in existing schools. The schools were to charge low fees (at appraisal, school fees were not to exceed Rs.300 per month). The performance of this model was to be evaluated during the Mid-Term Review (MTR), and the option of extending support to existing private schools was to be explored. 3 1.5.6 Component 3: Capacity Building (US$4.2 million): The objective of this component was to build the capacity of BEF and its partners under Components 1 and 2.  Sub-component 3.1 - Training and Skill Development: To ensure the provision of quality education in both community and private schools, the Project was meant to support quality improvements at the school level. These improvements included local teacher recruitment, teachers’ professional development, including subject training, close monitoring and the academic supervision, availability of free textbooks, student achievement testing and provision of teaching and learning materials. To ensure that key stakeholders were able to perform their respective functions effectively, intensive training was to be given to them. PECs, CIPs and PIPs were to be trained in school management, book keeping, accounting, participatory techniques, gender and monitoring and supervision. BEF was to receive training in project management functions including financial management, procurement, environment and monitoring and evaluation. This capacity support was to be delivered through technical assistance provided by a variety of consultants, NGOs, and public and private training institutes, known as TIPs.  Sub-Component 3.2 - Institutional Support to BEF: This sub-component was meant to support: (i) BEF’s staff salaries and operational costs (including evaluation studies); and (ii) if land was provided by GoB, the construction of a permanent office for BEF. 1.6 Revised Components 1.6.1 The Project components were not revised. 1.7 Other significant changes 1.7.1 Closing date: The original project closing date, January 31, 2011, was extended twice (by a total of 42 months) under two successive Project Restructurings in order to allow for the completion of construction activities under Component 1. The first closing date extension was to July 31, 2012, and the second to July 31, 2014. Construction of eligible community schools faced delays due to a worsening security situation in the province and difficulties faced in identifying appropriate partners to provide technical support to PECs during the construction process. Section 2.2 discusses this in further detail. 1.7.2 Reallocation: The Project included an unallocated expenditure category (US$1.8 million) to meet contingency costs and the expansion of successful interventions. This amount was reallocated in the second Project Restructuring, dated July 20, 2012, to finance CIP contracts for the extension period to meet equipment and operational expenses of BEF during the extension phase and to provide additional resources for school construction. At appraisal, Project funds had been allocated to the construction of a permanent office for BEF on land that was to be allotted by the GoB. However, no suitable site was provided. These proceeds were distributed to other expenditure categories (through the third and final Project Restructuring, dated May 27, 2014) at the end of the Project (July 2014) to fund activities which would promote the sustainability of project results. These activities included office equipment for BEF, such as computers, smart phones and tablets to serve as a delivery mechanism for supplementary education content and material for teachers and students in the community schools. Other activities included refresher teacher training courses for teachers 4 in community schools and third-party validation of project results. These funds were also used to cover additional costs associated with the recruitment of engineers to support the construction of schools under Component 1. 1.7.3 Revision of targets (non-PDO): The following targets were revised during the Project:  The number of private schools to be established under Component 2 was revised downwards from 300 to 200 schools during the MTR 1 because of the non- availability of private sector partners that could have qualified under the selection criteria of this component. Please see Annex 2.  The number of community schools to be constructed using project funds was decreased from the appraisal target of 450 to 225 schools as documented in the November 2011 Implementation Support Aide-Memoire (AM) due to significant escalation in the cost of construction (approximately three times the appraisal estimate). No additional funding could be secured to bridge the shortfall. 2 Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 2.1.1 Background Analysis: BESP was designed as a four and half year Specific Investment Loan (SIL) financed through an IDA Credit. The Project, in its focus on increasing access, especially for girls, was an appropriate response to challenges faced in the delivery of quality education in the country’s poorest province. Challenges identified by the task team during preparation were based on available data and are found to be sound by the ICR team. Most importantly, the Project builds on the success of community-supported and non-government and private sector entity involvement in the implementation of education, infrastructure, and micro-enterprise programs under BPEP and the Bank-supported Pakistan Poverty Alleviation Fund (PPAF)2. Key lessons learned from BPEP were fully reflected in the design of BESP and include, among others: (i) avoiding complex project design; (ii) outsourcing implementation to competitively selected IPs; (iii) ownership of project design by the borrowers during the preparation phase and actions demonstrating a long term commitment to the project design; (iv) the involvement of communities improves school performance; and (v) partnering with qualified NGOs contributes significantly to project success. BESP design also reflected important institutional and organizational lessons learned from PPAP, some of which include: (a) establishing the autonomy of the apex body, BEF; (b) recruiting senior management and staff competitively from the private sector; and (c) establishing clear and transparent operational procedures and selection criteria for partner organizations. 2.1.2 Project Design: Given Balochistan’s unique challenges in education service delivery compared to the rest of the country (e.g., large geographic spread, low population density, poor infrastructure), BESP’s approach was to first test and potentially scale up innovative models of education delivery outside the public sector. It was the first Bank-supported 1 This is documented in the ‘Preliminary findings of the Mid Term Review of BESP dated March 15, 2010 (Folder ‘2009 Preliminary Findings) and in ISR 10 dated December 21, 2010. 2 The Pakistan Poverty Alleviation Fund Project was implemented from 1999-2004. 5 education project in Pakistan to support an apex organization (BEF) which would use IPs to execute and implement project components. The design of the Project was simple in its support to two different models of delivery, and both components were heavily supported by the technical assistance and capacity building component (Component 3). The Project made adequate allowances for course correction and design changes by leaving a portion of the Credit amount unallocated and by setting clear targets for the mid-term. The Project provided Credit proceeds directly to BEF, which had the following advantages: (i) it mitigated the risk of provincial budgetary constraints on the Project; (ii) it bypassed institutional capacity constraints faced by the public sector; and (iii) it emphasized the accountability that the implementing agency would be held to and thus generated a high level of ownership of the Project by the implementing agency. The arguments presented in the PAD for alternative approaches and design elements which were considered and subsequently rejected by the preparation team are found to be compelling and well thought out. 2.1.3 The PDO and its indicators are deemed appropriate to the Project. Targets set for the PDO indicators were realistic and phased according to the pace of implementation expected at the time of Project design. 2.1.4 Risks and Mitigation Measures: Given that BESP sought to test different and new approaches for service delivery outside the public sector in a challenging context, the Project was properly identified as high risk. Diminishing government commitment, political interference, security challenges, and weak fiduciary capacity all posed substantial to high risk to the Project. All risks were identified and mitigation of risks was handled consistently and in a timely manner during the Project. The commitment of the GoB in maintaining BEF’s operational autonomy and its predominantly private sector Board contributed to successful implementation. 2.1.5 Quality at Entry: No quality-at-entry assessment of the Project was carried out by the former Quality Assurance Group (QAG). A Quality Enhancement Review (QER) was conducted in November 2005 during appraisal. It endorsed the design and approach of the Project and provided guidance on issues of sustainability, capacity building, access and quality, as well as implementation arrangements. The final design of BESP reflects the guidance received. It bears mentioning that a Quality Assessment of Lending Portfolio (QALP) conducted in November 2008 found the design of the Project to be highly satisfactory. 2.1.6 Quality at Entry is rated Satisfactory. This assessment is based on the following factors: (i) the PDO is simple and relevant; (ii) the evaluation framework and targets (except for one indicator, see paragraph 2.3.1) are sound and realistic; (iii) the design is an innovative response to clearly identified challenges in the sector and province; (iv) project design is not complex, and has a well-designed technical assistance component to support implementation of project activities; (v) lessons learned from other projects have been fully reflected in the design; (vi) risks are appropriately identified and mitigation measures are deemed proportionate to the magnitude of risk; and (vii) implementation arrangements, including for fiduciary and safeguards aspects, are suitable and realistic. 2.2 Implementation 2.2.1 Implementation progress remained satisfactory until early 2011 when the rating was downgraded to moderately satisfactory in the Implementation Status and Results (ISR) 6 reports to reflect delays in the construction of community schools under Component 1. This rating was maintained until project closing.. 2.2.2 The following factors contributed to successful implementation: (i) GoB commitment to the success of project interventions. This was demonstrated by: (i) the regular transfer of teacher salaries to PECs with few exceptions; (ii) the Department of Education’s (DoE) recognition of community schools and the subsequent allocation of Balochistan Education Management Information System (BEMIS) codes to each; and (iii) the provision of free textbooks to community school students and teacher training to community school teachers in the last year of the Project. The recognition of community schools was of particular importance insofar as students who were issued a completion certificate from a BESP-supported community school could continue their education at a public middle school. (ii) Timely recognition and resolution of implementation challenges by BEF and the Bank team. Regular and candid dialogue between the Bank and the counterpart on implementation progress, and a well-substantiated MTR 3 allowed timely restructuring of the Project on three occasions – the first two extending the Closing Date to allow the completion of construction of community schools4, and the last reallocating funds to other expenditure categories as required. The dialogue also ensured that adjustments to BESP design and targets could take place to reflect ground realities. A key instance is the discontinuation of Component 2 – Support to New Private Schools5 after it was apparent that the model was not sustainable due to a lack of potential private sector partners and poor cost recovery by some PIPs. (iii) Satisfactory Project management and M&E. A dedicated and proactive implementing agency such as BEF has been pivotal to the success of the Project. BEF’s operations continuously took into account the ground realities of operating in a challenging security environment. BEF developed effective protocols to handle problems related to internal community conflict and teacher replacement in case of teacher resignation or unavailability. (iv) Effective use of capacity building and technical assistance funds. Component 3 – Capacity Building - was instrumental to the successful implementation of Components 1 and 2. It funded activities such as teacher training, PECs training, CIPs, PIPs and BEF staff training, student achievement testing, and monitoring and evaluation (including Third Party Validation (TPVs)). The timing of these activities was well-aligned to the pace of implementation. (v) Close project supervision despite difficult security environment. Despite the volatile and steadily worsening security situation in the province, both the Bank team and BEF did a commendable job of supervising BESP on a regular basis, using appropriate alternatives when required. These included the use of third-party monitoring when Bank visits to the field were not feasible due to security constraints and meetings between the task team and BEF in locations other than Quetta or via videoconference on several occasions. 3 A Third Party Validation (TPV) of Project activities was completed in November 2009, well in time to feed into the MTR. 4 The second Restructuring also reallocated funds to other expenditure categories as required. 5 This Component was implemented for a period of four years as originally designed in the Project. 7 2.2.3 The following factors hindered or delayed implementation: (i) Difficulties in provision of technical support to PECs and CIPs during construction of schools under Component 1. This, along with a worsening security situation, the unavailability of raw materials and skilled labor in some communities and occasional delays in the release of funds to PECs for construction resulted in delays throughout the life of the Project. Several alternatives were considered to address the low capacity of the PECs with regard to community school construction and to ensure a high quality of construction. However, none proved satisfactory: a management supervision firm could not be identified, the cost of services of a similar firm proved too high, and the supervisory services offered by the GoB’s own Civil and Works Department proved inadequate. The final option exercised, i.e., the hiring of Individual Engineer Consultants (IECs) by BEF to facilitate, supervise, and validate school construction was eventually successful, particularly under the management and supervision of a Construction Manager hired by BEF. (ii) Non-transfer of non-salary recurrent costs to community schools. GoB committed to take on recurrent costs of community schools supported by the Project from the third year of each school’s operations, as reflected in a legal covenant. Despite being raised at several levels by the implementing agency and the Bank, from 2009 onwards, GoB only provided recurrent funds sufficient to cover teacher salaries. Non-salary recurrent funds had to be raised by communities themselves, or from reduced teacher salaries. Teacher salaries have not been raised in six years6, making teacher retention at the current salary level difficult. (iii)Ineffectiveness of the BEF Board. The BEF Board did not meet as frequently in 2013 and 2014 as before. The ICR team has not found any satisfactory reason for this. The less frequent meetings have resulted in the delay of important actions such as hiring of BEF staff, hiring of an internal audit firm, and implementation of the satellite monitoring initiative. The position of Managing Director (MD), BEF, remained vacant from August 2013 to January 2014 resulting in delays in funds transfer to PECs for school construction as the MD is the principal signatory for all PEC transfers. 2.2.4 Quality of Supervision: Quality of Supervision is rated Satisfactory. This assessment is based on (i) close supervision by the Bank team in a difficult security environment; (ii) timely identification and response to implementation issues; (iii) attention to project sustainability; and (iv) intensive capacity development of BEF and IPs. As noted above, a QALP conducted in November 2008 found the quality of supervision to be satisfactory. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 2.3.1 M&E Design: M&E arrangements designed for the Project were adequate and appropriate for generating data to assess the success or failure of the different models of 6 At the time of Project closing. 8 education delivery tested under the Project. Five key PDO-level outcome indicators and three Intermediate Results (IR) Indicators were identified to monitor progress towards achieving the PDO. Information on the indicators was to be measured regularly by BEF’s IPs and reported to BEF at specified intervals. BEF Field Supervisors were to monitor the operation of private schools and conduct spot checks of at least 20% of community schools every quarter. TPVs were appropriately embedded in the M&E framework to verify and provide an additional layer of monitoring. All project data and information were to be transmitted to the M&E Unit at BEF which would be responsible for validating and incorporating data into a computerized education management information system. Data were to be used to assess the effectiveness of BESP components, inform adjustments in implementation strategy and refine the Project’s design. 2.3.2 One issue identified in the design of the M&E framework is that the measurement of the fifth PDO indicator, i.e., annual improvement in learning achievement, is not explained in sufficient detail, given the highly technical nature of standardized testing. From the PAD, it would seem that CIPs were to monitor the quality of learning and report this annually to BEF. Given the high cost and technical complexity of developing and administering standardized tests on an annual basis, the method and frequency of collecting student achievement data were unrealistic. 2.3.3 M&E Implementation: The M&E system was implemented as planned for the majority of project indicators. A baseline for all PDO and intermediate results indicators was established on time, with delays only in the establishment of a baseline for student learning. 2.3.4 IPs prepared quarterly monitoring reports against targets assigned to them and submitted these to BEF. Staffing issues faced by the M&E Unit for field monitoring were resolved by 2010, after which monitoring by BEF remained regular, with over 20% of schools visited each year until project closing. BEF Field Supervisors monitored private schools until this component reached its conclusion in December 2010. The M&E Unit, BEF, set up a computerized database, BEF’s Management Information System (MIS), for collecting project data early during the Project. The Unit also initiated the dissemination of a newsletter to communicate the Project’s activities and achievements. 2.3.5 A major milestone achieved through the Project was the assignment of BEMIS codes to each Project-supported community school in April 2013. This ensured that community schools were included under the overall provincial education sector Management Information System (MIS) and in the census reports. 2.3.6 However, implementation of the M&E system as designed was incomplete in the area of student learning assessment. Student learning assessments were not carried out on an annual basis, despite a PDO indicator for annual increases in Language and Math learning achievements. Although national expertise was engaged to design and implement surveys for learning achievement in project schools, only three rounds were held in 2007 (baseline), 2008/2009, and 2013. It should be noted that 2013 round used Early Grade Reading Assessment (EGRA) tools which could not be used to assess improvements in student learning. Although it yielded important information on student learning, it made uncertain the achievement of the PDO indicator that year. The ICR team feels that the technical nature of learning achievement measurement, as well as the high level of attention that the BEF dedicated to the construction component of the Project, discouraged a regular administration of the learning assessments as envisioned at project design. 9 2.3.7 As per appraisal, BEF had to establish a website that included complete data on all project components including the selection process of IPs. The BEF established a website for the Project which was intermittently updated. The website page did not include complete data on all project components, including the selection process of IPs. BEF also established a complaint management system in 2008. Through this mechanism, BEF received 44 complaints of which 43 were addressed. Almost half the complaints were related to PEC performance, teacher absenteeism, and school management. Over one-third of complaints were related to school construction. At the time of this ICR, BEF was investigating the remaining complaint. 2.3.8 M&E Utilization: M&E utilization is found to be satisfactory. Encouragingly, data generated by BEF’s M&E system was not limited to the monitoring of implementation progress. Analysis of the data: (i) allowed design readjustments to the Project; (ii) revealed particular issues for further investigation by BEF for which reports and studies were carried out including on low female enrollment in selected regions of the province, performance comparison of local vs. non-local teachers in community schools, community schools’ drop- out and the sustainability of PIPs. 2.4 Safeguard and Fiduciary Compliance 2.4.1 Safeguard and fiduciary compliance are rated Moderately Satisfactory based on the following assessments: 2.4.2 Environmental aspects: The Project was given an environmental category of B at the concept review stage. The Bank’s safeguards policy OP 4.01 required an environmental assessment for BESP’s proposed activities, such as the construction of school buildings, which could have had low to moderate, short-duration and minor negative environmental impact. Other anticipated project activities were expected to have minimal environmental impact. The construction of community school building required conformity with standard engineering design requirements to avoid safety hazards associated with earthquake and floods as well as to ensure proper ventilation and lighting in the school buildings. Some other minor environmental impacts included issues such as a scarcity of water during construction and improper waste disposal. These were characterized as ‘low negative impacts’. 2.4.3 BEF prepared the Environmental Management Framework (EMF) at the design stage of the Project. BEF was also responsible for implementation of the EMF and the Project’s compliance with it. Implementation of the EMF in the field was the responsibility of the PEC and IPs which ensured compliance with the EMF’s environmental guidelines. 2.4.4 BEF trained PEC and IPs engineers on the EMF. All PECs were trained in environmental aspects. Due to high turn-over, training was not conducted for new PEC members as the membership changed from year to year. The BEF did not recruit an Environmental Coordinator for the Project. This responsibility stayed with BEF’s Director (Operations) as an additional responsibility and, later, with the Manager (Construction) as an additional charge until project closure. 2.4.5 An area of significant accomplishment was in school design and construction using roller-compacted concrete (RCC) structures. These are stronger and less vulnerable to earthquakes. Boundary walls were built with prescribed heights for 189 schools out of total 219 schools constructed under the Project. Boundary walls were not built for the remaining schools because by the time construction reached this phase, contracts were closed due to 10 Project closure. Boundary walls made it safer for children to attend school and helped increase girls’ enrolment. As required in the EMF, the Project also conducted TPV for the implementation of the EMF. 2.4.6 Social aspects: Communities provided land for schools in rural areas free of cost. Successful implementation of BESP contributed to a significant increase in girls’ enrollment in rural areas. Out of a total of 48,354 students enrolled in project schools at closing7, female enrollment comprised 42% of community school enrollment and 34% of private school enrollment. 2.4.7 The Project extended quality education to underserved communities in rural, peri- urban and urban areas across all 31 districts in the province. Project-supported schools (under Components 1 and 2) have increased overall enrollment in the province by 5%, and girls’ enrollment by 5.3%8. 2.4.8 Financial management: Financial management compliance is rated Moderately Satisfactory based on the following assessment: Financial management capacity in Balochistan is generally weak with ‘high risk’ rating. The project design included measures to mitigate these risks, such as the training of IPs, and payments to IPs on their achievement of milestones as agreed in the Partnership Agreement and through regular monitoring by BEF. Overall, financial management staffing and arrangements remained adequate and in compliance with annual audit throughout the life of the Project. CIPs closely monitored PECs and community schools. The external audit firm reviewed the performance of all IPs on a sample basis on an annual basis and IPs had their accounts audited annually. However, two issues persisted during the Project, i.e., delays in putting in place internal audit arrangements within BEF, and failure by the GoB to release the non-salary budget for community schools (see paragraph 2.2.3). 2.4.9 Procurement: Procurement compliance is rated Satisfactory. Selection of all IPs under the BESP was carried out in accordance with the Bank’s guidelines. BEF developed a procurement plan for project implementation which provided a basis for the choice of procurement methods and review thresholds. This plan was agreed between BEF and the Bank. The procurement plan was updated annually or when required in agreement with the Bank. 2.4.10 PECs were given grant funds for community school construction. There was no centralized procurement of building material or for contracting school construction at the BEF level. The design of community schools buildings was done by a consulting firm hired by BEF through competitive bidding. Individual Engineer Consultants (IECs) were hired to facilitate, supervise and validate school construction in accordance with designs provided by BEF. IECs facilitated the PEC in identifying potential areas for the procurement of labor and materials and trained communities in skills to support construction. PECs either (i) contracted construction labor and material on a competitive basis, or (ii) contracted labor and procured construction material themselves. The joint supervision of school construction by consultants and the local communities provided a highly effective oversight model. 7 Before support to schools under Component 2 reached its conclusion, enrollment in private schools supported under the Project was 22,381 students. At the end of the Project, enrollment in community schools was 25,973. 8 BEF calculations using BEMIS Annual Census 2013/14, BEF data 11 2.4.11 A lack of procurement staff with relevant expertise led to minor delays in the award of contracts. Procedural difficulties were also encountered for obtaining approval through the BEF structure which hindered the efficient and timely procurement of goods and services. 2.4.12 The institutional capacity of BEF, CIPs, PIPs and PECs was strengthened through training. The capacity of IPs was built to follow procurement procedures agreed between the BEF and the Bank. Third Party monitoring of the IPs ensured compliance with agreed deliverables. PECs were made accountable to communities through disclosure of school procurement and financial matters. 2.5 Post-completion Operation/Next Phase 2.5.1 Post-completion Operation: Community schools will continue to be monitored by BEF and their functionality is expected to continue provided their recurrent costs are met by the GoB. BEF is piloting the use of smart phones to engage with students and teachers on a regular basis. Please see Section 0 – Risk to Development Outcome for a discussion on sustainability of project outcomes and activities. 2.5.2 Next Phase: A clear indication of the success and ownership of the community- supported school model in the province is the GoB’s use of community involvement in expanding access to the public school system. 2.5.3 The Bank’s response to the GoB’s request for continued support to the education sector was the Promoting Girls’ Education in Balochistan (PGEB) project, which became effective in September 2012. It is funded by the Multi-Donor Trust Fund (MDTF) for Khyber Pukhtunkhwa, Federally Administered Tribal Areas and Balochistan. PGEB which is a US$10 million grant to the GoB, aims to improve the availability and accessibility of education services for all children, with a special emphasis on girls, in the province of Balochistan. Community-supported mechanisms similar to those used under BESP are used by the GoB in its efforts to rehabilitate shelter-less schools to improve their functionality, improve school physical infrastructure and construct new schools in areas where no schools are available. The implementation of all activities is based on active community participation at all levels of project design and implementation. 2.5.4 In March 2014, the GoB submitted an application for US$34 million in grant financing from the Global Partnership for Education for the Balochistan Education Project (GPE-BEP) prepared by the Bank as supervising entity for Balochistan under the GPE. This application was approved by the GPE Board and is to be negotiated with the GoB in 2015. The proposed development objective of the Project is to increase school enrollment and retention in project-supported schools, with a special focus on girls’ participation, and to develop mechanisms for information collection and use for the improved management of education. Under the project, 2,000 public schools will be established with community partnership and involvement in school management. 2.5.5 Several design aspects have been built into both PGEB and GPE-BEP based on lessons learnt from BESP: (i) the schools established are recognized upfront as ‘government schools’ using successful community support mechanisms, as opposed to being set up as ‘community schools’; (ii) contract teachers hired for newly established community schools set up under PGEB are employed using rules and procedures applicable to regular 12 government teachers, ensuring they can be regularized onto the GoB payroll after the life of the projects; and (iii) supervision support must be made available to the government for better management of construction related activities. 3 Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 3.1.1 Project objectives remained highly relevant during the life of the Project, and were well-aligned with the Government of Pakistan’s development and education sector priorities articulated in the PRSP II (Government of Pakistan, 2009) (Pillar VI, Chapter 9) and the Bank’s Country Partnership Strategy (FY10-13 and extension to FY14). 3.1.2 Objectives: BESP’s objective to increase access to quality primary education, in particular for girls, remains highly relevant and consistent with the Government of Pakistan’s current development priorities as articulated in the draft five-year development vision, Vision 2025, and the Bank’s Country Partnership Strategy (CPS) for FY2015-19 for Pakistan. The four priorities of Vision 2025, Energy, Economy, Extremism, and Education, are anchored within the CPS as results areas. The objective of the Project is well-aligned and specifically relevant to Result Area 3, ‘Inclusion’, which supports inclusive growth by reducing inequities for vulnerable groups, including women and youth, and those in poor or conflict-affected areas including Balochistan, Khyber Pukhtunkhwa (KPK) and Federally Administered Tribal Areas (FATA). It is also aligned to CPS (FY2015-19) Result Area 4, ‘Service Delivery’, which supports the acceleration of improvements in service delivery, including education. The project objectives are strongly relevant to the achievement of monitoring indicators under Outcome 3.2, ‘Reduced Vulnerability of Groups at Risk’ (Result Area 3; page 21 CPS FY15-19) and Outcome 4.3, ‘Increased School Enrollment and the Adoption of Education Quality Assessment’ (Result Area 4; page 24 CPS FY15-19). 3.1.3 Design and Implementation: The design and implementation of the Project remain relevant in the current context in Balochistan. Sector and provincial challenges identified at the time of design remain unchanged except for a worsened security situation. The GoB’s and Bank’s response to these challenges builds on BESP design and lessons learned from its implementation. These are evidenced by the adoption of community-supported mechanisms in public education service delivery in two subsequent Bank-supported projects, i.e., PGEB and GPE-BEP. 3.2 Achievement of Project Development Objectives 3.2.1 Achievement of the PDO was to be measured using five PDO level outcome indicators (paragraph 0.). By the end of the Project, 633 community schools and 197 private schools had been established. Activities under Component 2 – Support to New Private Schools – were completed by 2011. Details of component performance are available in Annex 2 – Output by Component. 3.2.2 Four out of five PDO indicators were exceeded. Most PDO indicators were maintained above their target level for the duration of the Project, with the exception of that for female enrollment in private schools (see paragraph 3.2.4). Highlights include: 13 i. PDO 1 – An additional 35,000 students (at least 40% girls) enrolled in project- supported schools, was met well ahead of time in 2009. Project-supported schools (under Components 1 and 2) have increased overall enrollment in the province by 5%, and girls’ enrollment by 5.3%9. Targets for enrollment in both community schools and private schools were exceeded by 33% and 38%, respectively, by Project end, the latter despite a revision of Component 2 targets (new private schools to be established was decreased from 300 to 200 at MTR - see paragraph 1.7.2). Female enrollment in community schools was maintained at above 40% throughout the Project and stood at 42% at project closing. ii. Average student attendance, teacher attendance and student grade completion rates for both community and private schools were maintained consistently above their target levels for the duration of the Project and at project closing. 3.2.3 There is insufficient data available to assess the fifth PDO indicator, ‘Annual Improvement in Learning Achievement’. Assessments conducted over 2008/09 showed an increase of 16.17% and 9.60% in mean scores of student learning achievement in Language and Mathematics, respectively, from the 2007 baseline, exceeding annual targets. However, as mentioned in paragraphs 2.3.1 and 2.3.5, student assessments were not carried out on an annual basis. An assessment of student learning outcomes at Grades 2 and 5 in BESP community schools was carried out in 2013/14 using Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA) tools to measure learning in Urdu and Mathematics. While this assessment yielded important information on learning outcomes, it did not measure improvements in learning achievement and hence cannot be used to measure PDO indicator progress. Student Learning Assessment: Oral Reading Fluency and Basic Numeracy in Community Based Schools in Balochistan, Pakistan (2014) Key findings from the study show that the oral reading proficiency and basic numeracy skills of both grades two and five children are weak and significantly lower than expected levels commensurate with their grades. Children’s oral reading fluency in both grades two a nd five is significantly lower than recommended benchmarks of 45 and 100 correct words per minute (CWPM), respectively. Less than 5% of children in grade two and only 6% in grade five can correctly read a grade-appropriate paragraph. Similarly 7 percent of children in grade two and 13 percent in grade five have the capacity to write a sentence in Urdu without a mistake. While children in grade five on average read twice as many correct words per minute as grade two children — indicating positive, but inadequate progression between grades two and five—their oral reading fluency is significantly lower than the recommended benchmark of over 100 CWPM. Boys tend to outperform girls by a small but significant margin in grade two. However the difference diminishes in grade five. Children’s basic computational and problem solving skills are weak across the province. Boys tend to have significantly stronger foundational mathematical skills than girls. Significant differentials exist between boys and girls at both grade two and five levels reflecting that girls’ learning outcomes in mathematics are weaker than boys. 3.2.4 Female enrollment in private schools fell below the target of 40% . The reason for this offered by the BEF and Bank team that private schools were co-educational and girls dropped out after Grade 4 due to the prevalence of male teachers in private schools is found to be incomplete by the ICR team. Data from December 2010 (a few months before the conclusion of Component 2) shows that boys and girls enrolled in private schools have 9 BEF calculations using BEMIS Annual Census 2013/14, BEF data 14 similar drop-out rates of 10% and 9%, respectively. Female teachers constituted almost 53% of the teaching force at these schools. This issue clearly warranted further investigation given that female enrollment in community schools, which were located primarily in rural areas, was maintained at over 40%. 15 Table 2: Progress on PDO Indicators End of PDO Indicator Project Jun-07 Jun-08 Jun-09 Jun-10 Dec-10 Sep-11 Dec-12 Jun-13 Jun-14 target 1 - Total enrollment of students in primary schools (community supported and private) 12,582 27,216 47,656 49,053 50,214 27,687 26,300 26,648 25,973 35,000 Enrollment in Community Schools 7,000 16,783 27,435 27,610 27,833 27,687 26,300 26,648 25,973 % girls' enrolled 49% 52% 44% 44% 43% 43% 43% 42% 40% Enrollment in Private Schools 5,582 10,433 20,221 21,443 22,381 N/A N/A N/A N/A % girls' enrolled 48% 36% 35% 34% N/A N/A N/A N/A 40% 2 - Average student attendance rate Community Schools N/A 87% 86% 84% 81% 78% 85% 88% 85% Private Schools N/A 89% 87% 87% 86% N/A N/A N/A N/A >70% 3 - Average student grade completion rate Community Schools N/A 70% 74% 75% 78% 80% 76% 83% 74% Private Schools N/A 89% 87% 87% 85% N/A N/A N/A N/A >70% 4 - Average teacher attendance rate Community Schools N/A 95% 96% 94% 95% 91% 93% 96% 95% Private Schools N/A 95% 95% 96% 94% N/A N/A N/A N/A >90% 5 - Annual Improvement in Learning Achievement Language N/A N/A 16.17% N/A N/A N/A N/A N/A >3% See para 3.2.3 Mathematics N/A N/A 9.60% N/A N/A N/A N/A N/A >5% 16 3.2.5 Achievement of the PDO is found to be Moderately Satisfactory. Based on the assessment above, the Project contributed significantly to increasing access to primary education in Balochistan through its support to the establishment of community- and private sector-supported schools in all 31 districts of the province. Student and teacher attendance and student grade completion rates were maintained well above target values for the entire life of the Project. This is commendable in a province characterized by governance, security and poverty challenges. However, given the shortfalls in targets for female enrollment in private schools and the lack of data on student learning outcomes, a rating of Moderately Satisfactory is deemed appropriate for achievement of the PDO. 3.3 Efficiency 3.3.1 A comparison of the cost of education delivery at appraisal based on 2004/05 data estimated that recurrent expenditure per student in government schools would be less than half (US$44.78) than expenditure per student in community schools, which would cost US$98.26 (including monitoring costs) due to project low student-teacher ratios (STR). Costs would equalize (at US$44.78)) if community schools had an STR of at least 25 (the STR for government schools was 43). Per student recurrent expenditure was projected to double for community schools in thinly populated areas where the STR was expected to be only 15 (page 74, PAD). 3.3.2 Based on similar analysis conducted by the ICR team using actual data from FY14, the community school model is found to be more cost-efficient than delivery by the government school system. However, in the absence of a comparison of quality at community schools and government primary schools, overall efficiency cannot be assessed. Government primary schools in Balochistan are delivering primary education at a recurrent cost of US$104.91 (PKR 10,729) per student per year (in FY14). In comparison, the recurrent cost of delivering primary education per student per year under the Project community schools (in FY14) with monitoring through external organizations is US$69.15 (PKR 7,072), which is 34% less than government costs. The cost, excluding monitoring through external organizations, is US$37.52 (PKR 3,838)which is 64% less than the cost of delivering primary education through government schools. Please see Annex 3 for more details, including the fiscal implications of the community schools on the GoB. 3.4 Justification of Overall Outcome Rating 3.4.1 The overall outcome is rated Moderately Satisfactory to reflect the following: (i) the PDO and design remained highly relevant throughout project implementation and continue to remain relevant, as can be seen in subsequent operations; (ii) almost all key PDO and intermediate results indicators were met and exceeded and the Project made significant contributions to increasing access to primary education in the province; and (iii) the Project’s model of delivery (under Component 1) was found to be more cost-efficient than the alternative, i.e., government provision of primary education, although overall efficiency cannot be assessed in the absence of a reliable comparison of education quality. Finally, the achievement of the BESP’s objectives in a security environment which steadily worsened since the time of project design is commendable. 17 3.5 Overarching Themes, Other Outcomes and Impacts 3.5.1 Poverty Impacts, Gender Aspects and Social Development. BESP had a direct impact on increasing enrollment in the province, specifically in areas where there was no school in a one kilometer (Component 2) or two kilometer (Component 1) radius. Project- supported schools (under Components 1 and 2) have increased overall enrollment in the province by 5%, and girls’ enrollment by 5.3% 10 . The provision of these schools led to increased enrollment from urban, peri-urban, and rural poor households. Community mobilization and its involvement in the selection of PECs are likely to have had a beneficial impact on people’s perception of schooling and generated increased social demand for education. A high teacher attendance rate likely contributed to consistent enrollment of students in project schools over the life of the Project, despite the fact that the majority of community schools are still without a building. The construction of buildings for 219 community schools and the provision of free textbooks by GoB in the last two years of the Project are also likely to have contributed to increases in stabilization of enrollment, especially for girls. BESP also had an impact on gender equity. The percentage of female enrollment at the end of the Project was 42%, higher than the government primary school average of 39%11. The construction of boundary walls in community schools which were provided with a building is likely to have had a positive impact on female enrollment. 3.5.2 Institutional Change/Strengthening. BESP implementation resulted in significant institutional changes and made progress towards strengthening the capacity of BEF and its partners. For instance: i. During project design, amendments were made to the BEF Act to increase private member representation on its Board to mitigate risks stemming from political interference. ii. An enabling environment was created for the private sector. All private schools were established in low-income communities where private operators were unlikely to open schools without BEF support. All private school operators and teaching staff were trained in private school financial management and administration, record keeping and managing relationships with parents among other topics. iii. BEF’s capacity was enhanced through Component 3 to expand its operations into low enrolment areas, evidenced by its presence in remote, rural areas in all districts of Balochistan. iv. PECs were trained to deliver school management, financial management, record keeping and, for some, school construction and functioning. This contributed to improvements in governance at the community level. v. Under the Project’s capacity development component, 1,100 teachers were trained (764 community school teachers and 336 private school teachers). Training included on-site classroom support and multi-grade teaching modules, and teaching kits were provided to all community schools. These initiatives are expected to have had positive impacts on teacher professional development and overall teaching quality. 10 BEF calculations using BEMIS Annual Census 2013/14, BEF data 11 BEMIS 2013-14 Annual Census data 18 vi. BEF and CIP professional staff members were trained in project management, financial management, social mobilization, human resource management and M&E areas. 3.5.3 Other Unintended Outcomes and Impacts (positive or negative). There are no other unintended outcomes or impacts other than what has been reported. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 3.6.1 Not applicable. 4 Assessment of Risk to Development Outcome 4.1 The risk to development outcome is rated High based on the following assessment:  There are positive signs of sustainability of project outcomes and delivery mechanisms: (i) the adoption of the community-supported model of education service delivery by the GoB (see Section 2.5); and (ii) 75% of schools established under Component 2 were still operational over three years after support to them ended12. However, several risks, if realized, threaten the sustainability of BESP outcomes. These include: (i) withdrawal of GoB funding for community schools’ recurrent expenditure (non-salary recurrent funding has not been provided to community schools since 2009); (ii) dwindling GoB commitment for supporting a community-based model of education service delivery once ongoing support to the GoB concludes; (iii) low teacher retention due to non-competitive salary increases over time; (iv) a decline in school-level and community-level capacities for effective school management once capacity building activities are no longer externally funded; and (v) a worsening security situation which can result in school closure.  The risks to development outcome sustainability which have been identified are mitigated, in part, by the Bank’s ongoing dialogue with the GoB as part of two education projects (BPEP and GPE-BEP) in the province, and options to promote sustainability of project outcomes were discussed in detail prior to project closing with BEF, SED and the GoB. No decision on the future of the community schools established under Component 1 has been taken as yet. Of concern, however, is that the options discussed, including the regularization of community school teachers and the handover of community schools to the Secondary Education Department (SED), entail the sacrifice of PEC and thus community authority over teacher recruitment and termination. This could potentially dilute the success of the BESP delivery model and instead allow governance issues which afflict the public sector service delivery system to take root. 5 Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry 5.1.1 The Bank’s performance in ensuring quality at entry is rated Satisfactory based on the following assessment: 12 Based on a sample-based sustainability review conducted by BEF in end 2013. 19 i. The PDO was highly relevant and remained so throughout the Project, as discussed in Section 3. The PDO was well aligned to the Bank’s CPS, and GoB and Government of Pakistan priorities in the sector. ii. The preparation task team incorporated the advice and lessons learned from other similar Bank-financed operations, as well as guidance received throughout project preparation, including from a QER held in 2005. Technical aspects of the Project were well-developed and appropriate for sectoral, institutional and policy constraints in the province and to address capacity limitations among implementing partners. iii. M&E arrangements designed for the Project were adequate (with the exception of one indicator; see paragraph 2.3.1), with appropriately built in and timed third party monitoring. iv. Risks (see Sections 2.1 and 0) were aptly identified at entry and the mitigation measures proposed were adequate. v. Fiduciary and social and environmental aspects of the Project were well-designed and aligned to the Project’s design. 5.1.2 BESP was covered under a Quality Assessment of Lending Portfolio conducted in November 2008. The Assessment found the Quality of Design to be Highly Satisfactory based on findings similar to those described above. It highlighted two main shortcomings with the design: (i) there was no economic analysis or cost-benefit analysis conducted to provide a basis for the Project; and (ii) international TA should have been included under the Project to work with BEF and local IPs to set good implementation standards. The ICR team feels that while there are merits to the first point, it is highly unlikely that an international firm would have been able to provide support to IPs in Balochistan given the challenging security situation. (b) Quality of Supervision 5.1.3 The Bank’s performance with regards to quality of supervision is rated Satisfactory based on the following assessment: i. The Bank team was comprised of an appropriate skills mix (education, fiduciary, environmental and social experts, economics/poverty reduction experts, engineers) for supporting and supervising the Project. Missions were conducted regularly, and during times of heightened security risks, the team did a commendable job of using alternatives to supervise and monitor the Project (videoconferences with BEF, holding missions in Karachi/Islamabad, using third party monitoring services). ii. Risks and implementation issues were identified and addressed in a proactive manner, and candidly highlighted and discussed in all supervision documentation (e.g., ISRs, AMs). Project ratings are found to be realistic and in line with implementation progress. The Bank team maintained a strong focus on the achievement of the PDO throughout the life of the Project. iii. Attention was paid to the sustainability of Project initiatives from an early stage of implementation, which contributed significantly to the design of follow-on operations in the sector in Balochistan. 5.1.4 The QALP assessment conducted in November 2008 found the Quality of Supervision to be Satisfactory based on similar findings to those described above. It 20 highlighted one main shortcoming with project supervision, i.e., delays and inconsistencies in support from the environmental safeguards team. However, this issue was eventually resolved and the level of support provided to the Project once the construction of community schools commenced has been found to be acceptable. Justification of Rating for Overall Bank Performance Rating: Satisfactory 5.1.5 The Bank’s overall performance is rated Satisfactory based on the assessments above. 5.2 Borrower Performance (a) Government Performance 5.2.1 Overall, the GoB showed a high level of commitment to and ownership of activities supported under the Project and to the achievement of the PDO. At the design stage, the GoB facilitated amendments to the BEF Act to allay risks stemming from political interference by increasing private member representation on the BEF Board. During implementation, the GoB maintained close oversight of the Project and met regularly with visiting Bank missions. Also, as mentioned in paragraph 2.2.1(i), the DoE’s recognition of community schools and the allocation of BEMIS codes to each, the regular transfer of teacher salaries to PECs (with a few exceptions) and the provision of free textbooks to community school students and teacher training to community school teachers serve as further demonstrations of the GoB’s commitment. 5.2.2 Government performance was found lacking in one critical area, i.e., the undertaking of financial responsibility for operation and maintenance of project community schools from the third year of each school’s operation, as committed by the GoB through a legal covenant contained in the BESP Legal Agreements. The GoB only provided recurrent funds sufficient to cover teacher salaries. Non-salary recurrent funds were raised by the communities themselves, sometimes at the expense of reductions in already-stagnating teacher salaries (see paragraph 2.2.3(ii)). Despite repeated commitments by the GoB to address this issue, non- salary recurrent funds were only provided once in April 2011. In addition, the GoB did not allocate land to BEF for the construction of a new building to be funded under the Project. 5.2.3 Given the risk posed to the sustainability of community schools established under the Project due to the non-provision of adequate non-salary recurrent funds, Government performance is rated as Moderately Unsatisfactory. (b) Implementing Agency or Agencies Performance 5.2.4 Implementing Agency performance is rated as Satisfactory. Overall, BEF demonstrated strong commitment and ownership to achieving the BESP development objectives and coordinated with the provincial government, in particular the Department of Finance and DoE. In its role as the implementing agency for the Project, BEF: (i) implemented several models of delivering primary education to 48,629 students, achieving targets under most PDO indicators; (ii) proactively identified, resolved, and reported implementation challenges that arose during the Project, especially in a challenging security environment; (iii) efficiently verified, analyzed and shared data collected by the IPs in a timely manner to concerned stakeholders, including the Bank task team; and (iv) complied with the Bank’s fiduciary and safeguards requirements. BEF performed consistently well 21 throughout the life of the Project, albeit with some challenges in the last two years of implementation: there were delays in important actions such as hiring of BEF staff due to infrequent meetings of the BEF Board, and delays in funds transfer to PECs for school construction due to a vacant Managing Director position (see paragraph 2.2.3(iii)). (c) Justification of Rating for Overall Borrower Performance 5.2.5 Overall Borrower performance is rated Moderately Satisfactory, given the assessments above. While the performance of BEF in a context such as Balochistan is commendable, as is the achievement of the PDO, the risk posed to the sustainability of community schools established under the Project is significant enough to warrant the rating assigned by the ICR team. 6 Lessons Learned 6.1 Significant lessons have been learned from BESP’s experience, several of which have been included in the design of PGEB and GPE-BEP:  Government commitment and ownership. Strong government commitment and ownership are critical for successful implementation and sustainability of service delivery models which make use of mechanisms outside the public sector. For instance, the recognition of community schools by the DoE was fundamental to enabling community school students to transition to higher grades in mainstream public schools. Similarly, project objectives are at risk because of a lack of financial commitment by the Government to sustain community schools.  Empowerment of communities can lead to high levels of ownership. Community involvement in designing and monitoring schools cultivated a strong sense of ownership. Communities raised non-salary recurrent funds in the absence of GoB funding, ensured female student enrollment and retention, and helped identify and attract teachers to their communities.  School-based management through the community also allowed for better accountability. Empowering communities with teacher hiring and firing ensured that teacher absenteeism and student drop-out rates were low.  Community school models that are well designed and well-managed are examples of successful public, private and community partnerships. These models are cost- and time-effective and address the issues of out of school children’s access to education.  The community school model tested under the Project proved to be a viable alternative to government service delivery, especially in conflict-prone areas. The Government should play a role in providing adequate resources for models using community-supported mechanisms to provide education service delivery in underserved and conflict-prone areas.  Private sector participation in low-cost education delivery is not as viable in Balochistan when compared to the rest of the country given a lower propensity of households to pay.  Community involvement in designing and managing schools is crucial for increasing children’s access to school, in particular girls, and for decreasing drop-out and reducing teacher absenteeism rates.  Capacity development support is crucial for ensuring timely and effective implementation of projects in provinces and regions with limited public sector 22 capacity, weak private sector presence and deteriorating law and order environments. Continuous engagement and support to community partners under the community-supported model is important to ensure that no knowledge is “lost” due to migration, etc.  Community schools need regular monitoring to remain operational. Low cost technology based monitoring systems need to be explored and introduced in projects that are implemented in conflict-prone and hard to reach terrains.  The public sector’s institutional capacity to measure learning achievement in Language and Mathematics in schools is weak. This capacity needs to be built within public sector institutions that deal with such assessments. 7 Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 7.1 BEF reviewed the ICR and concurred with the assessment. No substantive issues were raised in the Borrower’s ICR. No comments were received from the SED, FD and P&D. (b) Co-financiers 7.2 There were no co-financiers in the Project. (c) Other partners and stakeholders 7.3 Not Applicable. 23 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Actual Percentage of Components Estimate Expenditure Appraisal (USD million) (USD million) Investment Costs Capital Costs for BEF 0.30 - 0% Furniture & Equipment-BEF 0.11 0.19 174% Training and Skill Development 2.26 2.30 102% Set-up & Capital Costs for Schools 6.61 11.24 170% Sub-Total 9.28 13.73 148% Recurrent Costs Salaries BEF 0.73 2.21 302% Schools 3.10 0.61 20% Sub-Total 3.83 2.82 74% Operational Costs BEF 0.81 1.11 137% CIPs' Delivery 3.08 3.35 109% Schools' Non-Salary 3.16 0.57 18% Sub-Total 7.05 5.02 71% Unallocated/ Contingencies (@9%) 1.83 - 0% TOTAL PROJECT COSTS 22.00 21.57 98% (b) Financing Appraisal Actual Percentage Type of Source of Funds Estimate Expenditure of Cofinancing (USD million) (USD million) Appraisal Borrower - 1.55 3.27 211% International Development Association - 22.00 21.57 98% (IDA) 24 Annex 2. Outputs by Component Balochistan Education Support Project Overall Achievement: Four out of five PDO indicators exceeded targets set. These PDO indicators were maintained above target level for the duration of the Project, except female enrollment in private schools. Project-supported schools (under Components 1 and 2) have increased overall enrollment in the province by 5%, and girls’ enrollment by 5.3%13. a) Total enrollment of students stood at 48,354 with 42% girls in community schools and 37% in private schools (Target: enrollment increased by 35,000, with at least 40% girls); b) On average, annual student attendance was maintained above 70% in both community and private schools. At project closing, student attendance was 85% and 86% in community schools and private schools14, respectively (Target: average attendance not less than 70%); c) On average, average grade completion rate was maintained above 70% in both community and private schools. At project closing, the grade completion rate in community schools and private schools15 was 74% and 85%, respectively (Target: average completion rate not less than 70%); d) On average, annual teacher attendance was maintained above 90% in both community and private schools. At project closing, teacher attendance was 95% in community schools and 94% in private schools (2010) (Target: average annual teacher attendance, net of leave entitlement, not less than 90%); and e) The target for increases in learning achievement in Language and Mathematics was exceeded (6.17% and 9.60%, respectively) in 2008/09 (Target: an annual increase of 3% and 5% respectively). Component 1: Establishment of New Community Schools in Rural Areas (US$13.9 million) Status of Targets Comments At project closing: All the targets for Component 1 were met.  Total enrollment in the 633 The target of enrolling an additional 35,000 community schools stood at 25,973 students (at least 40% girls) in Project supported students (Target: Enrolment schools was met well ahead of time in 2009. The maintained or increased above percentage of female enrollment at the end of the 19,500 students in 650 community Project at 42% was higher than the government schools established in the first three school primary school average of 39%16. years of the project). Year wise total enrollment in community schools is  42% of the students were girls presented in the table below: (Target: At least 40% the students are girls.) 13 BEF calculations using BEMIS Annual Census 2013/14, BEF data 14 Data for private schools is from December 2010 when support to this component was concluded. 15 Ibid. 16 BEMIS 2013-14 Annual Census data 25  The average annual student Year Total Enrollment Jun-07 7,000 attendance was 85% and the Jun-08 16,783 average grade completion rate Jun-09 27,435 was 74%. (Target: Attendance Jun-10 27,610 and grade completion rate> 70% Sep-11 27,687 throughout.) Dec-12 26,300  The average annual teacher Jun-13 Jun-14 26,648 25,973 attendance was 95%
(Target: Teacher Attendance > 90%) Average student attendance, teacher attendance,  Mean score in Language and student grade completion rates for community increased by 6.17% and in schools were maintained consistently above the Mathematics by 9.6% in 2008/09. target for the duration of the Project. (Target: Annual learning Achievement Improvement > 3% The Project established 649 community schools for language > 5% for against a target of 650 by July 2011. At Project Mathematics) closing, 633 of these were operational. The remaining 17 community schools were closed due to internal community conflict mainly tribal in nature, unavailability of local teachers or finding a new teacher when the existing one resigns or non- compliance with fiduciary requirements. (Reference: AM February 3, 2011 and AM April 12, 2013). Encouragingly, BEF developed protocols to deal with community school closures. The number of community schools to be constructed using project funds was decreased from the appraisal target of 450 to 225 schools in 2011 due to significant escalation in the cost of construction (approximately 3 times the appraisal estimate). Difficulties in provision of technical support to PECs and CIPs during construction of schools along with a worsening security situation, unavailability of raw material and skilled labor in some communities, and occasional delays in the release of funds to PECs for construction further delayed school construction. By the close of the Project, 219 schools building were constructed. Teacher turnover remained high because the salaries paid to the teachers are about 30% less than the minimum wage rate. Non-salary expenditure for community schools of PKR 3,000 per month has not been released by the GoB from April 2011 onwards. Community schools were formally recognized by the DoE to ensure that school leaving certificates and Grade 5 completion certificates were duly recognized. 26 Component 2: Support to New Private Schools (US$2.1 million) Status of Targets Comments At the conclusion of support to this All the targets for Component 2 were met except component (December 2010): the number of schools and proportion of female  Total enrollment in the 197 new enrollment. The number of private schools to be private schools supported stood at established under Component 2 was revised 22,381 students; 34% of these downwards from 300 to 200 schools during the were girls (Target: 200 new MTR because of the non-availability of private schools supported with enrolment sector partners that could have qualified under the maintained at or increased above selection criteria of this component. 15,500 with and at least 40% of Two sustainability studies of the PIPs were the students are girls.) conducted by BEF in 2011 and 2013. The findings  Total average annual student of these studies highlighted that 75% of the private attendance was 86%; average sector schools funded by the Project were grade completion rate was 85%; operational. and average annual teacher attendance was 94%. Component 3: Capacity Building (US$4.2 million) Status of Targets Comments  1,100 teachers were trained. 764 BESP implementation resulted in significant teachers were trained in institutional changes and made progress towards community schools and 336 strengthening capacity of BEF and its partners. teachers in private schools. 3,245 All private school operators and teaching staff were members of 649 PECs were trained in private school financial management and trained. 100% of BEF and IP staff administration, record keeping, managing was trained. (Target: 2,100 relationships with parents, etc. teachers trained, Members of 650 BEF’s capacity was enhanced through this PEC’s trained, all BEF component to expand its operations into low professional staff and 80% of enrolment areas, evident from its presence in project related IP staff trained.) remote, rural areas in all districts of Balochistan. PECs were trained to deliver school management,  BEF’s capacity through financial management, record keeping, and for Component 3 was enhanced to some PECs, school construction, thus contributing expand its operations into low to improvements in governance at the community enrolment areas and scale up its level. operations as evident from its The target of 2,100 estimated at appraisal was high presence in the remote rural areas as it anticipated a larger number of teachers per of all districts in Balochistan. The school. Training included on-site classroom support TPV highlighted that the and multi-grade teaching modules, and teaching advantage of a capable and kits were provided to all community schools. These experienced Managing Director initiatives had and will continue to have positive coupled with motivated impacts on teacher professional development and managers, remained a core reason overall teaching quality in the province. for successful operations by BEF. BEF and CIP professional staff members were (Target: BEF has successfully trained in project management, financial enhanced its capacity to expand management, social mobilization, human resource its operations into low enrolment management and M&E areas. areas and is in a position to scale It was observed during the ICR Mission that there 27 up its operations as indicated by is limited ability within the GoB to build capacity an independent performance of PITE to conduct Training of Trainers for assessment by a third party.) primary school teachers. Community school teachers who were hired as replacements for teachers who resigned were not trained. Similarly, PEC members who replaced the members who left PECs were not trained. 28 Annex 3. Economic and Financial Analysis PAKISTAN: BALOCHISTAN EDUCATION SUPPORT PROJECT Fiscal Impact of Community Schools established under the Project on GoB’s Current Budget The establishment of community schools under BESP (Component 1) was the only component that had and will have fiscal implications for the current expenditure of the Government of Balochistan (GoB). The Project funded all current costs associated with new community schools through the BEF. The Bank funded these costs for the first two years of operations of each newly established community school. The GoB agreed to finance current costs of community schools from its budget via BEF from the third year of each school’s operation. The analysis that follows looks at the fiscal impact on the GoB’s budget for supporting these schools. 633 schools were completed over the life of the Project and all have now been shifted to the GoB for the financing of operational costs. These include teachers’ salaries and fixed non-salary expenditures per school per month. The total cost to the GoB of these schools in FY 14 was PKR 76.88 million (US$0.75 million) under Scenario I. This scenario is based on the actual release of funds by the GoB for both salary and non-salary expenditures over the life of the Project. The total cost to the GoB of these schools in FY 14 was PKR 85.58 million (US$0.84 million) under Scenario II. This scenario is based on the actual financial requirements of these schools irrespective of whether these were released or not by the GoB. This includes the actual teacher salary and the non-salary expenditures as agreed by the GoB. Based on the GoB’s total current expenditure, total current expenditure on education, total current expenditure on primary education (Table 1 below) and the total cost of maintaining BESP’s Community Schools under the two scenarios discussed above (Tables 2A and 2B below), it is estimated that the GoB spent PKR 75.6 billion (US$739.3 million) on education in FY 14. Of this, PKR 9.3 billion (US$90.9 million) has been spent on primary education. Based on these figures, the GoB has spent 0.26% of its education expenditure and 0.83% of the primary education expenditure in FY 14 for maintaining these schools under Scenario I. The GoB would have spent 0.29% of its education expenditure and 0.92% of the primary education expenditure for maintaining these schools under Scenario II. In both these scenarios, the cost to the GoB is insignificant when compared with GoB’s total current expenditure on education and on primary education. 29 Table 1: Balochistan Education Expenditure (PKR Million) 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 2013/14 Total Balochistan Government Expenditure 13,540 22,236 27,370 30,375 49,515 58,415 65,244 75,608 (Current PKR Million) Total Balochistan Education Expenditure 6,611 7,072 8,760 10,955 18,483 22,289 26,601 29,978 (Current PKR Million) Total Balochistan Primary Education 2,420 2,452 3,307 4,019 5,576 6,001 8,570 9,305 Expenditure (Current, PKR Million) Source: Poverty Reduction Strategy Paper (PRSP) Progress Reports, Ministry of Finance, Government of Pakistan. 30 Table 2A and 2 B: Fiscal Impact on the GoB's Education Budget (Current) for funding the BESP Community Schools Table 2A (Scenario 1): Based on release of funds by the GoB (PKR Million) No Component 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 2013/14 Years I. Community Schools 1 Number of new schools set up 191 434 649 643 633 633 633 633 2 Number of students enrolled 7,000 16,783 27,435 28,000 26,892 26,301 26,136 25,973 3 Number of teachers employed 200 434 700 747 755 764 764 764 4 Government of Balochistan * I Recurrent costs 0.00 0.00 6.36 31.20 48.52 71.88 76.88 76.88 A Salary 0.00 0.00 3.98 18.96 48.52 71.88 76.88 76.88 B Non-Salary ** 0.00 0.00 2.38 12.24 0.00 0.00 0.00 0.00 Cost of CIP for Delivery 5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 (10% of Recurrent Costs) *** 6 Total Government of Balochistan 0.00 0.00 6.36 31.20 48.52 71.88 76.88 76.88 * Figures are reported on basis of actual releases by the GoB during different financial years. ** PKR 3,000 per month per school was agreed by the GoB as non-salary expenditure. This was released only in FY 2008-09 & 2009-10. *** No releases have been made by the GoB under this category over the life of the Project. 31 Table 2B (Scenario 2): Based on required operational cost (Not linked to GoB financing) (PKR Million) No Component 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 2013/14 Years I. Community Schools 1 Number of new schools set up 191 434 649 643 633 633 633 633 2 Number of students enrolled 7,000 16,783 27,435 28,000 26,892 26,301 26,136 25,973 3 Number of teachers employed 200 434 700 747 755 764 764 764 4 Government of Balochistan I Recurrent costs 0.00 0.00 21.28 76.93 77.15 77.80 77.80 77.80 A Salary 0.00 0.00 14.18 53.78 54.36 55.01 55.01 55.01 B Non-Salary 0.00 0.00 7.09 23.15 22.79 22.79 22.79 22.79 Cost of CIP for Delivery 5 0.00 0.00 2.13 7.69 7.71 7.78 7.78 7.78 (10% of Recurrent Costs) 6 Total Government of Balochistan 0.00 0.00 23.40 84.63 84.86 85.58 85.58 85.58 32 Cost of Education Delivery: A Comparison Government primary schools in Balochistan have an average student teacher ratio of 43 and are delivering primary education at a cost of PKR 10,729 (US$104.91) per student per year. Government schools do not provide free textbooks and instructional materials and the non-salary repairs and maintenance budget for primary schools is almost negligible. In comparison, the cost of delivering primary education per student per year under the project community schools is substantially less under similar circumstances. This has been calculated under two scenarios. Scenario I calculates this cost based on actual expenditure borne previously by the Bank and now by the GoB. This includes salary, non-salary expenditure and monitoring costs of these schools by TIPs (NGOs). Scenario II calculates this cost based on actual expenditure borne that includes salary and non-salary expenditure and excludes the monitoring cost of these schools by TIPs (NGOs). The reason for excluding TIPs (NGOs) is that these NGOs now do not have a contract for monitoring community schools due to the non-availability of funds with the GoB/BEF. Community schools established by the Project have an average student teacher ratio of 34. Based on this analysis, the cost of delivering primary education per student per year under the project community schools in both scenarios is given in Table 3 below. Under Scenario I, the cost of delivering primary education per student per year through the Project’s community schools is PKR 7,072 (US$ 69.15), compared to the government school cost of PKR 10,729 (US$ 104.91). This is 34% less than the cost of delivering primary education under government schools. Similarly, under Scenario II, the cost of delivering primary education per student per year through the Project’s community schools is PKR 3,838 (US$ 37.52) as compared to the government school cost of PKR 10,729 (US$104.91). This is 64% less than the cost of delivering primary education under government schools. The cost of delivering primary education with monitoring through TIPs (NGOs) or without monitoring through TIPs (NGOs), remains substantially less than the cost of delivering primary education through government primary schools. If the student teacher ratio in community schools goes down to 22 from 34, the cost of delivery of primary education will match that of government schools, where the current student teacher ratio is currently at 43:1. 33 Table 3: Comparison of Government and Community Based Cost of Education Delivery (PKR) Current Student Children in Total Current Expenditure Type of Primary School Teacher Primary Expenditure per Student Ratio Schools (2013-14) (2013-14) Government Schools (Salaries, with 43 867,282 9,305,000,000 10,729 negligible non-salary recurrent budgets (US$90,984,648) (US$104.91) and no fees) Scenario I: Project Community Schools 34 25,973 183,677,188 7,072 (Salary, non-salary expenditure, and (US$1,796,003) (US$69.15) monitoring cost) Scenario II: Project Community Schools 34 25,973 99,671,400 3,838 (Salary and non-salary expenditure only) (US$974,590) (US$37.52) Project Community Schools with Student Teacher Ratio that matches the per student recurrent expenditure of 22 17,120 183,677,188 10,729 the Government schools (expenditures (US$1,796,002) (US$104.91) on Salaries and non salary (current) and monitoring for community schools) Source(s): (i) For Government Schools Expenditure: Poverty Reduction Strategy Paper (PRSP) Progress Reports, Ministry of Finance, Government of Pakistan. (ii) For Government Primary Schools Enrollment and Student Teacher Ratio, Balochistan Education Management Information System. (Iii) For Community Schools: Balochistan Education Foundation 34 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending Naveed Hassan Naqvi Program Leader ECCU8 Ameer Hussein Naqvi Senior Education Specialist GEDDR Michelle Riboud Consultant GEDDR Tahseen Sayed Country Manager ECCAL Amna W. Mir Senior Program Assistant SACPK Anwar Ali Bhatti Financial Analyst SACPK Rosita Maria Van Meel Senior Education Specialist ECSH2 - HIS Huma Ali Waheed Senior Operations Officer GEDDR Supervision/ICR Javaid Afzal Senior Environmental Specialist GENDR Syeda Madiha Mansoor Ahmed E T Consultant SASHD - HIS Asif Ali Senior Procurement Specialist GGODR Irajen Appasamy Senior Operations Officer GEDDR Amer Zafar Durrani Senior Partnership Specialist ECCKA Qazi Azmat Isa Senior Rural Development Specialist GFADR Nasreen Shah Kazmi Program Assistant SACPK Mohammad Khalid Khan Program Assistant GSPDR Riaz Mahmood Financial Management Analyst GGODR Amna W. Mir Senior Program Assistant SACPK Amir Munir Senior IT Officer, Program Man ITSCR Sheila Braka Musiime Chief Counsel LEGES Ameer Hussein Naqvi Senior Education Specialist SASHD - HIS Michelle Riboud Consultant GEDDR Hasan Saqib Senior Financial Management Specialist SARFM - HIS Tahseen Sayed Country Manager ECCAL Corinne Siaens Economist SASED - HIS Rosita Maria Van Meel Senior Education Specialist ECSH2 - HIS Huma Ali Waheed Senior Operations Officer GEDDR Ayesha Khan Operations Analyst GEDDR Naveed H. Naqvi Program Leader ECCU8 Rehan Hyder Senior Procurement Specialist GGODR Aazar Wali Bhandara Consultant GEDDR (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including travel No. of staff weeks and consultant costs) Lending FY05 25.34 67.11 FY06 54.93 113.45 FY07 0 0.00 35 Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including travel No. of staff weeks and consultant costs) FY08 0 0.00 Total: 85.80 180.56 Supervision/ICR FY05 0 0.00 FY06 0 0.00 FY07 35.31 85.84 FY08 36.33 88.60 FY09 22.50 57.17 FY10 18.56 52.08 FY11 18.57 44.83 FY12 16.56 44.32 FY13 11.82 44.86 FY14 12.67 80.29 FY15 10.63 23.36 Total: 182.95 521.38 36 Annex 5. Summary of Borrower's ICR and/or Comments on Draft ICR The Note of Thanks and Annexes (Fiscal Analysis and List of Supporting Document) have not been included in this Summary and are available in the complete Borrower’s ICR. Executive Summary Balochistan Education Support Project (BESP) started in August 2006 with the support of World Bank financed IDA credit. Government of Balochistan was the borrower and Balochistan Education Foundation was the client of the project in coordination with Government of Balochistan. The project rationale was the improvement and development in the public private and community partnership in the province. The Project implementation mechanism was designed after assessing the lessons learned from the previous project in the name of BPEP (Balochistan Primary Education Project) from 1993 to 1999. Revised mechanism was a unique design which hired local NGOs as CIPs (Community Implementation Partners) private individuals and organizations as PIPs (Private schools Implementing Partners) and public and private institutes as TIPs (Technical Implementing Partners). The organizations collaborated the partnership with BEF for achievement of three set Project Development Objective intermediate outcomes as establishment of 650 Community Schools, 200 private Schools and building capacity of Teachers, Parents Education Committee members, Community Implementing Partners (NGOs), Private schools Implementing Partners (PIPs) and BEF staff. The Project Development Objective (PDO) based on the outcome indicators of enrolling 35000 children with 40% girls, retaining the students’ attendance to 70%, teachers attendance to 90%, students grade completion rate to 70%, and annual increase of 3% and 5% in learning achievement in language and mathematics respectively. All the PDO level outcome indicators were achieved in access by the end of the project except the backlog of 3% in overall target of 40% girls enrollment in first outcome indicator on enrolling the children in community schools. All the Project Development Objective indicators were achieved through three set intermediate outcomes. The project contributes 5% of total Balochistan primary enrollment. The project design introduced a new cost and time effective model to Balochistan Education Department in terms of immediate access to all the deprived population/locations. Planning and Development Department, Government of Balochistan strongly recommends that the design should be adopted for an instant enrollment of an alarming number of out of schools children in Balochistan. Project Background and Context With a population of over 150 million people, Pakistan is the second most populous country in the South Asian region. Economic growth has increased from an average of 3.3 % during 1997- 2002 to an average of nearly 6 % during 2002-05. The FY06 data indicates that economic growth would continue to be strong at about 6.3%. The earthquake in the north of Pakistan had no direct impact on Balochistan, and was expected to have only a marginal impact on economic growth. 37 However, the fiscal impact of the earthquake was expected to be substantial due to large additional expenditure on relief, rehabilitation and reconstruction. Balochistan’s economy differs from that of the rest of Pakistan. It is the poorest of the country’s four provinces, with standards of living and social indicators lagging substantially the rest of Pakistan. With 43% of the total land mass of the entire country it has very low population density with5% of the country’s population. This makes service delivery and infrastructure development extremely challenging. A large proportion of the population lives in small and dispersed rural settlements with only 24 % of the population living in urban areas. Moreover, Balochistan’s financial resources for addressing its development agenda are limited and continued prevalence of governance issues deeply impair development prospects. Education attainment in Balochistan reflected the province’s overall low development indicators. Literacy levels at 37% lagged way behind the other provinces, and the national average of 53%. Some of its districts had one of the lowest rates in the world, with one district recording literacy of 4%, and with only 2% enrollment at primary level. The issues of the sector in Balochistan can be summarized as follows: Low and declining enrollment in primary education: National survey statistics 17 for Balochistan show that Primary Net Enrolment Rate (NER) has increased only marginally from 36% in 1998-99 to 37% in 2004-05. Over this period net enrolment rates have shown an alarming lack of progress for both boys (44% in 1998-99 and 44% in 2004-05) and girls (28% in 1998-99 and 29% in 2004-05). Large gender disparities: Primary gross enrolment rate (GER) for girls, at 49% compares very unfavorably with that of 83% for boys. Female enrolment is 35% of total primary enrolment. Female participation is constrained by insufficient female teachers for girls’ schools, lack of qualified local women who could potentially become teachers, large distances affecting female mobility, and cultural and social issues of a tribal society in rural areas which places less value on girls’ education. Then there were slightly over 40,000 teachers in the government system from primary to higher levels out of which only about 12,000 were female. It had a total of 9734 government primary schools, out of which 6862 were male. The number of schools at the middle level were only 772 (228 for girls), and there were 550 (124 for girls) high schools in the province. Government schools’ enrolment was 707,000 at the primary level, only 100,000 at the middle level, and 40,000 at the high school level. Poor quality of education: Poor teacher training, lack of qualified teachers, especially women, high teacher absenteeism, poor monitoring and supervision, lack of textbooks, overcrowded classrooms in some schools, ineffective use of facilities in others, and outdated pedagogical 17 Pakistan Integrated Household Surveys (PIHS) in 1998-99 and 2001-02, and its successor the Pakistan Social and Living Standards Measurement Survey (PSLM) in 2004-05, the PSLM results are preliminary as raw data is not yet available for a detailed analysis. 38 practices have resulted in very poor quality primary education. Poor quality of teaching has contributed to low levels of student learning Lack of continued community focus in successful community support programs: The spirit of community participation is strong in Balochistan, and builds upon the successful experience of the Community Support Program (CSP) under the IDA supported Balochistan Primary Education Project (BPEP) between 1993-99, which established about 1,300 new girls schools, with 60,000 girls’ enrolment, in partnership with communities. After the end of BPEP, the CSP Schools were converted into government schools under government management. The teachers no longer felt accountable to the communities and as a result the community involvement declined over time. The government system was unable to sustain community participation due to its own institutional constraints. Limited participation of the private sector: Private schools account for about 6 % of enrollment in primary education 18 in Balochistan, compared with 28 % for the country as a whole. The primary GER in government schools in Balochistan at 63 % compares well with the Pakistan average of 62 %. The overall GER of 67 % for Balochistan was lower than that for Pakistan at 86%. While the rest of Pakistan had experienced an expansion in low cost private schools, in Balochistan, private schools had only increased from around 250 schools in 1993-94 to around 1,300 in 2003-04, 19 mainly operating in the urban and semi-urban areas. Private sector expansion into rural areas was limited by financial constraints on both the potential private school operators and the target population. Weak institutional capacity of the public sector: In 2004-05 there were 707,00020 primary school students enrolled in government schools, taught by about 17,000 teachers. The Provincial Authorities at various levels (provincial, district, local) lacked the capacity to plan, regulate and monitor the delivery of quality primary education, especially in the rural areas. Weak capacity has been further affected by a poor governance environment which in turn affected quality of public service delivery. Lack of financial resources: Balochistan has limited revenue generation capacity with low growth of the private sector. This was exacerbated by the province’s geographic spread, with low population density, which makes it difficult for government to offer full coverage in a cost- effective manner. 18 PSLM 2004-05. 19 Estimated by the private school operators association in Balochistan. In the year 2000 the Federal Bureau of Statistics carried out a sample survey of private educational institutions in Balochistan and estimated the total number of private schools at 452, of which 261 were primary schools. This data will be updated with an ongoing census of all educational institutions in Pakistan. 20 GER rates from PSLM (CWIQ) 2004-05 together with population estimates from the National Institute of Population Studies (NIPS) indicates that there are 707,000 primary school students in government schools in 2004- 05. Administrative data from BEMIS also reports 707,000 primary school students and 16,872 primary grade teachers in government schools in 2004-05. 39 Achievement of Project Development Objective and Intermediate Results Project Development Objective and Key Indicators: The development objective of the project was to promote public-private and community partnerships to improve access to quality primary education, in particular for girls. Key Indicators: For the achievement of the PDO, following Key Indicators were set as project Key Performance Indicators (KPI’s): (a) Total enrollment of students (Target: enrollment increased by 35,000, with at least 40% girls); (b) Average annual student attendance (Target: average attendance not less than 70%); (c) Average grade completion rate (Target: average completion rate not less than 70%); (d) Average annual teacher attendance (Target: average annual teacher attendance not less than 90%); and (e) Average annual increase in learning achievement in Language and Mathematics (Target: an annual increase of 3% and 5% respectively). Below table shows the achievement against the set targets to contribute into PDO: Target Achievement Remarks Indicators Degree Total enrollment of 1- enrollment Enrollment increased by 26248 of the students increased by 48629 with 37% girls enrollment was from 35,000, with at Community Schools least 40% girls and 22381 enrollment was from Private Schools. The ratio of the girls enrollment being 37% determines standing behind target of 40% girls enrollment which chalks out the variance between target and achievement as of 3%. Average annual 2- average 86% This indicator has been student attendance attendance not achieved with access less than 70% achievement of 16% Average grade 3- average 81% This indicator has been completion rate completion rate achieved with access not less than achievement of 11% 70% 40 Target Achievement Remarks Indicators Degree Average annual 4- average annual 94% This indicator has been teacher attendance teacher achieved with access attendance not achievement of 04% less than 90% Average annual 5- an annual 6% and 9.6% This indicator has been increase in learning increase of 3% achieved with access achievement in and 5% achievement of 03% Language and respectively and 4.6% Mathematics Indicators: (1) Total enrollment of students (Target: enrollment increased by 35,000, with at least 40% girls); (2) Average annual student attendance (Target: average attendance not less than 70%); (3) Average grade completion rate (Target: average completion rate not less than 70%); (4) Average annual teacher attendance (Target: average annual teacher attendance, net of leave entitlement, not less than 90%); and (5) Average annual increase in learning achievement in Language and Mathematics (Target: an annual increase of 3% and 5% respectively). All five PDO level targets were met except the enrollment of girls. The enrollment of girlswas 37% of the total enrollment (42% in community schools and 30% in private schools) against a target of 40%. By the end of the project, the total enrollment in 633 community schools and 197 private sector schools was 48,629 with 17,794 (37%) girls covering all districts of Balochistan.The total enrollment of students in community schools was 26,248, average annual student attendance was 84%, average grade completion rate was 81%; and average annual teacher attendance was 96%. The total enrollment of students in private schools was 22,381, average annual student attendance was 86%, average grade completion rate was 85%; and average annual teacher attendance was 94%. The average annual increase in learning achievement in Language and Mathematics for community schools was 6.17% and 9.60% respectively in both the schools. No assessment was carried out for measuring this for private schools. The student attendance, completion rate, and teacher attendance increased by 3%, 3%, and 1% respectively; the reason was continuous monitoring and follow up from BEF and its IPs. The 42 months extension period was focused on construction of building for community schools. The enrollment during these 42 months only dropped by 1,585 in community schools because of the low retention rate of school teachers. The target for community school construction was reduced from 450 at appraisal to 225in 2011. The target was revised because the funding for this activity was only sufficient to construct 225 schools due to increase in the cost of construction material and labor that was not anticipated at the time of appraisal. Out of these 225 community schools, only 219 were constructed and the remaining 6 were dropped because of the community conflict. The construction of community schools started in 2012 instead of 2009 because the construction modality could not be finalized in time. Eventually it was agreed between the Bank 41 and the GoB that the community schools will be constructed through PECs with facilitation of Individual Engineering Consultants (IEC) hired by BEF. During the extension period, only community schools were monitored and the private sector schools component was dropped in December 2010 because it was not in the original design of the project. The target for private schools was revised from 300 to 200 schools because of the non-availability of PIPs that could have qualified under the selection criteria of this component. Parent Education Committees (PECs) of all 633 community schools were registered as legal entities. Their accounts were opened and made operational either in a bank or a post office. To ensure sustainability of the Project schools, the Government of Balochistan allocated and subsequently transferred funds for salaries and non-salary expenditure. Under the capacity building component of BESP, 1,100 teachers were trained (764 community school teachers and 336 private school teachers), against a target of 2,100. The target of 2,100 estimated at appraisal was high because it anticipated a larger number of teachers per school. All BEF and CIP professional staff was trained in project management, financial management, social mobilization, human resource management, M&E along with other training areas as per the need of the professional staff. All private school operators and teaching staff was trained in private school financial management and administration, record keeping, managing relationships with parents, etc. against a target of 80%. 3,245 Members of all 633 PEC’s were trained. PEC members were trained only once during the duration the entire project. These members left PECs when they migrated to other villages or became ineligible for PEC membership once their children graduated. The new members that replaced the old members were not trained. BEF’s capacity through component 3 was enhanced to expand its operations into low enrollment areas and scaled up its operations as evident from its presence in the remote rural areas of all districts in Balochistan. The TPV highlighted that the advantage of a capable and experienced Managing Director coupled with motivated and hardworking managers, remained a core reason for successful operations by BEF. Factors Affecting Implementation Widespread Geographical Area: The project dared a good successful attempt to access to the communities where the GoB ED could not access. The purpose contributed a model support to Education Department but the widespread and scattered communities hurdled service delivery and communication means. However, the commitment of BEF and the IPs and communities made it possible to establish the schools. Lack of interest of community participation: The low literacy rate in rural Balochistan ultimately results lack of the interest of rural communities in educational activities and development. The regular social mobilization being the part of the project design made the participation of the communities sure throughout the project period. 42 Risk to Development Outcome The risks identified during project preparation were appropriate and addressed through implementing mitigation measures. These included strengthening BEF’s Board membership to reflect a majority of the members from the private sector, establishment of a central complaints mechanism, intensive training to BEF, IPs, Teachers, and PECs, regular programme and financial monitoring, and improvement in procurement procedures. Risks that still persist are: (i) weak economic management and volatile security situation can limit GoB’s investments in education; (ii) GoB’s commitment diminishes with regard to supporting community based primary education; and (iii) and Government funding for the recurrent expenditures of community schools eventually dwindles or is withdrawn due to a tight fiscal situation Sustainability of the Project The project has been managed by BEF from its inception to closeout. BEF was able to manage the establishment, operationalization, and construction of primary schools. It built the capacity of teachers, IPs, and PECs; successfully produced and distributed child centered learning materials; established a fully operational education monitoring system; and created partnerships with private sector and NGOs partners to help deliver quality primary education. The project has built sufficient institutional capacity in BEF and its local IPs to manage the project interventions after the close out of the project. The community schools established under the project will continue to function if the financial support that includes teacher fees and community schools’ operation and maintenance charges continue to be funded by the GoB’s annual budget for primary education and the community schools are regularly monitored by BEF. Private schools will function on a sustainable basis through charging fees that cover their operational cost. The low cost private schools established under the project are not monitored since July 2011. This component was closed in December 2010 because of the non-availability of PIPs that qualified under the selection criteria for PIPs. The Third Party Validation conducted in 2009 highlighted that about 29% of these schools were operating at a break even and the remaining 71% were saving between PKR 15,000 to PKR 50,000. Banks involvement in the province will continue in the education sector. The Bank plans to transition its existing US$14.46 million ‘Promoting Girls Education in Balochistan’ (PGEB) project when it ends in June 2015 to a new US$34 million ‘Global Partnership for Education’ project for Balochistan in 2015. Other Outcomes and Impacts of the project Poverty Impacts, Gender Aspects, and Social Development BESP had an impact on gender equity. The Gender Parity Index (primary school) in project schools was 0.60 for rural areas that is quite close to the provincial ratio of 0.69. Since the project was focused on vulnerable groups, it promoted inclusion of disadvantaged populations in the education system. 43 Institutional Change/Strengthening BESP implementation resulted in significant institutional changes and made progress towards strengthening capacity of BEF in education development, financial management, grant management, monitoring and evaluation, fostering relationships with the communities and the private sector education institutions, and inclusion of disadvantaged populations. Lessons Learned  It is very important that the public sector commitment and ownership for the implementation and successful completion of any project are efficient.  The PECs when migrate or their children graduate, they have to quit the membership of the PEC. The PECs should be imparted with regular training every year or every second or third year.  In retaining children and teachers attendance, and increasing children enrollment and decreasing dropouts, community active participation is important.  Public sector institutions capacity to measure average annual increase in learning achievement in Language and Mathematics in Schools is weak. This needs to be built within institution, which deal with the education sector.  To retain teachers at Community Schools, their salaries need to be at the minimum equivalent to the minimum wage rate.  Capacity development support is crucial for ensuring timely and effective implementation of projects in provinces and regions that have limited public sector capacity, weak private sector presence and have deteriorating law and order situation.  Community School construction should be undertaken through professional engineering contractors under the supervision of PECs.  Early Childhood Education should be made a part of primary school teacher training.  Most importantly, the community school establishment design is far time and cost effective. The province is suffering from and alarming access issue. Education Department of Balochistan should adopt this design to ensure access throughout Balochistan. 44 Borrower’s comments on Draft ICR (received on December 31, 2014 from Prof. Dr. Abdul Tawab Khan, Managing Director, Balochistan Education Foundation) Dear Aysha, Thank you very much for sharing the draft ICR. We have gone through it and it seems aligned with the discussion and consultations during ICR preparation process. However, through this communication I would avail opportunity to bring on record, the continual support and professionalism of The World Bank team during implementation of BESP project. No doubt, without which achievements and successful implementation would have not been possible. Certainly, it is an appreciable contribution of The World Bank in education sector of most underprivileged rural communities of deprived province. I would also like to record that Balochistan Education Foundation (BEF) as an implementing agency of the BESP has demonstrated visible progress and learning enthusiasm, despites of several challenges of distances, peace & security, cultural barriers and lack of infrastructure. BEF has successfully completed the challenge of establishing 633 Community Driven Schools in far flung rural areas of Province, enrolling 27,000 children with 44% girls’ students. The performance has been recognized by line departments and the donor at various forums. BEF remained successful in maintaining and ensuring transparency and good financial management practices during the implementation of BESP. During the period of project, BEF is turned into performing organization and able to establish professional, honest and dedicated team. Now, this is the time to make efforts in sustaining Community School Model and enhance it to scale, so that development indicators on access to quality education in the province could be achieved. In this regard, BEF expects and requests all possible support of The World Bank in future as we enjoyed in past. Best Regards, Prof. Dr. Abdul Tawab Khan, Managing Director, Balochistan Education Foundation 45 Annex 6. List of Supporting Documents 1. Project Appraisal Document for Balochistan Education Support Project. Report No. 34999-PK, The World Bank, May 24, 2006 2. BESP Aide-Memoires and Management Letters 2005-2014 3. BESP Implementation Status Reports Seq. 01-17 4. Restructuring Paper for Balochistan Education Support Project. Report No. 57315-v1, The World Bank, July 15, 2010 5. Restructuring Paper for Balochistan Education Support Project. Report No. 71544-v1, The World Bank, July 20, 2012 6. Restructuring Paper for Balochistan Education Support Project. Report No. RES14628, The World Bank, May 27, 2014 7. Project Implementation Completion Report for Balochistan Primary Education Program. Report No. 20464, The World Bank, May 30, 2000. 8. BESP Quality Enhancement Review Panel Report. Email to BESP TTL. November 16, 2005. 9. Pakistan: Country Assistance Strategy (CAS), FY 2006-09, Report No. 35718-PAK, dated April 4, 2006. 10. Pakistan: Country Assistance Strategy (CAS), FY 2010-13, Report No. 53553-PK, dated July 20, 2010. 11. Pakistan: Country Assistance Strategy (CAS), FY 2015-19, Report No. 84645-PK, dated April 4, 2014. 12. Pakistan’s Balochistan Economic Report. 2008. World Bank, Asian Development Bank and Government of Balochistan. Report No. 40335-PK. Volumes I and II. 13. Third Party Validation of Balochistan Education Support Program. Sustainable Development Consultants (Pvt.) Ltd. 2009. 14. Provincial Budget (2014‐15), Government of Balochistan. 15. Learning for All, Investing in People’s Knowledge and Skills to Promote Development. Education Strategy 2020, World Bank Group. 16. BEMIS Annual Census 2009-10. 17. BEF Quarterly Progress Monitoring Report June 2011. 18. Federal Budget 2014-2015, Ministry of Finance, Government of Pakistan. 19. Student Learning Assessment: Oral Reading Fluency and Basic Numeracy in Community Based Schools in Balochistan, Pakistan. Dr. Jahanzaib Khan. June 2014. 20. Educational Competencies of Post - Kachi Class Children. Balochistan Education Foundation. January 2009. 21. Baseline Basic Competencies of Children Entering into Kachi Class. Balochistan Education Foundation. June 2008. 22. Narrative report of Study to assess the sustainability of BESP Private schools Phase 1 (2007-2009). Balochistan Education Foundation. 23. Brief Narrative Report of the Sustainability Study of BESP Private Schools. Balochistan Education Foundation. 24. BESP Community Schools Dropout Report 2012. Balochistan Education Foundation. 46 MAP 47