Document of The World Bank Report No: ICR00004003 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-13140) ON A GLOBAL PARTNERSHIP FOR EDUCATION FUND GRANT IN THE AMOUNT OF US $75.5 MILLION TO THE REPUBLIC OF GHANA FOR A GHANA PARTNERSHIP FOR EDUCATION GRANT PROJECT February 28, 2017 Education Global Practice AFCW1 Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective 1/31/2017) Currency Unit = New Ghana Cedi New Ghana Cedi 1.00 = US$0.23 US$1.00 = 4.37 New Ghana Cedi FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ADEOP Annual District Education Operational Plan MoF Ministry of Finance AESOP Annual Education Sector Operational Plan MTR Mid Term Review APW Annual Programs of Work NAR Net Admission Rate BECE Basic Education Certificate Examination NCTE National Council for Tertiary Education CBI Cluster Based In-service Training NEA National Education Assessment CSO Civil Society Organization NER Net Enrolment Ratio DED District Education Directorate NESAR National Education Sector Annual Report DFID UK Department for International Development NGO Non-Governmental Organization DP Development Partner NIB National Inspectorate Board EDSEP Education Sector Project EFA-FTI Education for All Fast Track Initiative PBME Planning, Budgeting, Monitoring and Evaluation Dept EGRA Early Grade Reading Assessment PTA Parent Teacher Association EGMA Early Grade Mathematics Assessment PTR Pupil Teacher Ratio EMIS Education Management Information System PTTR Pupil Trained Teacher Ratio ESP Education Sector Plan RED Regional Education Directorate ESPR Education Sector Performance Report SBI School Based In-service Training GAR Gross Admission Rate SHS Senior High School GDHS Ghana Demographic and Health Survey SMC School Management Committee GDP Gross Domestic Product SPAM School Performance Assessment Meeting GER Gross Enrolment Ratio SPIP School Performance Implementation Plan GES Ghana Education Service SRC School Report Cards GEU Girls Education Unit TED Teacher Education Division GLSS Ghana Living Standards Survey TIMSS Trends in International Math and Science Study GoG Government of Ghana TVET Technical and Vocational Education and Training GNECC Ghana National Education Campaign Coalition UNICEF United Nations Children’s Fund GPASS Girls Participatory Approach to Student Success USAID United States Agency for International Development GPE Global Partnership for Education UTDBE Untrained Teachers Diploma in Basic Education GPI Gender Parity Index VIPs Ventilated Improved Pit-Latrines IE Impact Evaluation WFP World Food Program INSET In-Service Education and Training JHS Junior High School JICA Japan International Cooperation Agency MDG Millennium Development Goal MMDAs Metropolitan, Municipal and District Assemblies MoE Ministry of Education Senior Global Practice Director: Amit Dar Practice Manager: Halil Dundar Project Team Leader: Deborah Newitter Mikesell ICR Team Leader: Deborah Newitter Mikesell GHANA Ghana Partnership for Education Grant CONTENTS Data Sheet .................................................................................................................... ii A. Basic Information ................................................................................................ ii B. Key Dates ............................................................................................................ ii C. Ratings Summary ................................................................................................ ii D. Sector and Theme Codes.................................................................................... iii E. Bank Staff ........................................................................................................... iii F. Results Framework Analysis .............................................................................. iii G. Ratings of Project Performance in ISRs ........................................................... vii H. Restructuring (if any) ........................................................................................ vii I. Disbursement Profile...................................................................................... viii 1. Project Context, Development Objectives and Design ............................................ 1 2. Key Factors Affecting Implementation and Outcomes............................................ 7 3. Assessment of Outcomes ....................................................................................... 18 4. Assessment of Risk to Development Outcome ...................................................... 29 5. Assessment of Bank and Borrower Performance .................................................. 31 6. Lessons Learned ..................................................................................................... 34 Annex 1. Project Costs and Financing ................................................................... 37 Annex 2. Outputs by Component........................................................................... 38 Annex 3. Economic and Financial Analysis .......................................................... 49 Annex 4. Grant Preparation and Implementation Support/Supervision Processes 60 Annex 5. Beneficiary Survey Results Summary.................................................... 62 Annex 6. Stakeholder Workshop Report and Results ............................................ 65 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR ................. 69 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ................ 85 Annex 9. List of Supporting Documents .............................................................. 86 MAP of Ghana – IBRD 33411 ................................................................................... 88 i GHANA Ghana Partnership for Education Grant Data Sheet A. Basic Information Country: Ghana Project Name: Ghana Partnership for Education Project ID: P129381 L/C/TF Number(s): TF-13140 ICR Date: 10/27/2016 ICR Type: Core ICR GHANA MINISTRY OF Lending SIL Grantee: FINANCE AND ECONOMIC Instrument: PLANNING Original Total USD 75.50M Disbursed Amount: USD 75.50M Commitment: Revised Amount: USD 75.50M Environmental Category: B Implementing Agencies: Ministry of Education B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept 02/14/2012 Effectiveness: 02/20/2013 02/15/2013 Review: 1st Restructuring: 05/14/2013 Appraisal: 05/08/2012 Restructuring(s): 2nd Restructuring: 08/25/2015 Mid-term Approval: 10/11/2012 01/19/2015 02/23/2015 Review: Closing: 10/31/2015 08/31/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Grantee Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Highly Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: ii Overall Borrower Overall Bank Performance: Satisfactory Satisfactory Performance: C.3 Quality at Entry and Implementation Performance Indicators QAG Assessments (if Implementation Performance Indicators Rating any) Potential Problem Project at Quality at Entry No None any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision No None (Yes/No): (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Public administration - Education 7 7 Tertiary Education 22 22 Primary Education 71 71 Theme Code (as % of total Bank financing) Education for all 83 83 Gender 17 17 E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Makhtar Diop Country Director: Henry G. R. Kerali Yusupha B. Crookes Practice Manager/Manager: Halil Dundar Peter Nicolas Materu Project Team Leader: Deborah Newitter Mikesell Deborah Newitter Mikesell ICR Team Leader: Deborah Newitter Mikesell ICR Primary Author: Christin McConnell F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project Development Objective is to improve the planning, monitoring, and delivery of basic education services in deprived districts of the Recipient’s territory. Revised Project Development Objectives (as approved by original approving authority) The PDO was not revised during the life of the Project. iii (a) PDO Indicator(s) Original Target Formally Actual Value Achieved Values (from Revised Indicator Baseline Value at Completion or Target approval Target Years documents) Values Deprived districts disbursing 75% or more of their district education grants as planned in their APW (Percentage) Value 0 80 N/A 100 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Source: APW reports collated annually and reported in Financial Comment Analysis Report under supervision of Financial Controller of the Ghana Education Service. Public basic schools in deprived districts with up-to-date School Report Cards (Percentage) Value 0 75 N/A 98 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Source: School Report Card Reports prepared annually capturing and collating data from districts bi-annual school grant status reports. Note: Comment Additional sample surveys also conducted to verify findings (conducted by the Ministry’s PBME monitoring team). Internal and external audits were also conducted. P3 students achieving proficiency in English and Mathematics (NEA results for deprived districts) (Percentage) English: 12.4 English: 15 English: 16 Value N/A Mathematics: 9 Mathematics: 11 Mathematics: 13 Date Achieved 10/10/11 10/31/15 N/A 09/15/2013 The project target was achieved by 2013 with the 2013 NEA results. Source: NEA report 2014. Note: A confirmation of further improvements in learning outcomes through additional data at endline was not available as the 2015 NEA was postponed to 2016 and the level of assessment was moved from P3 to P4 level. However, EGRA/EGMA assessments were also conducted in 2013 and 2015 (with USAID support) which confirms that there were statistically significant improvements in number identification, letter sounds in English, oral reading in Comment English, and letter sounds for local languages in the GPEG deprived districts. The project outputs of improved percentage of qualified teachers in deprived districts, in-service teacher training and the purchase of teaching and learning materials from district and school grants contributed to improved learning outcomes observed within the deprived districts targeted by the project. Improvements of 29% for English and 44% for Mathematics in NEA between 2011 and 2013 are significant and compare favorably to other countries in the region. Teachers trained under the project in deprived districts who obtain satisfactory rating or higher in the SBI/CBI Lesson Observation Sheet for (a) lesson planning (b) teaching methodology (c) classroom organization and management (Percentage) iv Average: 75.7 Average: 36 Average: 50 Lesson Planning: 69 Value N/A Teaching Method.: 80 Classroom Org: 78 Date Achieved 10/10/10 10/31/15 N/A 8/31/2016 Target exceeded. Source: Baseline data collected from JICA supported pilot of the Lesson Observation Survey. Three surveys conducted since baseline by independent firm (Associates for Change) hired to implement survey drawn from a representative Comment sample of schools. The highest values for satisfactory performance for lesson observation actually occurred during midline data collection in 2015 with Lesson Planning at 85%, Teaching Methodology at 87%, and Classroom Organization and Management at 90%. Direct Project Beneficiaries (Number), of which female (Percentage) 1.6 million 0 1,991,030 Value Of which female: N/A Of which female: 0 Of which female: 48 40 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Source: Project status reports and EMIS. Note: Beneficiaries Comment included students, teachers, head teachers, Circuit supervisors, district officers, regional officers, PTA/SMC. (b) Intermediate Outcome Indicator(s) Original Formally Actual Value Achieved Target Values Revised Indicator Baseline Value at Completion or Target (from approval Target Years documents) Values Head teachers and Circuit Supervisors in deprived districts trained under the project in the use of School Report Cards (Number) Value 0 6,930 N/A 7,087 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Note: This included 5,993 Head Teachers and 1,094 Circuit Comment Supervisors. Source: Training Reports from districts, collated by Teacher Education Department, Ghana Education Service. Regional and District Education officers trained under the project (eg. Financial management, data collection and analysis, etc.) (Number) Value 0 500 N/A 2,288 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Note: This included 1,568 District and regional officers trained in work planning and school report cards, 83 district gender officers trained in guidance and counseling, 308 regional officers trained in reporting, Comment monitoring and evaluation, and 329 regional and district officers trained in internal controls and auditing. Source: Training Reports from districts, collated by Teacher Education Department, Ghana Education Service. Teachers trained under the project (disaggregated by UTDBE and INSET) (Number) v UTDBE: 0 UTDBE: 5,000 UTDBE: 6,480 Value N/A INSET: 0 INSET: 30,000 INSET: 42,541 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Source: These figures are based on information collected through the monitoring and evaluation system, Teacher Education Department Training Reports and District and School Grant reports. Note: Reporting collected yearly participants in each INSET in which it is impossible to know if Comment the same teacher was trained more than once in different areas. The cumulative number of INSET trainings was calculated to be 94,827. As a proxy, in 2015 28,056 teachers participated in the core Mathematics INSET, and 14,485 in non-core subjects of KG and Gender, indicating that the target was met – for both UTDBE and INSET. District Education Directorates collating School Report Cards from at least 70% of schools within the district (Percentage) Value 0 70 N/A 100 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Source: District- level School Report Card Reports submitted Comment with School Grant Report. Basic schools in deprived districts participating in all INSET core courses supported by the project (Percentage) Value 0 65 N/A 97.8 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Source: School Grant Report. Note: INSET annual report Comment used to verify findings. Schools that have received and displayed the most recent School Report Card on their notice board (Percentage) Value 0 75 N/A 93 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Comment Target exceeded. Source: School Grant Report. Schools visited by Circuit Supervisors at least two times in a year (Percentage) Value 0 100 N/A 100 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target met. Source: District Education Directorate annual report based on Comment Circuit Supervisors Reports. Note: Also reported in School Grant Reports for cross validation. Regional Education Directorates submitting annual M&E reports to GES HQ & PBME by incorporating data from all districts in the region (Percentage) Value 0 100 N/A 100 Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target met. Source: Annual M&E reports using the regional consolidation Comment templates. Schools in deprived districts with completed SPIPs approved by SMCs using the revised template (Percentage) Value 0 75 N/A 100 vi Date Achieved 10/10/12 10/31/15 N/A 8/31/2016 Target exceeded. Source: School Grant Report Template. Note: Districts also Comment verify with Circuit Supervisor Reports. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 04/28/2013 Satisfactory Satisfactory 10.00 2 12/14/2013 Satisfactory Satisfactory 22.68 3 06/01/2014 Satisfactory Satisfactory 34.21 4 12/24/2014 Satisfactory Satisfactory 46.60 5 06/27/2015 Satisfactory Satisfactory 60.48 6 12/31/2015 Satisfactory Satisfactory 70.49 7 06/22/2016 Satisfactory Satisfactory 75.50 8 08/31/2016 Satisfactory Satisfactory 75.50 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Approved Reason for Restructuring Restructuring Date(s) PDO & Key Changes Made DO IP in USD Change millions To amend the legal agreement to add "works" as 05/14/2013 S S 14.96 a category under District Grants. To extend the Project for 10 months from October 31, 2015 to August 31, 2016 and to introduce a reallocation of 08/25/2015 N S S 60.48 funding among categories, particularly from the unallocated category to category 2 (district grants). vii I. Disbursement Profile viii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Country Context 1. Ghana, a lower middle income country with sustained economic growth, was considered a strong Sub-Saharan African economy at the time the Ghana Partnership for Education Grant (GPEG) Project was appraised. Just prior to Project appraisal, the beginning of oil production and high global commodity prices and good rains had boosted the economy; real gross domestic product (GDP) growth was expected to reach 12.2 percent in 2011 and projected to remain greater than 7 percent in 2012 and 2013. 2. Ghana’s performance in terms of human development outcomes had been improving and steady progress had been made with respect to the education Millennium Development Goals (MDGs). The proportion of children completing primary school had increased to 89 percent by 2012 from 70 percent in 1991. Progress on gender parity also showed improvement (92 percent ratio of girls to boys in primary and lower secondary education). However, Ghana was not on track to fully achieve these two MDGs by 2015. At appraisal, the share of the population living in poverty was reduced by half between the early 1990s and 2005-2006, but significant regional disparities remained between the poorer regions in the north compared to the rest of the country. Sector Context 3. The education sector in Ghana is divided into three levels: basic education (including kindergarten, primary school, and junior secondary school), secondary education (including senior high school and technical and vocational education), and tertiary education, all under the jurisdiction of the Ministry of Education (MoE). The Ghana Education Service (GES), an agency under MoE, is responsible for pre-tertiary education service delivery while the National Council on Tertiary Education (NCTE) is responsible for tertiary education. Ghana has been operating using a decentralized system since 1988 when local governments began to play a central role in administering greater resources and taking on new functions. In education, basic schools are overseen by the District Education Directorates (DED), which are local offices of the GES. 4. At the time of project appraisal, access to education in Ghana had improved substantially. Between 2002 and 2012, gross enrollment rates (GER) increased from 49 to 99 percent in kindergarten, from 76 to 96 percent in primary, and from 63 to 81 percent in junior high school (JHS). These improvements were driven by efforts to improve supply of education as well as the elimination of school fees which led to the provision of school capitation grants starting in 2004. In terms of learning outcomes, the National Education Assessment (NEA) showed some improvements in English and Mathematics between 2005 and 2009 and some progress on the Trends in International Math and Science Study (TIMSS) assessment of Grade 8 pupils between 2003 and 2007. 1 5. Despite this progress, key challenges in the education sector remained including disparities in learning outcomes, insufficient number of qualified teachers, and inequitable access. There were significant disparities in learning outcomes between the north and south of the country and between rural and urban schools. Education outcomes largely followed regional poverty trends with poor districts challenged by weaker education outcomes, untrained teachers, and fewer financial resources for education spending. In 2009, despite observed improvements, less than a quarter of Grade 3 students in public schools were able to read a single word in English. The 2011 NEA and Basic Education Certificate Examination (BECE) exams reflected wide gaps in learning between Northern, Upper West, and Upper East Regions and the rest of the country. While net enrollment rates increased1, a sizeable group of children (about 400,0002) were not in school (either never entered or having dropped out). In 2008, over 65 percent of girls over the age of 15 in the Northern region had not received any formal education compared with a national average of 21 percent. Trained teachers were more likely to work in urban or peri-urban schools; at the time of the Project appraisal there were over 155 students per qualified teacher in districts in the Northern, Upper West, Upper East, and Western regions. Project Context 6. During the preparation of the Global Partnership for Education Fund Grant (GPEG), the IDA Education Sector Project (EdSeP- US$78 million) was drawing to a close (October 31, 2011). The EdSeP aimed to improve (i) equitable access to and completion of basic education in deprived districts; and (ii) quality of teaching and learning, management efficiency, and relevance of post-basic education. In addition to the EdSeP which began in 2004, the Education for All-Fast Track Initiative (EFA-FTI) approved a US$32.2 million Catalytic Fund allocation for Ghana that was to be processed as three separate grants. The first two grants (US$8 million in 2005 and US$11 million in 2007) were for direct budget support to improve the quality of basic education. The third grant (in the amount of US$14.2 million) in 2009 was a sector investment loan3 and unlike the first two grants which were considered satisfactory, this operation experienced significant challenges with large procurement packages and a short implementation period4. 7. While work had been done under the EdSeP in sector capacity building at the central and decentralized levels, and piloting budgetary support to deprived districts – there 1 From 2011-2015 improvements are evident across all levels of education (from 4.2 percent to 80 percent in kindergarten, 81.7 percent to 91.5 percent at primary and 46.1 percent to 50.3 percent in JHS). 2 UNICEF projections 2015. 3 The use of the Sector Investment Loan instrument conformed to the new EFA-FTI guidelines 4 By late 2008, presidential and parliamentary elections occupied the attention of key Ministry of Finance and Economic Planning (MOFEP) officials which delayed final execution of the grant agreement to 2009. Between the close of the second grant and the signing of the third, the Ministry of Education (MOE), in anticipation of signing a third EFA-FTI grant, initiated a textbook procurement that they expected to be financed with the new grant without Bank prior review of procurement documents or a no objection from the Bank. This set the stage for many of the problems that occurred during the implementation of the third EFA- FTI grant.” (ICR, Education Sector Development Project, December 2011, page 2) . 2 was still a need to mainstream these decentralized processes (e.g., district and school grants) and further improve the existing monitoring and evaluation (M&E) systems. Moreover, the importance of ensuring targeted resource allocation and significant capacity- building in less endowed districts were lessons that had been learned from previous projects. At the time of appraisal, the UK Department for International Development (DfID) had also begun planning a scholarship program for girls in JHS with a plan to overlap this program’s targeting with that used by GPEG. In addition, the United States Agency for International Development (USAID) had begun planning and designing a learning project which focused on early grade literacy and numeracy. 8. While the introduction of district and school grants under EdSeP was quite promising for the education system, particularly with the political push to implement decentralization to the district level, additional resources would be needed. This was particularly the case in those areas of the country where there are significant disparities in economic and social opportunities. While public spending on education was roughly 6 percent of GDP and Ghana compared favorably to higher middle income countries who spend on average 5 percent of GNP on education,5 there was often little funding available outside of financing salaries. In 2010, roughly 97 percent of the education budget was allocated to salaries, leaving little for non-salary recurrent spending. 9. Since 2004, capitation grants (GH¢ 4.50 per student per academic year) had been supplied in order to provide non-salary resources directly to schools by the Government. The Government had also introduced the Untrained Teachers Diploma in Basic Education (UTDBE) program in 2004 to upgrade the skill sets of unqualified teachers through distance education. While the program had been successful in upgrading 16,000 untrained teachers in rural schools, the portion of trained teachers in public basic education had been steadily declining from 65 percent in 2001 to 48 percent in 2009 as enrollment increased. As such, the UDTBE program needed to be accelerated in order to increase the number of trained teachers in rural and underserved communities. The use of school report cards (SRCs) was introduced in 2010 as a pilot (with USAID support) to provide greater accountability in the areas of attendance and student learning outcomes while school management structures existed (e.g., School Management Committees (SMCs), School Performance Appraisal Meetings) but needed strengthening. A 2010 World Bank Survey6 found that even though SMCs existed in over 80 percent of schools, they were only active in just over 61 percent of schools and members had not been adequately trained in preparing a School Performance Improvement Plan (SPIP). 10. In 2010, The Ministry of Education (MoE) developed the Ghana Education Sector Plan (ESP) for 2010-2020 as well as the three-year rolling Annual Education Sector Operational Plan (AESOP) for 2011-2013 in close partnership with the Local Education 5 UNESCO Monitoring Report, 2015 comparison shows higher income countries such as Cambodia and the Philippines spending about 2.7 and 2.2 percent respectively of their GNP on education and only 14 percent and 15 percent of their total budgets for education. 6 Impact Evaluation of School Based Management Committees, 2010. M-H Cloutier (A survey of 300 primary schools). 3 Group (LEG). The main priorities in the ESP were to: (i) improve equitable access to and participation in quality education at all levels; (ii) bridge the gender gap in access to education; (iii) improve the quality of teaching and learning; and (iv) improve the management of education service delivery. The GPEG Project, financed by the Global Partnership for Education (GPE) (formerly EFA-FTI), provided direct support for this ESP while the Bank was selected as the Project supervising entity. Following on the heels of the EdSeP – a large operation which had piloted support for district grants – the GPEG provided an opportunity to expand and strengthen the district grant processes that were considered successful under the EdSeP. 11. Rationale: In the context of Ghana’s geographic disparities and recent economic growth, opportunities for equitable growth and shared prosperity were largely contingent on improving the level of education and employability of the workforce in deprived regions of the country. Education was considered one of the main mechanisms to allow individuals living in poverty to share in Ghana’s economic growth. The support was consistent with the World Bank Country Assistance Strategy (CAS) (2008-2012) objectives to focus on targeted programs to the poor and improve basic health and education systems through a decentralized system. The Project was also aligned with the GPE Global Strategic Objectives of 2012-2015, and as noted above, provided direct support to Ghana’s 2010- 2020 ESP. The Project aimed to achieve its project development objective (PDO) by: (i) making non-salary financial resources available to schools and districts in deprived areas to improve education service delivery by local actors; (ii) improving decentralized decision-making for school-based management and accountability; (iii) funding the upgrading of skill sets of untrained teachers already teaching in underserved areas;; and (iv) integrating and further developing ongoing and donor-led management tools such as the SRC activity. 1.2 Original Project Development Objectives (PDO) and Key Indicators 12. The Project Development Objective was to improve the planning, monitoring, and delivery of basic education services in deprived districts of the Recipient’s territory. Successful achievement of the PDO would be measured by the PDO-level indicators presented in Table 1 below. 4 Table 1. PDO-level Indicators PDO-level Indicators Baseline End of Project Target (2010/2011) (2015/2016) 1. Deprived districts disbursing 75% or more of their district education grants as planned in their Annual 0 80 Program of Works (%) 2. Public basic schools in deprived districts with up-to- 0 75 date School Report Cards (%) 3. P3 student achieving proficiency in English and English 12.4 English 15 Mathematics (using National Education Assessment Mathematics 9 Mathematics 11 results for deprived districts) (%) 4. Teachers trained under the project in deprived districts who obtain satisfactory rating or higher in the SBI/CBI Lesson Observation Sheet for lesson 36 50 planning, teaching methodology, and classroom organization and management (%) 5. Direct project beneficiaries (number) of which 1.6 million of which 40% 0 female (%) CORE female 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 13. The PDO and Key Indicators were not revised during the life of the Project. 1.4 Main Beneficiaries 14. The project beneficiaries identified at appraisal were the students, teachers and administrators (all disaggregated by gender) targeted by project interventions in the deprived districts. The targets for beneficiaries were disaggregated by gender and tracked throughout implementation. Under the Project, students, teachers, and administrators in over 7,000 project-supported schools in 75 deprived districts (out of 216 districts) had benefitted from Project activities. Secondary beneficiaries of the Project included the Colleges of Education responsible for the UTDBE program, SMCs, Parent Teacher Associations (PTAs), Regional Education Officers (REOs), District Education Officers (DEOs), GES central staff, National Inspectorate Board (NIB) and senior MoE staff involved in the implementation and M&E of the project. The targeted beneficiaries of the Project did not change during the life of the Project, with the exception of additional district-level staff which were included as a result of incorporating 18 newly created districts (in the same geographic area) that had been formed during the national redistricting exercise in 2014.7 7 In 2014, there was a re-delineation of Ghana’s 170 districts to 216 districts, leading to an increase of 18 targeted deprived districts (carved out of the original 57 deprived districts) for the Project during implementation and bringing to the total number of beneficiary districts to 75. 5 1.5 Original Components 15. Component 1: Sub-Grants to Deprived Districts to support key education objectives (Appraisal estimate: US$44.86 million; Actual: US$48.41 million). The goal of this component was to provide annual non-salary supplemental resources to deprived districts to support districts’ annual programs of work (APW) focused on local needs as well as government strategic priorities of improving access, equity, quality and management of basic education. Under this component, the government planned to increase accountability and participation by districts, schools, and communities through: (i) earmarked funding for the upgrading of unqualified teachers’ skills sets through the UTDBE program; (ii) mandatory in-service teacher training in literacy, numeracy, and science; (iii) capacity building for GES staff, regional and district officials, Circuit Supervisors, SMCs, head teachers, etc.; (iv) facilitation and knowledge sharing by regional and central governments; (v) M&E activities; and (vi) continuous sensitization activities to maintain social accountability. 16. Component 2: School Sub-Grants (Appraisal estimate: US$24.06 million; Actual: US$20.38 million). The goal of this component was to improve teaching and learning through supplements to the capitation grants at all basic education schools in deprived districts. The component aimed to improve the management and accountability of resources at the school level and to strengthen the School Performance Improvement Plan (SPIP) process and community engagement in approximately 6,600 schools across the deprived districts. Component 2 was to fund training on the school grant process, monitoring visits, communication campaigns, school grants, and an impact evaluation (IE) of school grants (this IE was later combined with the district grant evaluation and funded under Component 3). 17. Component 3: Project Management and Institutional Strengthening (Appraisal estimate: US$6.58 million; Actual: US$6.71 million). The purpose of this Component was to strengthen government systems for the implementation and supervision of decentralized education services in the deprived districts. This component would fund consultancy services, training and operational costs, materials, IEs, and external and internal audits. 1.6 Revised Components 18. There were no major changes in the components other than reallocation of resources following the mid-term review (MTR) in February 2015. 1.7 Other significant changes 19. The project was restructured twice. The first restructuring was approved on May 14, 2013 in order to add “works” to Category 2 which supported sub-grants to the deprived districts (District Grants). 20. The second restructuring was approved on August 25, 2015 in order to extend the project closing date by ten months from October 31, 2015 to August 31, 2016, and to 6 reallocate funding among expenditure categories. The extension would allow additional time to: (i) provide support to teacher trainees in the UTDBE program to complete their diploma program; (ii) start-up support for the 18 additional districts (carved from the original 57 districts); and (iii) complete the IEs. 21. With respect to the teacher trainees in the UTDBE program, the project had intended to support the trainees through their third year only;8 however, once a project extension was requested by the Ministry of Finance, it was agreed that the project could continue its support to the program participants for their fourth and final year of the program. There were significant savings under the project as a result of exchange rate gains, lower costs than estimated, and efficient procurement of goods and consultancies. As mentioned previously, the expected re-delineation of Ghana’s districts in 2014 led to the eventual establishment of 18 additional deprived districts carved out of the original 57, bringing the total number of project districts to 75. This increased administrative functions as the number of district offices involved increased by 18, but did not alter the targeted project beneficiaries (schools, teachers) as the same population was being captured within the 75 districts. The extension and reallocation under the restructuring supported capacity- building for the new district officers, technical assistance and training on project processes, and start-up funding to equip the new offices and support implementation monitoring. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 22. Project Preparation. Project design was in alignment with key education sector priorities for the country. As described earlier, its design was harmonized with the ESP for 2010-2020 as well as the three-year rolling AESOP for 2011-2013 in close partnership with the LEG. The project reflected the ESP strategic goal to provide “equitable access to good- quality child-friendly universal basic education” and as such focused on targeting improvements in the deprived areas of Ghana which were lagging behind the rest of the country in terms of education outcomes. The Project design was also in line with the CAS 2008-2012 which focused on targeted programs for the poor and improving basic health and education services through the decentralized system. Moreover, the GPEG was consistent with the GPE Global Strategic Objectives (2012-2015) of supporting basic education for girls, improving literacy and numeracy skills, improving teacher effectiveness, and expanding the volume, effectiveness, efficiency, and equitable allocation of education funding. 23. Project preparation relied extensively on significant analytical work by both the World Bank and the LEG. A significant strength of the GPEG was the participatory nature of the preparatory process. The Project was prepared in close partnership between the Government and the LEG, which included the DfID, the United Nations Children’s Fund (UNICEF), the Japan International Cooperation Agency (JICA), USAID, and the World 8 As mentioned in the PAD, additional donor funding or government budget was expected to cover the costs of the UTDBE beneficiaries in the fourth year. 7 Food Program (WFP) among others, through joint missions, monthly development partner (DP) meetings, and weekly preparation meetings starting in October 2011. The appraisal of the ESP coincided with the preparation and design of the GPEG Project, allowing close involvement of key partners and a strong focus on supporting the objectives of the ESP. Project preparation also included stakeholder workshops and multi-donor missions to ensure strong ESP alignment and wider ownership of the Project. 24. Project Design. The Project is Ghana’s fourth EFA-FTI (now GPE) grant. While the first two grants provided budget support and were rated Satisfactory, the third grant (which was a SIL) aimed to procure textbooks and learning materials and provide incentives to attract and retain teachers in deprived districts, was rated Unsatisfactory. This rating was attributed to the operation’s overly ambitious timeline, implementation delays, misunderstandings of new guidelines, post-election government changes, and ultimately the repayment of ineligible expenses to the GPE Fund.9 As a result of this experience, the GPEG project application was prepared in a more collaborative and comprehensive manner and avoided centralized procurement of goods or works. In addition, the production of a detailed Project Implementation Manual (PIM) by the Government enhanced accountability among all stakeholders. The lessons learned from the experiences with the EFA-FTI grants on strong government ownership and leadership, strong project supervision, and realistic implementation period and targets were critical in informing the design of GPEG. 25. The GPEG design also benefitted from other investments in the sector and in Ghana. Activities and approaches that were piloted under the EdSeP became the evidence-base for the GPEG (e.g., equity-based targeting, district and school grants, local capacity-building for planning and monitoring) 10. The GPEG design was informed by lessons learned from EdSeP including the importance of: (i) having a simplified project design; (ii) setting realistic ambitions for decentralized delivery of education services of district grants; (iii) ensuring indicators are linked to Project interventions and are measurable; (iv) ensuring transparent and objective targeting of Project support to more economically disadvantaged regions; (v) establishing more integrated and comprehensive M&E systems to collect data, measure impact, and influence decision making; (vi) ensuring the timely release of funds along with capacity-building at the district level to improve success of district grants against APWs; and (vii) providing capacity-building support at the school level to include circuit supervisors, head teachers, and SMCs. The GPEG revised the criteria for selection of “deprived” districts to include both Ghana’s poverty index as well as key education 9 The implementation period of just over one year also coincided with the long transition period after the 2008 election and substantial confusion about the change in fiduciary rules from budget support to investment project. The large centrally procured packages were problematic and the remaining activities (civil works for teacher accommodations) required a longer implementation period. In addition, supervision was inconsistent and led to misunderstandings between the Ministry and the Supervising Entity which led to a neglect of World Bank procurement guidelines with respect to the textbook component. These expenditures were declared ineligible and were repaid to the EFA-FTI Fund. 10 Details of lessons learned were included in the PAD as an optional Annex (PAD, Annex 8, page 106) and included a literature review of school-based management interventions and evaluations to inform the school grant component. 8 indicators11. The bottom third of the districts, ranked according to the new criteria were then selected to benefit from the GPEG. All basic schools within these districts would be supported under the Project, and upgraded teachers (through training under the Project) would be assigned to the schools in the targeted districts. 26. Project Risks. During preparation, risks were rated substantial related to stakeholders, implementing agency capacity and overall decentralized service delivery. Part of the rating reflected the upcoming 2012 elections and concerns about staff turnover, but most of the issues highlighted were potential risks related to consistent leadership and coordination and the participation of local communities to ensure that education services would be delivered. The extensive GPEG consultations combined with the well-vetted and publicized ESP helped maintain and foster consensus among a large number of stakeholders. In addition, the PIM guided implementation at both the central and district/school levels. A separate (and simple) school grant manual was also developed and widely disseminated during frequent monitoring visits. In feedback from Impact Evaluation (IE) surveys, the use of GPEG management and planning tools was highly evident as was the involvement and satisfaction of parents/communities. 12 Institutional assessments had found an overall low capacity at the district and school levels, especially in deprived regions/districts, so this posed a potential risk. It was decided, therefore, that, intensive supervision would be provided over the first 18 months of project implementation (at a significant cost to the Bank supervision budget) to help encourage technical support and address challenges quickly. 27. Involving the REDs provided additional resources at the local level and helped build capacity; having developed simplified data monitoring and reporting templates during project preparation helped to improve M&E systems. In addition, the Project budgeted for significant training at the local level and monitored this training through the Project’s Result Framework. Multiple monitoring tools already in use or recently developed under the GPEG helped mitigate risks to delivery monitoring. The risk of sustainability, while rated moderate, was highlighted during preparation (and throughout implementation) and only partially mitigated because of the fiscal crisis that emerged by 2014. The most significant mitigation measure to address these aforementioned risks under the GPEG was the use of government systems which ensured ownership, political leadership and adoption of standardized management tools. 28. Quality at Entry. As mentioned above, the preparation of the GPEG was launched from the ongoing ESP process. The ESP process included considerable technical guidance and support from the GPE, the LEG, and DP-funded consultancies in the areas of girls’ education, teacher training, learning assessment, decentralization, and kindergarten roll out. 11 Education indicators looked at performance in retention in the basic cycle, share of girls enrolled in P6 and JHS 3, pass rate in BECE English and share of trained teachers in all public primary schools. 12 The GPEG impact evaluation surveyed 353 schools out of the over 7,000 schools in the 75 GPEG districts. See sampling design and findings in the 2016 report “Impact evaluation and monitoring of the District Grant and School Grant component of the Ghana Partnership for Education Grant. ” by Edburgh consultants and Innovative Services consortium. 9 The additional analytical work from these activities as well as the Bank’s sector work provided a sound framework for the GPEG components.13 The selection of the Bank as supervising entity for the Project preparation also introduced the Bank’s quality assurance process to complement GPE’s processes. The LEG played a particularly critical role, under the leadership of the Coordinating Agency (UNICEF) in ensuring consistency of the objectives with the ESP and donor-supported programs. 29. Several rounds of quality-at-entry assessment were carried out as part of Project Preparation. This included two rounds of Quality Assurance Review (QAR) from the GPE Secretariat as well as a Quality Enhancement Review (QER) with the World Bank. The GPE was using new procedures for the processing of projects and there was considerable confusion about how to align the new GPE guidelines with Bank project processing guidelines. The GPEG navigated these challenges by carrying out both quality assurance reviews. Both the QAR and QER endorsed the design and approach of the Project, its consistency with the ESP, and provided guidance on issues related to the PDO, RF, selection of project activities, elaborated implementation arrangements, and greater donor harmonization. Similarly, the Bank’s QER assessment provided further feedback on the PDO, RF, M&E arrangements and activities, and readiness for implementation. The final design reflects the guidance received, although with two separate quality assessments, the team had to navigate through some contradictions in guidance.14 Overall, the ICR team found the process to be technically sound and operationally appropriate, with strong collaboration from the LEG. 2.2 Implementation 30. Implementation of the project was satisfactory – as a result of focused yet relevant project design (supporting a limited number of activities) which built on ongoing investments, strong donor coordination, use of existing government mechanisms, and strong Project M&E, management, and supervision. 31. By limiting activities to those which had already received support (and had been field-tested) in Ghana, the Project was able to disburse all funds and complete project activities within a four-year period (the large majority of activities were completed by Year 3). The school capitation grants, district grants, and UTDBE program had all been previously (and recently) implemented in Ghana, allowing for easier uptake of improved and expanded versions of those programs through GPEG. 13 Documents included the 2012 PFM Review, JICA; 2011 Ghana Educa tion Report “Improving Equity, Efficiency and Accountability of Education Service Delivery”, World Bank; Education Sector Development Project Implementation Completion Report 2012, World Bank; Education Support to Education Strategic Plan and KG operational plan, DFID. 14 For example, the GPE QAR placed emphasis on showing a contribution to improved student learning, while the Bank QER suggested that PDO indicators be in the scope of the Project and attributable to the Project. In the end, the Project compromised with a PDO indicator on student learning kept but a PDO on teacher absenteeism dropped. 10 32. Harmonization among DPs was strong throughout GPEG implementation and this greatly aided successful implementation. Progress under the Project was monitored and reported at monthly LEG meetings, allowing for frequent status updates. The project was launched and closed by the same Minister of Education. Six formal joint implementation support missions were held since effectiveness, occurring approximately every 6 months. These missions included joint field visits with other DPs (most of whom provided parallel financing) including DfID, UNICEF, and USAID which led to greater coordination, collaboration, and technical support to GPEG. The Bank and LEG supported four Government-led National Education Sector Annual Reviews (NESAR) during the Project where an expanded group of education stakeholders participated in discussions on the sector. GPEG was also highlighted in the Government’s Annual Education Sector Performance Report (ESPR), published at the time of each NESAR. Moreover, the US$10 million DfID-financed Girls Participatory Approach to Student Success (GPASS) Scholarship program was operating in GPEG districts, aligning with GPEG APWs and using the same structures as GPEG. This allowed both projects to complement each other for shared achievement of outcomes. 33. The use of existing government structures was an important feature of the operation and strengthening decentralized models of governance in the education sector was an important objective of the GPEG. By working through existing government mechanisms, GPEG fostered strong project ownership within the MoE and GES. The beneficiary assessment credited the GPEG for empowering the existing systems and helping to make them better functioning. 34. The project also benefited from dedicated project management and supervision and this facilitated the effective implementation of the project. MoE/GES and Bank teams closely supervised the Project, and regular and candid dialogue between the Bank and the Ministry on implementation helped resolve implementation bottlenecks in an efficient manner. On the Bank side, having in-country core team members and a team task leader who prepared and implemented the project allowed for sustained and consistent support and supervision. The Project’s funding of a video conference facility at GES further supported strong communication between the Ministry and Bank teams. 35. Despite having an overall satisfactory implementation, GPEG encountered a number of challenges including the collection and validation of large amounts of data and high staff turnover (especially at the local level), as well as reporting challenges and attrition in the UTDBE training program. The collection and validation of data at district and school levels experienced some delays throughout the project period as districts struggled to capture and code all school level data (including individual teacher training). There was a significant amount of data to collate. Further, staff turnover in districts and schools was high. While the project design included trainings for SMCs, Head Teachers, Circuit Supervisors, District and Regional officers, the high turnover resulted in schools and districts not always well versed in guidelines and procedures. New head teachers faced difficulties in preparing SPIPs which led to mistakes and delays in vetting. At the district level, high turnover was especially challenging for procurement officers. This was partially alleviated through support from REDs, the availability of refresher trainings and well as ad 11 hoc trainings for new staff. When district directors had an increased awareness of GPEG, district grants ran smoothly. 36. The Government committed significant resources (human and financial) to twice yearly district collation exercises to bring staff together to go over key financial and monitoring reports. These workshops also served as opportunities for additional capacity- building and training of key project staff/officers, especially those that may have been newly appointed at the district level. The School Grants activity also provided ample resources for ongoing training/capacity building for head teachers and teachers so new staff could better acquaint themselves with the GPEG processes. 37. The Project also faced some initial eligibility concerns and reporting challenges on the UTDBE program which were largely resolved during implementation. There were initial misunderstandings within the UTDBE program on the eligibility requirements particularly as a result of the parallel Northern UTDBE program (sponsored by the Colleges of Education) that had been launched just prior to GPEG effectiveness without the knowledge of the Bank team. This other program (held in the Colleges of Education based in the Northern regions) had enrolled many teachers in GPEG districts but without application of the GPEG eligibility criteria. With the strict eligibility criteria being applied under the GPEG, and many trainees already enrolled in the Northern program, there was considerable confusion about the initial intake. 38. Further, there was some degree of attrition with the numbers of trainees enrolled in the program with 7,027 UTDBE participants sponsored by GPEG in 2013 and by 2016 only 6,480 had completed the full training. Some of these participants dropped out of the program, passed away, or failed to pass (among other reasons).15 In addition, the inability of the GPEG to use the Colleges of Education in the North for the residential coursework (they were oversubscribed with the initial Northern program) meant that trainees had to travel long distances to attend their face-to-face sessions at other Colleges of Education. This increased transaction costs and made it more difficult for tutors to provide the in- school coaching and support required. GPEG was able to resolve some of the eligibility concerns during the first year through an extensive verification and cleaning exercise conducted by the quality assurance consultants hired to support the UTDBE program. These consultants were also expected to provide additional recommendations to improve the face-to-face sessions as well as the in-school and in between coaching sessions and they produced a monitoring report after the MTR when concerns were raised about the lack of regular and high-quality reporting on UTDBE program beneficiaries. However, the quality of the TA provided by these consultants could have been better. The team expressed its dismay at the level of assistance as well as the lack of proactivity by the consultants. The team hired two additional consultants to provide direct support to the Teacher Education Department on revising the UTDBE materials for both the residential college course and for the in-between trainings in order to ensure technical and quality assurance was provided. 15 The ICR does not yet have the final data on these graduates with regard to their inclusion on the Government’s teacher payroll. 12 39. Mission field visits and other monitoring visits found some cases of poor record keeping by head teachers, SMCs not operating as effectively as they should, and other challenges at the school level (including overcrowding from increases in attendance, need for additional teachers and resources beyond the school grant, difficulties monitoring remote schools, etc.). The project team set out to better understand the issues by conducting two in-depth fiduciary reviews in addition to the regular internal and external audits. Mitigation measures were identified from these reviews such as continual training for new staff, simplified school level templates and greater support from circuit supervisors. The latest management response to the audit report indicates actions that were taken to address the issues raised. 16 At the MTR stage, it was noted that additional attention to communication and sensitization of GPEG would be needed for greater Project effectiveness; by early 2016 the mobile SRCs piloted with support from UNICEF and USAID held promise for enhancing community engagement and accountability. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 40. Design. The system as described in the PAD, PIM and impact evaluations was sound and capable of collecting information on all performance indicators. One of the great strengths of the M&E design, and ultimately the GPEG Project, was the richness and diversity of M&E data built into the Project design. While Ghana had an Education Management Information System (EMIS) system in place for a decade, its credibility continued to be an issue and the GPEG helped to strengthen the integrity of EMIS data and cross-verify how schools were reporting. GPEG included APW analyses, SRC analyses, quality assurance on the UTDBE program, a beneficiary study, internal and external audits, an extensive MTR report, and two IEs. Given the focus on better planning and monitoring, the GPEG set up standardized systems for better analysis of data from schools and districts. The systems evolved over the course of implementation as it became clear when reporting was too burdensome, or school heads had difficulties inputting data. The PIM served as a roadmap, offering explicit instructions on M&E reporting, and has since been used as a sample for other donor-funded projects. 41. The quality of the RF was good overall. The PDO-level and intermediate results indicators were linked to the PDO, with at least one PDO-level indicator for each sub- objective. There are a reasonable number of PDO-level and intermediate results indicators and realistic targets for a three-year project – all of which benefitted from QAR/QER feedback. This allowed for regular reporting on Project progress and accurate indications that the Project was performing well throughout implementation. The causal chain in the RF is good with all intermediate indicators supporting a PDO indicator with only a couple of exceptions. There is perhaps an overemphasis on SRCs in the RF indicators; the RF may have benefitted from greater emphasis on the development of SPIPs from SRC data and the funding of activities according to what had been planned in SPIPs. 16 This issue of school-level non-compliance was quite low considering that the GPEG reached over 7,000 deprived schools with low levels of capacity at the start of the Project. 13 42. The PDO-level indicator of “P3 students achieving proficiency in Math and English” was ambitious for a three-year project. Attribution for learning is difficult to apply only to GPEG given the other interventions going on in these districts during the Project period (e.g., non-formal education, school feeding, etc). While the GPE requested that the RF include a learning outcome indicator, a more appropriate PDO-level indicator for improved service delivery could have been included such as improvements in teacher/student attendance rates or improvements to access or retention. While not in the RF, these indicators were tracked through the SRC analysis and improvements were observed throughout the project period. However, this data is self-reported, and therefore is less reliable than assessment data that was conducted through a standardized national education assessment. Targets for some indicators (disbursement of grants, up to date SRCs) were exceeded. Given the significant challenges in disbursement of grants under the EdSeP, these targets appear to be realistic for GPEG at the design stage. The RF design may have been enhanced if more information had been provided on how targets were calculated. For example, providing information on how targets were set based on observed trends and assumptions on the reach of the intervention. 43. In addition to regular monitoring instruments, two IEs were conducted that assessed the UTDBE diploma program as well as the district and school grant programs. The aim of the UTDBE IE was to: (i) assess the extent to which the distance training program had achieved its objective of upgrading the skill sets of untrained teachers and impacted learning outcomes; (ii) compare the classroom performance and content knowledge of UTDBE trainees with teachers from the conventional teacher training program in Ghana; and lastly to (iii) make an assessment of cost-effectiveness of the program. The district and school grant IE was designed in order to: (i) estimate the impacts of the district and school grants on higher-order outcomes as well as intermediate outcomes at the district and school level; and (ii) to carry out a survey amongst a sample of schools to report on a selection of RF indicators and verify and validate the ongoing M&E system. This IE faced implementation challenges as it became very delayed owing to the slow procurement process in selecting and contracting the consultant and the difficulties redesigning an evaluation after implementation had already commenced. Ultimately the scope had to be reduced which limited the quality of evidence produced from the evaluation.17 The low capacity of MoE to manage a difficult contractor hired for the evaluation was one challenge which ultimately affected the timeline and scope. Limited knowledge of technical aspects of IEs further delayed implementation. Most IEs had been managed by donor partners in the past. Nevertheless, GPEG adopted mitigation measures to improve their capacity, by seeking technical support from the Bank and local research firms (e.g., Innovations for Poverty Action), and conducting IE training for Planning, Budget, Monitoring, and Evaluation (PBME) staff. The PBME is now managing and supporting two IEs under the current IDA-funded Secondary Education Improvement Project (SEIP) project. 17 The study was conducted without a control group and baseline data collection, but rather through a desk review, discontinuity analysis and endline data collection in July 2016. See the 2016 report “Impact evaluation and monitoring of the District Grant and School Grant component of the Ghana Partnership for Education Grant.” by Edburgh consultants – Innovative Services Consortium. 2016. 14 44. M&E Implementation: GPEG M&E implementation had some impressive successes and only minor shortcomings. Led by GES, the strong implementation of M&E activities was integral to the overall success of GPEG. This included the well-prepared annual performance reporting on GPEG, the APW analysis, regular monitoring visits (including joint sector field visits by LEG members) during the lifetime of the GPEG project, internal and external audits, a beneficiary study, and management of two IEs. Examples of strong M&E implementation also included the use of detailed M&E templates, developed in close collaboration with the Government and stakeholders, with key staff in the education system trained in their use. The APW analysis for GPEG was the first time the central Ministry had a clear sense of what activities (and their cost) were being implemented by schools using their capitation grants. Further, annual analyses allowed for a feedback loop between the central Ministry, districts, and schools; results were incorporated into capacity building and outreach activities to best ensure relevant grant activities.18 The coding used for the district grants also allowed better understanding of district priorities. In addition, the IEs provided extremely rich data on management at the district/school levels, parent and community involvement, teacher behavior in the classroom, UTDBE cost-effectiveness, student learning outcomes, and access and completion rates at schools, among other results. The extensive data from the IE, annual progress reports/missions, and the collection of such detailed data (at the decentralized level) were among the standout successes of GPEG. 45. The implementation of M&E under GPEG faced some challenges in terms of RF indicator reporting and IE management. The collection and validation of decentralized data on teacher training, from over 7,000 schools and 75 districts was difficult and could have been improved with computerization of reporting. The number of teachers trained in In- Service Education and Training (INSET) was being reported as the cumulative values of trainings given each year. As a result, the exact number of teachers trained can only be calculated through a proxy indicator for trainings conducted in one subject over one year at the end of the Project. This also led to overly complicated reporting on direct project beneficiaries. 19 In addition, the endline value for the PDO indicator on “P3 students achieving proficiency in English and Mathematics” could not be calculated in the final year of the project as the 2016 NEA changed the grade level tested to P4.20 Unfortunately the team did not learn of this change until July 2016 (one month prior to closing). However, the project benefitted from additional learning assessments conducted during implementation- the USAID supported EGRA and EGMA rollout in 2013 and 2015 and these results are discussed below in section 3. 46. In the end, the MoE was successful in implementing the district and school grant study in a short period, and the Bank team mobilized international experts to support the 18 For example, officers from GES headquarters held quarterly meetings with district officers to check and discuss project activities and status report of the districts. This high quality reporting prior to implementation support missions allowed focused mission discussions to address bottlenecks. 19 Reporting could have been simplified by using the EMIS count of teachers and students in Project districts, as all were direct beneficiaries. 20 The NEA receives funding and technical support from USAID. 15 Ministry. While the IE results have informed project performance, there were some missed opportunities for learning from baseline results and a more robust study design because of delays and challenges in procuring the survey firm by the Government. The problems in contracting were not immediately communicated to the Bank, however once the Bank learned of the difficulties in reaching agreement on deliverables and scope of the research, the team mobilized international experts to help support the evaluation and get the most out of the more limited study design. 47. Utilization: Active monitoring of Project activities were critical to the overall success of GPEG. In general, GPEG did an excellent job of developing feedback loops to inform and improve the quality of GPEG activities and ultimately the impact of the Project. The APW analysis is an excellent model which could serve as a best practice for other Projects using a similar decentralized model of non-salary resource provision. The joint review missions used these APW reports in order to discuss improvements, provide recommendations and determine next steps and Project adjustments. These reports evolved over the course of GPEG with constructive adjustments. Following the MTR, additional monitoring was contracted including a second in-depth fiduciary review and a separate in- depth procurement review. Moreover, the Lesson Observation Sheets (LOS) as part of the UTDBE IE provided a very rich source of data on UTDBE trainee performance throughout GPEG’s implementation. The UTDBE and district and school grant IEs are providing key lessons for the Government and other key stakeholders. For example, the IE findings on the similar performance of UTDBE and DBE-trained teachers but the greater cost- effectiveness of the UTDBE program will be extremely valuable in advocating for the continuation of the UTDBE program beyond GPEG. At the time of Project closing, the Government is considering subsidizing UTDBE trainees for kindergarten teachers since this is still an area where there are huge deficits of qualified teachers. 2.4 Safeguard and Fiduciary Compliance 48. Safeguards: The project was classified as a Category B given the potential environmental and social impacts of district and school grants that might involve minor rehabilitation of existing buildings or construction of new buildings on existing sites. An Environmental and Social Management Framework (ESMF) was developed through consultation and disclosed by the time of project appraisal. The ESMF proposed specific institutional arrangements to enable sustainable execution of all measures identified in the ESMF. The PIM included a negative list of items that could not be funded through district or school sub-grants to discourage items/activities that could trigger additional environmental and social safeguards. Safeguards reviews were conducted during implementation in May 2014 and March 2016. During the May 2014 review, recommendations for screening of minor civil works at the district level were provided and an Environmental Management Plan (EMP) checklist was developed and circulated. Despite the initial delays of undertaking environmental and social screening of the sub- projects, the March 2016 mission confirmed that all minor rehabilitation works undertaken under the project had been screened and were well documented using the Rehabilitation and Construction Works Mitigation Measures Checklist provided in the PIM. The interventions undertaken by GPEG did not trigger additional Safeguards policies nor additional instruments aside from the approved ESMF for the project. 16 49. Fiduciary: Fiduciary compliance is rated Satisfactory. Overall the client has complied with all legal covenants related to FM arrangements - there were no outstanding audits and annual audit reports were received on time and were of unqualified opinion. Interim unaudited financial reports were also submitted in a timely manner and in accordance with Bank procedures. Interim Financial Reports were provided on a quarterly basis. An internal auditor performed independent and objective internal audit activities including inspections and verification of the use of funds at the district and school levels. Procurement under the Project was sound but the delay in collating the district level procurement plans was raised by the project team. Procurement related to project- supported activities at the central level were adequately implemented and documented. At the decentralized levels, while monitoring was adequate, reporting was not comprehensive. After the MTR, the project was able to report on district level procurement and more recently captured school-level information through an in-depth procurement audit. 50. The GPEG also conducted two in-depth fiduciary reviews, one in December 2014 and the subsequent review in August 2016. The most recent in-depth review (conducted by an independent consultant) found overall good FM performance based on a sampling of 35 districts and at least 3 schools per district. The fiduciary review attributed the quality of FM performance to the mainstreaming of FM arrangements with existing Government processes complimented by strict procedures and processes highlighted in the PIM. While capacity of district internal auditors was variable and there was some uneven collation of data by Regional offices, the reviews concluded that the FM systems were in place and being effectively utilized. There were some challenges related to FM which included: (i) changes in the accounting personnel, particularly at the district level; (ii) delays in making payment for district claims (for procurement related activities) which required travel from the districts to Accra; and (iii) poor filing systems in some districts and schools. 2.5 Post-completion Operation/Next Phase 51. Ghana has experienced a number of external and domestic shocks since 2013/2014 due to the decline in the price of oil, gas, and gold which has limited its prospects for revenues. The Government, as a result of the oil crisis, has faced fiscal pressure and may not be able to sustain the level of district and school grants under GPEG. This has had a direct impact on the ability for the Government to sustain certain GPEG activities in its budget, such as grant allocations to districts and base grants for schools. That being said, by working within Government systems and building capacity for better monitoring, planning, and management, GPEG-supported systems have the institutional knowledge to continue to employ activities at the school, district, and headquarters levels without additional budget support. At the closure of the Project, several GPEG activities are expected to continue while others remain uncertain. A sustainability workshop was held and attended by representatives at the central, regional, and district levels. During this workshop it was discussed which GPEG activities and procedures do not require funding to be implemented. The new Government elected in January, 2017 promises to resolve the fiscal gap and ensure more predictable funding at decentralized levels. 17 52. At the activity level, continued GPEG interventions include school and cluster- based INSETs, teacher mentors, empowering SMCs and PTAs, SRC evaluation and APW analyses, and encouraging the use of local teaching and learning materials, amongst others. At a procedural level, there are many no-cost aspects of GPEG that would continue, including appraising performance and promotion of head teachers, more efficient deployment of teachers, simplifying the format of Circuit Supervisor reporting, and continuation of the UTDBE program. The use of supervision and monitoring templates at the district-level, capacity-building trainings on SPIPs and APWs, reliable and predictable release of capitation grants, efficient deployment of teachers, are all capacities created under GPEG which continue to be implemented by the Government. Section 4, Risk to Development Outcome, provides more details on the sustainability of project outcomes and activities. 53. No new GPE grants are currently planned for Ghana but the Government will continue to engage with the GPE on potential opportunities if and when additional allocations become available. In addition, the Bank will be preparing a new Country Partnership Framework (CPF) in which new investments in education are expected to be considered in basic and post-basic education. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives 54. As described earlier, the Project was aligned with the sector priorities of the Government, LEG, Bank, and GPE at appraisal. At closing the Project continues to be very strongly tied with Ghana’s education sector priorities. As the GPEG Project was designed while the ESP of 2010-2020 was being appraised -- it is highly aligned with the ESP goal of providing “equitable access to good-quality child-friendly universal basic education”. The Project was also very relevant medium-term objectives in the Ghana Shared Growth and Development Agenda II (2014-2017), which were to: (i) increase inclusive and equitable access to, and participation in quality education at all levels; (ii) improve the quality of teaching and learning; (iii) promote the teaching and learning of science, mathematics, and technology at all levels; and (iv) improve the management of education service delivery. Similarly, Ghana’s National Action Plan for Education (2016 - 2030) identifies three sub-objectives for basic education, which all very closely align with the GPEG’s objectives. These sub-objectives include: (i) improving equitable access to participation in quality basic education; (ii) improving quality of teaching and learning outcomes; and (iii) promoting sustainable and efficient basic education management and accountability systems for education service delivery. 55. The Project also continues to be aligned with the World Bank’s Country Partnership Strategy (CPS) for Ghana (2013-2016) which was recently extended to 2018. The CPS places emphasis on human capital development as being central to the country’s growth and shared economic prosperity. Consistent with Pillars II and III, the Project focuses on improving competitiveness and job creation while also protecting the poor and vulnerable 18 by targeting improvements in the deprived areas of Ghana which are lagging behind in human development indicators. Between the time that the CPS was designed in 2013 and its extension in 2016, the serious external and domestic shocks Ghana experienced as a result of the decline in the global prices of gold, oil, and gas has limited Ghana’s prospects for revenue. As a result, the priorities of the CPS continue to be relevant. 56. The Project also remains relevant to the World Bank and GPE strategies. The vision of the World Bank Group is the eradication of extreme poverty and promotion of shared prosperity by fostering income growth of the bottom 40 percent in each country. As education is considered essential to achieving this goal, the World Bank Education Strategy 2020 shifts the approach from an emphasis on input-driven programs to a focus on improving learning outcomes, increasing accountability and targeting results, and strengthening the knowledge base on education. The goals of the GPE Strategic Plan for 2016-2020 are three-fold with a focus on: (i) improving and promoting more equitable learning outcomes; (ii) increasing equity, gender equality and inclusion; and (iii) supporting more effective and efficient education systems. With a focus on improving education service delivery in the most deprived areas of Ghana through decentralized decision-making, the GPEG is well in line with the current strategic priorities of many development actors in the education sector. 57. The Project’s relevance of objectives is rated substantial given that the Project’s objectives were and continue to be consistent with the overall goals and sector priorities of the Government and the World Bank. Relevance of Design/Implementation 58. Given the country context at the time of Project design, the strong alignment with Government priorities, and the development of GPEG and the ESP with the LEG, the areas on which the Project focused were appropriate. These project-supported activities were selected and designed to improve the planning, monitoring, and delivery of education services in underserved areas in the country – in alignment with the PDO. The use of district or school-sub grants was highly relevant given the Government and Bank priorities to use and, in turn, to strengthen decentralized systems. Working through district and school systems more directly supported education stakeholders and mainstreamed the mechanisms and tools for managing service delivery. The project supported existing processes and introduced enhancements to strengthen planning, reporting and supervision at all levels. Project activities were highly relevant given that they all had been piloted/taken to scale in Ghana, including the UTDBE program, use of SRCs, and provision of district grants and school capitation grants. The choice of activities was strategic given the original short implementation period of 3 years. 59. The Project’s targeting of “deprived districts of the Recipient’s territory” reflects key priorities of the Government, Bank, and GPE. The Project’s design reflects the situational analysis of wide regional disparities in poverty and human development by enforcing a clear selection criteria for project beneficiaries. The transparent selection criteria for districts included both the district-level poverty index (share of population 19 below the poverty line), weighted at one-third, as well as education indicators (retention in primary education, retention in the basic cycle, share of girls enrolled in P6, share of girls enrolled in JHS3, pass rate in BECE English, and share of trained teachers in public primary schools), weighted at two-thirds. The targeting criteria captures the social context, individual education results, and student flow indicators of schools well, but may have benefitted from additional information on resources available to schools in order for targeting to take into account cost effectiveness. 60. On the basis of the information above, the Project’s relevance of design and implementation is rated substantial. 3.2 Achievement of Project Development Objectives 61. Overall efficacy for this Project is rated as substantial. As shown in the section below – and in greater detail in Annex 2 – the Project delivered all of the expected outputs. At the outcome level, indicators related to all three sub-objectives of improved planning, monitoring, and service delivery were met or exceeded. Outcomes for planning, monitoring, and service delivery are all extremely intertwined as the achievements in education service delivery are largely dependent on good planning and monitoring. The following section provides an assessment of achievement by the sub-objectives of improved planning, monitoring, and delivery of basic education services in deprived districts. Sub-objective 1.1: Improve the Planning of Basic Education in Deprived Districts 62. The GPEG contributed to improvements in basic education planning in deprived districts as evidenced by the successful completion of yearly district annual work planning, school planning activities (School Performance Assessment Meetings, SPAMs, and SPIPs) to access school grants, and more systematic in-school teacher training. Planning activities supported by the project were implemented at the school, district, and central levels and project reporting indicated that plans were implemented to support key education objectives. Progress in this area is captured in PDO indicator 1 as well as Intermediate indicators 4 and 9, all of which were exceeded. Achievement of this sub-objective is rated as substantial. 63. At the school level, GPEG’s objective to strengthen planning was met through the SPAMs, development and implementation of the SPIPs, and management of the school sub-grants. At project close, 100 percent of schools in deprived districts were reported as having completed SPIPs that had been approved by their School Management Committees (SRCs), exceeding the Project target of 75 percent. Furthermore, the execution rate of the school grants increased as the project progressed, from 72 percent in 2012-2013 to 88 percent in 2014-2015 and to 92 percent by project closing. This was achieved through special assistance to head teachers in the preparation of SPIPs and close involvement of SMCs in the preparation and approval of SPIPs. When surveyed, Circuit Supervisors thought that the quality of SPIP preparation had improved since 2012 and over 80 percent 20 of head teachers were able to show a SPIP in the official template with all fields completed when asked.21 64. For grant activities, schools could choose activities from a pre-designed Menu of Activities that provided a list of activities that were eligible for expenditure. The spending of school grants was based upon a SPIP and schools were unable to access their funds if their SPIP was not approved and signed by the Chairperson of the SMC. According to SRC analyses, schools primarily used the grants for facilitating SMC meetings, small repairs, building of toilets and to improve the overall school environment. A fiduciary assessment conducted in a sample of 105 schools found that projects undertaken by the school were inspected by the district office and the expenditure of school grant funds were tracked at the district level. The execution rate of the funds within the same fiscal year increased as the project progressed from 72.3 percent in 2013, to 88.0 percent in 2014, and finally to 92.4 percent in 2015/16. 65. The development and implementation of APWs and management of district sub- grants fueled the achievement of planning objectives at the district level. District Education Directorates were well set up to administer GPEG with 84 percent of districts having established a technical working group; roughly half of which met at least three times per year. The Project exceeded its target for District officers trained with over 2,250 officers trained (of a target of 500), including 308 in reporting, monitoring, and evaluation and 329 in internal controls and auditing. Moreover, all District offices entered SRCs into the district’s software to improve planning, exceeding the target of 70 percent. The impact evaluation of the program found that District Directors and Circuit Supervisors, who visited schools to observe teaching practices and provide advice on school management at the primary level, are frequently using the GPEG funded management tools including SRCs, SPIPs, school assessment meetings, annual training plans, and GPEG working groups. There was a substantial increase in the number of districts which have district SRC summaries available – from 73 percent in 2012-2013 to 99 percent in 2014-2015. 66. Ultimately, 100 percent of deprived districts disbursed 75 percent or more of their district education sub-grants according to their APWs, exceeding the PDO-level target of 80 percent. In fact, the average total disbursement against planned activities in the APWs was over 98 percent at Project closing. District Annual Programs of Work were created by districts from a Ministry of Education approved Menu of Activities (included in the Project Implementation Manual). The APWs were created in workshops wherein districts were assisted by District Education Directorate staff, MOE and GES headquarters planning staff and Regional Directorates and district level education statistics from EMIS were used to inform decision making. According to APW analysis, the most popular district activity by count and expenditure was the provision of school supplies (chalkboards, desks, chairs, etc.) with other facility/supply related activities also ranked highly. Provision of teaching and learning materials and gender-friendly toilet and urinal facilities were among the top five activities by both count and expenditure. Management and training activities were particularly popular, especially related to providing support to school grant planning and 21 GPEG District and School Grant Impact Evaluation, 2016. 21 expenditure, training officers in Excel, operational expenses, and training for heads of schools in the use of the school grant. More detailed information on district grant activities can be found in Annex 2. 67. Achievements at the central level were also considerable with capacity-building provided for the management of project components. The APW analysis, for example, in which district grant activities and spending were consolidated and analyzed on a yearly basis, allowed GES/Ministry staff to properly monitor both spending and activity choices at the district level in line with their APWs. Quarterly workshops/meetings with district directors contributed to the feedback loop and informed the preparation of the following year APW. This process allowed GES/Ministry staff to both advise and ensure purchases were of good quality and of relevance to the schools. Project reporting in GPEG was noted as a best practice by the GPE Secretariat. Sub-objective 1.2: Improve the Monitoring of Basic Education in Deprived Districts 68. Available data also indicate improvements in monitoring of basic education at all levels through SRCs, district analysis and M&E reports. Achievement of this sub- objective under the Project is rated substantial – with progress in this area captured in PDO-level indicator 2 as well as intermediate-level indicators 1, 2, 6, 7, and 8. Out of 1 PDO-level indicator and 5 intermediate-level indicators for this sub-objective, all 6 indicators were met or exceeded. 69. At the school level, the GPEG aimed to improve monitoring through better implementation of SRCs, which would then inform SPIPs and ultimately the use of school grants. This was achieved with 98 percent of schools in deprived areas with up-to-date report cards (PDO-level indicator 2) and 93 percent of schools displaying their SRCs on public notice boards by the last year of the Project (intermediate-level indicator 6). Improvements in the updating and utilization of SRCs were driven by the Project’s training of 5,993 head teachers and 1,094 circuit supervisors in the utilization of SRCs (intermediate-level indicator 1) as well as increased visits by Circuit Supervisors (intermediate-level indicator 7). An analysis of EMIS data from 2008 indicated that circuit supervisors visited schools more than once a year in only 77 percent of schools 22 . In comparison, all GPEG schools (confirmed by district reports) were visited over 7 times per year on average in 2016. 70. The project activities also led to improved monitoring at the regional levels (REDs). Although compliance was low for the initial two years of the GPEG at only 13 percent in early 2015, by the end of the Project all REDs submitted adequate reports capturing data from all districts in their region (Intermediate Indicator 8). This was achieved through the training of 2,288 regional and district officers on financial management, data collection, and other key areas (Intermediate indicator 2). In addition, districts aggregated SRC results, helping them to have consistent information about all basic education schools and 22 Analysis was conducted by WB and GES and reported in “Basic Education Beyond the MDGS in Ghana” Balwanz and Darvas, World Bank, May 2013 22 ultimately make more informed decisions about their district priorities (and APW expenditures). The survey from the impact evaluation found that 97 percent of District Education Directorates agreed or strongly agreed that data from the SRC had helped them in improving girls’ school attendance by an increased focus on gender in their APW activities. District Education Departments collating SRCs from at least 70 percent of schools within the district (Intermediate Indicator 4) was also exceeded as 100 percent was reported in the final year of the project. 71. At the central level, improved education monitoring of deprived districts was evidenced through twice-yearly monitoring visits by central level staff members as well as through the APW analysis, the incorporation of a Lesson Observation Sheet monitored by the UTDBE IE to measure teacher performance, and the implementation of two impact evaluations of the Project. Apart from GES/Ministry staff, there was also project monitoring by civil society in which two independent reviews were conducted during the life of the Project. The GPEG project generated an impressive volume of education data which were incorporated into the annual Education Sector Performance Reports and presented at annual NESAR events during the lifetime of the Project. Sub-objective 1.3: Improve the Service Delivery of Basic Education in Deprived Districts 72. Achievement of the sub-objective of improving basic education service delivery in deprived districts under the Project is rated substantial. The level of achievement of this sub-objective is determined on the basis of RF indicators met or exceeded, impact evaluation reports for the UTDBE and grants to districts/schools, education indicators as reported by EMIS, as well as proxy indicators from other sources. The annual Education Sector Performance Reviews (based on annual school census data) also confirmed improvements in basic education indicators with regard to enrollment, trained teachers, completion and girls’ participation. As education service delivery can encompass many different types of activities, this report will view them in terms of the quality of teaching, quality of learning, access, and school management. All 3 PDO indicators related to this sub-objective were achieved/exceeded. For the two intermediate indicators, one was exceeded and the other was met through a proxy indicator for training. 73. The Project had many significant gains in terms of improving the quality of teaching in deprived areas. The percentage of trained teachers in deprived districts increased between 10-18 percent in kindergarten, primary, and junior high-levels between 2012-2013 and 2015-2016. While the percentage of trained teachers also increased in non- GPEG districts, the GPEG districts received between 3-5 percentage point gains over the non-GPEG districts allowing deprived districts to narrow the gap for this indicator. For example, the percentage of trained junior high school teachers jumped from 72 to 82 percent in GPEG districts, compared with 87 to 92 percent in non-GPEG districts. This can partially be attributed to the influx of 6,480 (of a targeted 5,000) teachers trained by the UTDBE program (Intermediate indicator 3). 74. As part of an impact evaluation on the UTDBE program, a representative sample of UTDBE teachers were observed teaching in their classrooms and rated based on their lesson planning and preparation, teaching methodology, and classroom organization and 23 management. UTDBE trainees’ scores in all three areas improved from participation in the course and exceeded Project targets, although performance was highest at the time of the midline survey. PDO indicator 4 measured the percentage of teachers trained who obtained a satisfactory rating or higher in the lesson observation sheet. At midline, 87 percent of UTDBE trainees (of a targeted 50 percent, and baseline of 69 percent) had obtained a satisfactory rating in their lesson observation assessment. 75. Compared with traditionally trained teachers in the DBE program, UTDBE trainees under the Project showed higher levels of content knowledge by out-performing their Diploma in Basic Education peers in the examination results of the “Trends in Education, School Management, and Integrated Science” course. The performance of the UTDBE teachers in this Project is of additional significance given that these teachers are from the local school communities and therefore speak local languages, are more integrated within the community, and have a much higher likelihood of remaining to teach in the deprived areas. As a result of the success of the UTDBE program through GPEG, the Government is considering subsidizing UTDBE trainees for kindergarten teaching as this is an area with large deficits in qualified teachers. The knowledge base of teachers was also improved in the Project through in-service trainings (INSET) in the core strategic areas of numeracy, literacy, and science as well as other areas such as early childhood education and gender sensitivity. Over the course of the Project, over 94,000 INSETs were provided with at least 42,541 teachers (of a targeted 30,000) trained in at least one core INSET subject (Intermediate indicator 3). Similarly, it is estimated that over 6,500 of basic schools in GPEG districts participated in all INSET core courses as part of GPEG (Intermediate indicator 5). 76. The Project showed some progress at the student learning outcome level, although given the short project timeline it is quite unlikely that advances in student learning outcomes would be attributable to the Project. The third PDO indicator related to the National Education Assessment results for P3 students was achieved during the implementation of the project in 2013, surpassing the target, [16 percent (of a targeted 15 percent) of P3 children achieved proficiency in English and 13 percent (of a targeted 11 percent) in Mathematics]. Such improvements of 29 percent for English and 44 percent for Mathematics in NEA over the project lifetime are significant and compare favorably to other countries in the region. Unfortunately a confirmation of further improvements in learning outcomes through additional data at endline was not available as the 2015 NEA was postponed to 2016 and the level of assessment was moved from P3 to P4 level. This was an unexpected change that was communicated in the last month of the project. As mentioned earlier, this initiative is funded by USAID and Government followed the technical advice on improving the assessment. 77. However, Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA) for P2 students in GPEG districts between 2013 and 2015 were also conducted during the project period (with USAID support) and confirmed that there were statistically significant improvements in number identification, letter sounds in English, oral reading in English, and letter sounds for local languages in the GPEG deprived districts. The District and School Grants Impact Evaluation provides an analysis 24 of the results. This study also analyzed results of the BECE23 exam (end of basic education- grade 9), even though project effects on student performance at the end of basic schooling would only be measurable with a much longer project duration. The results showed that the number of students taking the exam between 2012 and 2015 increased by 26 percent in GPEG districts in comparison to only 12 percent in non-GPEG districts. BECE pass rates in GPEG districts have declined and the gender gap in BECE pass rates have widened in comparison with non-GPEG districts during this time. One explanation for these results is the effect the GPEG project has had on increasing access to education, allowing for larger class sizes, fewer textbooks, and increased burden on teachers at that level. It would be interesting to follow BECE results in GPEG districts over a longer time period in order to see if those trends reverse. 78. The project outputs of improved percentage of qualified teachers in deprived districts, in-service teacher training and the purchase of teaching and learning materials from district and school grants contributed to improved learning outcomes observed within the deprived districts targeted by the project. While the proportion of trained teachers has increased nationally at all levels, the increase in GPEG districts has outpaced that of non- GPEG districts across all levels. The rate of increase in GPEG districts is 4 percentage points higher than the non-GPEG districts for KG, 3 percentage points higher for Primary, and 5 percentage points higher for JHS (EMIS 2016). 79. Gains in student access and retention in GPEG districts can also be attributed, at least in part, to the Project. The District and School Grants IE found that the Project had a positive impact on access, in comparison with similar non-GPEG districts at the Kindergarten, Primary, and JHS levels. For example, compared with similar non-GPEG districts, GPEG districts saw an additional 5 percentage point increase in Kindergarten net admission rate (NAR). The Net Enrollment Rate (NER) for Kindergarten, Primary, and JHS improved by 17, 4, and 2 percentage points respectively more in GPEG districts than in non-GPEG districts. In GPEG districts, dropout rates for Kindergarten increased during the time period, while staying constant at the P5 level (and slightly decreased in comparable non-GPEG districts) and decreased at the JHS2 level (but at a lower rate than in non-GPEG districts). Specifically related to gender, the APW analysis of this Project shows that on average, one-third of all district grant activities were supporting gender related-activities, the most popular of which was providing gender friendly sanitary facilities in basic schools. 80. There is also some evidence of improvements in school management as a result of the GPEG Project. Nearly 90 percent of head teachers reported that they provided coaching to teachers at their school with 75 percent reporting that lesson plans were reviewed weekly. It was also found that Lesson Observation Sheets were being used in approximately 70 percent of schools in a combination of self-evaluation and evaluation activities. According to the APW analysis, nearly 40 percent of all district grant activities were management related. 23 BECE is the Basic Education Certificate Examination at the end of Junior high school- a normative exam that determines placement in secondary education. 25 3.3 Efficiency 81. Overall efficiency of the project has been rated as Substantial. The findings that are detailed below and in Annex 3 indicate that the Project benefitted from exchange rate gains and cost effective implementation of activities, allowing the project to achieve and exceed many of its targets. 82. In terms of efficiency of district and school grants, the analysis found that district grants were allocated based on relative deprivation (the targeting mechanism worked). Two district level activities (accounting for 24.81 percent of overall district level expenditure) – construction of urinals and toilets and in-service teacher training - were analyzed and both activities were found to be relatively cost effective in comparison with government and other donor led initiatives in Ghana. At the school level, school grant disbursements were found to be highly correlated with improved planning (through SPIPs) and monitoring (through SRCs). A difference in differences analysis showed that in line with the available literature on district and school grants, districts that received grants witnessed substantial improvements in gross and age-appropriate enrolment (as compared to non GPEG districts over the same period). However, not all the improvements in access can be attributed to the project as other programs were operating in deprived districts during the same period. In terms of efficiency of the UTDBE program, the following was determined: UTDBE and the traditional teacher training (Diploma in Basic education, or DBE) model produce teachers with similar skillsets though UTDBE is more cost-effective. The program has led to a convergence in the percentage of trained teachers within GPEG and non GPEG districts though so far, the upgrade of teacher skills has not translated into vastly improved completion or transition rates. 83. As outlined in the PAD, to receive district grants, each district submitted an APW which detailed their planned activities. Districts planned for activities along four cross cutting thematic areas: Access, Quality, Gender, and Management found in a pre-approved Menu of Activities. At the end of each year, districts reported on the progress in implementing their activities within their APW, and were reported on actual expenditure, timeframe and quantity. The analysis of the executed APWs reveals that districts conducted 4,983 activities over the project period. These were comprised of 76 unique activities. 84. A comparison was made between the unit cost of construction of toilets and urinals under GPEG with identical models constructed under UNICEF and under government procurement, an activity that accounted for 18 percent of district grant expenditure (see Annex 3 for more details). The average cost for a 4 unit VIP was GH¢ 32,801 under GPEG’s decentralized process as opposed to GH¢ 36,083 under UNICEF and GH¢ 40,100 under the Government’s Funds and Procurement Management Unit (FPMU). Construction of urinals under GPEG was also found to be the most cost-efficient at GH¢ 5,665 per unit in comparison with GH¢ 6,368 under UNICEF and GH¢ 5,950 under FPMU. 85. The cost of providing in-service teacher trainings were also most cost-efficient using the INSET model under the GPEG. For example, under the GPEG Project the unit cost per participant in an INSET Mathematics training was GH¢ 39.48 in comparison with 26 the same training administered by the government in a non-GPEG district at GH¢ 45.8. Cost-comparisons of INSET trainings in literacy and science followed similar patterns. More details are available in Annex 3. 86. Analysis of the UTDBE program found the program to have an efficiency ratio of 1.6 in comparison to the traditional teachers’ diploma program, DBE. After cost sharing with the student, the cost of the training to the program was US$2,130 in comparison with US$3,409 per trainee in the DBE program. The attrition rate of 20.43% was also similar to the attrition rate of the DBE program. Training UTDBE teachers has sharply increased the percentage of trained teachers and the Pupil Trained Teacher Ratio in deprived districts. This convergence in the percentage of trained teachers between GPEG and non GPEG districts is likely to remain persistent as most UTDBE teachers had been teaching in GPEG districts for several years prior to being selected for the program and have roots within the communities in which they reside. 87. Several aspects of project design and implementation also attributed to project efficiency include strong project targeting and focus on equity and the mainstreaming of project activities through government systems. At the end of the project, districts were re- ranked per the original selection criteria applied during the design of the GPEG to see whether deprived districts have changed in ranking. Using the more updated indices, 57 of the 75 GPEG districts remain in the top 75 most deprived districts (or bottom third of the 216 districts). However, 18 districts (24% of the original deprived districts) can be considered to have transitioned from the deprived districts list. 88. To summarize, the GPEG project was able to successfully complete all project activities within a 3 and half-year period. Internal and external audits concluded that district and school funds were used for their intended purposes and in an efficient and cost- effective manner. Component level expenditure was in line with the budgetary allocations made during appraisal and cost savings from currency depreciation were redirected towards increased allocation for district grants. 3.4 Justification of Overall Outcome Rating 89. Rating: Satisfactory. Given the Project’s substantial ratings in Relevance, Efficacy, and Efficiency, the overall outcome rating of Satisfactory is appropriate. Project Relevance Achievement of PDO Efficiency Overall Rating (Efficacy) Substantial Satisfactory Substantial Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 90. Largely a result of the targeting mechanism that took into account poverty and education outcomes, the GPEG Project has meant better education opportunities for children in the most deprived areas of Ghana. As a result of the Project, children from 27 GPEG districts have improved access to education and an improved school environment and there are significantly more trained teachers then prior to the project. Since some of their trained teachers are graduates of the UTDBE program – it is likely that teachers will speak their local languages and remain teaching in these communities far longer than other trained teachers at the school. In addition to funding teacher training, the school and district grants were used to rehabilitate school infrastructure, such as bathrooms, making it easier for girls to remain in school. 91. The complementarity of the DfID-funded GPASS program24 and the GPEG project further supported and enhanced achievements under the GPEG operation, particularly with regards to increasing girls’ access to and retention in schools. The decline in girls’ dropout rates at the JHS level in GPEG districts outweighed the decline observed in comparable non-GPEG districts. This may be due in part to the GPEG as well as to the GPASS program which covered, among other costs (school fees), sanitary and other personal items needed for girls to remain in school. The Gender Parity Index (GPI) for GPEG districts was maintained at the kindergarten level, slightly decreased at the primary level, and increased at the JHS level.25 92. As a result of the systematic school performance improvement planning and school grant implementation, SMCs were re-energized under the GPEG. While SMCs existed in most schools across Ghana, their functionality varied widely. The School Grant manual and SPIP process (planning, sign-off and vetting of the plans) promoted more active engagement of SMCs in the schools in deprived districts. As mentioned before, a 2010 World Bank survey had found that SMC members did not have enough information about their roles and responsibilities or about SPIP and SPAM mechanisms. The GPEG aimed to address this by supporting training for all project SMCs given their key role in designing, vetting and ultimately approving the SPIP (a requirement for any school to receive a grant under the Project). (b) Institutional Change/Strengthening 93. By using the existing government structures and building capacity at the district and school levels, the Project has strengthened management procedures and mechanisms at the decentralized levels. GPEG has directly supported the Government’s decentralization agenda and the 75 participating GPEG districts now have a much stronger capacity in key functional areas, including budgeting, planning, and monitoring. For example, district staff are able to effectively carry out supervision and monitoring of activities, including capitations grants, and are able collate and evaluate SRCs and generate their APWs. This is likely to continue to have an impact beyond the GPEG Project with potential spillovers into other sectors. Furthermore, the graduation from deprived category 24 The scholarship program, spanning from 2013 to 2018, was designed to target 55,000 needy girls in the 75 deprived districts. In the 2016-17 academic year an additional 5,000 JHS3 girls will receive a scholarship package, bringing the total number of beneficiaries to 60,000 JHS girls. 25 As the GPASS program is still underway, an impact evaluation of the GPASS program in 2018 will help to better understand achievements in attracting and retaining girls in GPEG districts. 28 of 18 GPEG districts out of the 75 deprived districts in Ghana speaks to the institutional growth in those districts during the life of the GPEG Project. 94. Targeting for the Project had a lasting impression on Ministry staff. With a substantial risk of political influence in the selection of districts for the Project, it was important that the targeting be transparent and fully owned by the Government. In interviews for this ICR, Ministry staff were proud of the targeting mechanism for GPEG because it used clear education and poverty indicators as criteria for selection. Joint ownership of the process has allowed the Government to continue to use a robust targeting mechanism in other projects, such as SIEP and the USAID supported Ghana Partnership for Education Project. The policy discussions on the new ESP (2016-2030) and new programs includes more debate about equity and targeted interventions than had previously been considered during the 2010 ESP. 95. Capacity-building at the central level of the MoE has further facilitated the preparation and implementation of other large projects. The Ministry and GES have negotiated several large new programs since the GPEG rollout. Central Ministry staff reported that their experience under the GPEG Project was useful in the efficient preparation of the PIM for the SEIP. (c) Other Unintended Outcomes and Impacts (positive or negative) 96. An unintended positive impact has been the interest generated by the NGO/CSO community to track grant financing and independently review district and school grant activities. During the GPEG, there were several reviews conducted (GNECC 2013 and SEND Ghana 2015) that verified grant allocations, utilization, and community satisfaction. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 97. A Beneficiary Survey was undertaken with key stakeholders involved in the Project and a Stakeholders Workshop was held on September 28, 2016. A summary of the results of each activity are presented in Annexes 5 and 6. 4. Assessment of Risk to Development Outcome Rating: Moderate 98. The risk to development outcome is rated moderate for a number of reasons. First, the mainstreaming of activities into government systems allows for Project activities to continue beyond the life of the GPEG Project. However, the fiscal crisis slowed growth and reduced revenues that had been anticipated for continuation of district and school grants and elections in late 2016 made it difficult to forecast funds available for education without an approved budget. The financial cost of continuing the school and district grants in the deprived districts is estimated at US$19.3 million annually26. The government has 26 No school grants have been disbursed in 2016 apart from the preexisting Government-funded capitation grant (of approximately US$0.49 per student per semester). 29 no plans to increase the capitation grants provided to schools or increase district grants, leading to no recurrent fiscal implications for the government. Nevertheless, because GPEG was embedded within existing Government systems and not through a Project Implementation Unit, many of the outcomes and gains can be maintained in the post- project phase. A sustainability forum was held in February 2016 to discuss and determine which GPEG-funded activities could be continued beyond the life of the Project with minimal or no external funding. At the school level, SPAMs are now functioning and districts are able to enforce the preparation of SPIPs regardless of the level of funding in terms of school grants. However, there is real concern that continued delay in the release of the capitation grants will make it difficult for schools to implement their annual performance plans. While the scale and timeliness of grants are much more uncertain after the closing of the GPEG, with the systems established by GPEG, any mobilization of resources could be quickly channeled to schools. Moreover, districts are equipped to collate and evaluate SRCs and generate their APWs based on the needs of their districts. At the central level, MoE and GES institutions have been strengthened and staff are well versed in project targeting, M&E, and reporting which will continue to enhance management. 99. Additionally, some elements of GPEG are already being expanded by other donors. The strong collaboration and wider ownership of the GPEG project amongst development partners has led to the continuation of scholarships for girls in GPEG districts through 2017 (DfID) and the introduction of more efficient mobile monitoring tools for local level activities (USAID and UNICEF). That being said, there are sustainability concerns around the GPASS program as survey results have indicated that most of the GPASS girls are unlikely to be able to afford Senior high school education and continue their education beyond the GPASS scholarship27. The Bank-supported Secondary Education Improvement Project (funded with an IDA credit of US$156 million) is working to link the scholarship recipients from the GPASS with the scholarship program under the SEIP. 100. The high turnover of district staff and teachers who have been trained raises some sustainability concerns, while the UTDBE program shows great promise for retaining qualified teachers in underserved areas. Results from the IE showed that one-fourth of District Directors were a year away from retirement. With the median age of District Directors at 57 years old, skills gained by these district staff will be lost to the districts unless proactive steps are taken for refresher trainings and transfer of skills to new staff members. At the school level, the IE found that Head Teachers trained by GPEG have been at their post for an average of 3 years. Because teachers may not be keen to remain at remote posts, and/or as head teachers are close to retirement, it is expected that some of the school-level institutional strengthening will be lost when Head Teachers either retire or transfer out to schools in non-deprived areas of the country. On the other hand, the Project demonstrated that the UTDBE-trained teachers are of similar quality to DBE teachers. The large number of UTDBE teachers trained under the GPEG Project are not only more cost- effective, but they are able to provide mother-tongue instruction, and are bonded to their 27 This situation may change with the introduction of free senior high school education currently proposed by the new Government to commence September 2017. 30 schools for 4-6 years following the program with a high likelihood of staying longer due to personal ties to the community. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Highly Satisfactory 101. The Bank’s performance in ensuring quality at entry is rated Highly Satisfactory based on the following: a. The PDO was highly relevant and remained so throughout the Project, as discussed in Section 3. The PDO was well aligned to Ghana’s ESP, the Bank’s CPS, and GPE strategies. While the focus was on processes and less on learning outcomes, the project tried to incorporate assessment of quality by tracking performance for the deprived districts on the NEA as well as the EGRA/EGMA instruments. b. The preparation task team incorporated lessons learned from previous GPE grants in Ghana, the recently closed EdSeP, other donor funded programs and comprehensive analytical work under the ESP. The GPEG built upon programs that had already been piloted in Ghana and further strengthened their implementation with additional capacity-building and adopted objective criteria to support pro-poor targeting. The preparation team also incorporated the guidance received through the QAR and QER in 2011. Technical aspects of the Project were well developed and appropriate for the sectoral, institutional, and policy context in Ghana. The Bank team benefitted from extensive quality assurance through the ESP preparation and the GPE application process. The additional support from DPs included additional studies conducted by USAID and DFID with regard to M&E systems and gender issues respectively. c. The strong engagement with the country partners (including the LEG) and the strategic decisions to mainstream project management facilitated excellent collaboration. Parallel systems were not established, thereby strengthening the existing systems, capacities and roles of key education agencies. d. M&E arrangements designed for the Project were strong, with very rich and diverse M&E data, from both internal and third-party sources, built into GPEG design. The comprehensive project implementation manual included sample templates for all monitoring activities to help guide implementation. e. Risks identified at entry were appropriate, and the mitigation measures proved to be adequate. f. Fiduciary and social and environmental aspects of the Project were well designed with extensive guidance included in the project implementation manual, school grants manual and Environmental and Social Management Framework. 31 (b) Quality of Supervision Rating: Satisfactory 102. The Bank’s performance with regards to quality of supervision is rated Satisfactory based on the following: a. The Bank team possessed an appropriate skills mix (education, fiduciary, environmental and safeguards expert, procurement) for supporting and supervising the Project. Joint missions (with the LEG and Government) were conducted regularly and Bank staff was commended for their availability to support GPEG. There was minimal staff turnover allowing for enhanced continuity. The same Bank team prepared, implemented, and closed the project. The team received consistent and constructive support from the country office, in particular from the country management and fiduciary/safeguards staff. A generous preparation and supervision budget provided by the GPE contributed to the ability of the team to mobilize support on an as needed basis. b. Risks and implementation issues were identified and addressed in a proactive manner, and highlighted and discussed in detailed supervision documentation (e.g. Implementation Status Reports, Aide Memoires, MTR report). Project ratings were realistic and in line with implementation progress. The Bank team maintained a strong focus on the achievement of the PDO throughout the life of the Project. The Bank team was commended in the Government’s Project Completion Report (PCR, a summary can be found in Annex 7) for securing all approvals (e.g. for IFRs, no objections, work plans, budgets, etc.,) with minimal delays and for prompting the Government to submit relevant documentation at the appropriate time. Reacting to Project needs, the Bank provided intensive support during the first year of the Project, as seen in supervision missions and costs) to ensure the Government was ready to implement. The Bank also supported regular communication with the Government by supporting the establishment of a videoconference facility within the MoE. c. The Bank team was of significant support in helping the Ministry overcome capacity challenges in managing the school and district grant IE. Nevertheless, additional attention to RF calculations and reporting during supervision missions may have prevented some challenges in reporting. d. The Bank worked with a number of the other DPs – including working closely with the LEG while also benefitting from GPE Secretariat support and advice throughout project supervision. As mentioned previously, the degree of local donor involvement in GPEG preparation and supervision was greatly appreciated by the Government. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 32 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 103. Overall, the Government demonstrated a high level of commitment to the achievement of the PDO as well as ownership of activities supported under the Project. At the design stage, the Government helped facilitate the targeting mechanisms for the Project, allowing GPEG to be truly focused on supporting the most underserved areas of the country without political interference. The Government also helped ensure that Project preparation and appraisal were jointly conducted with the LEG to ensure that the GPEG was well aligned with ongoing and new commitments of development partners. During implementation, the Ministry maintained close oversight of the Project and met regularly with the local Bank team and during missions. GPEG was reported on at monthly LEG meetings. Four Government-led NESARs were held during the Project period and GPEG was also highlighted in the Government’s annual ESPR. A Project Steering Committee including the Ministries of Finance and Local Government and Rural Development provided additional oversight to GPEG, although they did not meet as frequently as expected, with only four meetings during the Project lifetime. 104. The Ministry of Finance was supportive of GPEG and ensured a quick turnaround on legal and formal requests to the Bank. However, the delayed disbursement of capitation grants throughout the project period reduced the ability of the school grants to be supplementary. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 105. The main implementing agency was the MoE and GES. Within the MoE and GES, various department were responsible for specific activities and the PBME unit played an overarching role in coordination and data collection and analysis. FM and Procurement were handled by units within MoE and GES. Overall, MoE/GES demonstrated strong commitment and a high level of ownership of GPEG with the Project embedded within MoE systems. A Project Coordinator was appointed by GES to work across all of the implementing agencies and reported to the Director General of GES. The Director of the Teacher Education Division (TED) was responsible for management and reporting on UTDBE and INSET. In its role as implementing agency, MoE and GES achieved the following: a. Disbursed 80 percent of all Project funds and delivered on nearly all Project activities within the original 3 year period. The project extension of 10 months allowed for disbursement of remaining project funds and delivery of all planned assessments. b. Development of the PIM which remained a reference document throughout the GPEG project at headquarters, district, and school levels. c. Compliance with Bank’s fiduciary and safeguards requirements. 33 d. Effectively verified, analyzed and reported on extensive data collected from the Regions and third-parties as well as conducted their own analyses (APW analysis, beneficiary study, SRC analysis, etc.). The sheer quantity of monitoring and evaluation data provided strong feedback loops and detailed analysis. The team also created four GPEG documentaries which are available on the GPE website. e. Proactively identified, resolved, and reported implementation challenges during the Project. In particular, GES is commended for quarterly meetings with district officers to check and discuss project activities and status report of the districts, as well as the high quality project status reporting by the Ministry prior to implementation support missions. f. The GES and MOE provided comprehensive annual status reports on GPEG implementation, organized joint supervision missions with the LEG, conducted annual NESARs, and published annual ESRPs which included special disaggregation for GPEG districts. 106. MoE and GES performed consistently well during the lifetime of the Project, with only some minor shortcomings in M&E. These shortcomings included identifying and addressing challenges in calculating a few RF indicators (teachers trained under INSET, P3 student outcomes with the switch of NEA to measure P4 achievement) prior to the end of the GPEG and handling of the district and school grant impact evaluation procurement to avoid a de-scoping of the evaluation design. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 6. Lessons Learned 107. Lessons learned from previous projects can play an important role in designing and implementing a short three-year project. Building on lessons learned from the prior EFA-FTI grants and the Bank-supported EdSeP allowed smooth and expedited transition/uptake of the GPEG Project. GPEG’s decentralized model worked well because it had benefitted from piloting under the EdSeP. Capacity building under EdSeP and other DP supported operations helped ensure institutional readiness. The inclusion of a lessons learned annex in the PAD (with evidence from other countries as well) also informed the design of the activities. Most importantly, the GPEG focused on the piloted and field tested activities. GPEG PDO-level and intermediate indicators were realistic and consistent with the Project timeframe. 108. GPEG capitalized on the consistency of political leadership. The Education Minister and other key policymakers remained constant throughout GPEG preparation and implementation. The Minister of Education was appointed just prior to the GPEG launch and participated in the closing workshop of the project held in September 2016. Since there was no parallel structure established to manage the GPEG, the responsibilities were embedded within the structures of the MOE and GES. Therefore, at the technical levels there also was minimal turnover of staff. 34 109. Strong project ownership and accountability can be strengthened by embedding the project in regular processes and within the government system. The level of ownership of the Project by the Government and stakeholders can play an important role in determining an operation’s overall success and long-term impact. The beneficiary assessment acknowledged the success of the GPEG Project in empowering the existing systems and helping to make them better functioning. Further, the capacity- building under the GPEG Project followed a “learning by doing” approach which allowed for more direct involvement of the stakeholders. The implementation arrangements were further influenced by past challenges when responsibilities were not in direct correlation to roles of the government agencies. For example, under previous projects, (e.g., EdSeP) implementation often faced challenges because the Ministry of Education was tasked with implementation, yet is not set up for playing such a role. Under the GPEG, the bulk of implementation rested with the GES which has the mandate for implementation particularly at the local level. GPEG also aimed to strengthen the decentralized system by empowering the Regions and Districts as they were critical in supporting the collection and collation of data. 110. More investments in project preparation can lead to subsequent efficiency savings through enhanced cooperation between the Bank, the Government and donor partners. Significant analytical work conducted just prior and during GPEG preparation provided a strong technical foundation for the selected activities. GPEG was also greatly advantaged by consistent Bank support and adequate preparation and supervision budgets. During the project preparation phase of GPEG, audio conference meetings were held weekly, which ensured that all stakeholders were aligned and roles and responsibilities was clear, leading to smooth flow of disbursements throughout the project period. 111. Investing in engagement with donor partners can lead to greater synergies between donor efforts and the project. For example, cooperation with DFID during project preparation led to the Girls PASS program prioritizing the GPEG deprived districts for the JHS scholarship and enhanced cooperation in monitoring and supervision. Joint supervision missions allowed for greater engagement among development partners around GPEG objectives. 112. Through transparent and objective targeting of deprived districts, incentives can be shifted to allocate resources where they are most needed. Despite initial concerns about political capture of district selection, the scientific approach of using poverty and education indicators as criteria in the selection and targeting of districts received full political support and was a source of pride at project closing. 113. Expenditure tracking at the district and school level can provide valuable information on education needs at the decentralized level. The project was able to collect detailed information on district and school expenditure priorities in the deprived districts and whether actual expenditures were in line with the budgeted amounts. Apart from improving capacities in decentralized forecasting, this information on expenditure priorities is useful to the Ministry and for academic research. 35 114. The UTDBE program is a cost-effective alternative to the DBE teacher diploma program and holds the most promise for increasing the numbers of trained teachers in underserved areas. The impact evaluation found that UTDBE-trained teachers perform at similar levels as conventionally trained teachers, but with a significant cost savings to the Government. UTDBE teachers are from local school communities and, therefore, speak local languages, are more integrated within the community, and have a much higher likelihood of working and remaining in the deprived areas of the country. 115. Systems need resources and strong M&E to work. The Project had a rich variety of internal and external data sources to measure the project’s performance and to identify challenges. GPEG made significant improvements to the frequency and quality of M&E in the system and strengthened supervision at the school and district levels. It is equally critical that adequate financial resources are provided to carry out the responsibilities and activities required within the M&E system. 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors N/A (a) Grantee/Implementing agencies (b) Cofinanciers/Donors (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) 36 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Sub-Grants to Deprived Districts 42.36 48.41 114.3 School Sub-Grants 22.06 20.38 92.4 Project Management and 4.58 6.71 146.5 Institutional Strengthening Total Baseline Cost 69.0 75.5 Physical Contingencies 3.25 0.00 Price Contingencies 3.25 0.00 Total Project Costs 75.5 75.5 Total Financing Required 75.5 75.5 (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Global Partnership for Education 75.5 75.5 100 Program Grant 37 Annex 2. Outputs by Component 1. According to the Ghana Partnership for Education Fund Grant Agreement, the Project Development Objective (PDO) was to improve the planning, monitoring, and delivery of basic education services in deprived districts of the Recipient’s territory (Ghana). The PDO as described in the PAD is identical. There were a total of 5 PDO indicators and 9 intermediate indicators at Appraisal. Project restructuring had no effect on indicators or targets. 2. The PDO was not revised during the lifetime of the project. There were two minor restructurings of the project in 2013 and 2015 in order to add works category to the district grants (2013), extend the project closing date by ten months and introduce reallocation of funding among expenditure categories (2015). The project consisted of three components with a total project cost of $75,500,000. In 2014, a re-delineation of districts in Ghana increased the number of project districts for GPEG to 75. This did not alter the project target group as the 18 new administrative offices were carved out of the original 57 targeted districts. There were no major changes in the components and only minor changes in project activities and allocations across categories. 3. Reallocation of Proceeds. The allocation of loan proceeds from the second restructuring was revised as follows: Allocation % of Financing Category of Expenditure At Appraisal Revised Goods, non-consultants’ services, consultants’ services, Training and Operating costs 19,700,000.00 14,936,759.52 100% under the Project (other than for Sub-grants) Goods, works, and services for Sub-projects to be 27,300,000.00 38,941,702.00 100% financed out of the proceeds of District Sub-grants Goods, works, and services for Sub-projects to be 22,100,000.00 21,621,538.48 100% financed out of the proceeds of School Sub-grants UNALLOCATED 6,400,000.00 0.00 (contingencies) Designated Account 0.00 0.00 Total: 75,500,000.00 75,500,000.00 75,500,000.00 4. Activities and their relationship to the PDO. The Project activities were selected and designed to improve the three sub-objectives of planning, monitoring, and education service delivery in deprived districts of Ghana. See Figure 1 below for examples of key GPEG activities and how they fit with the sub-objectives. 38 Figure 1: Activities and their Relationship to the PDO Component 1: Sub-Grants to Deprived Districts to support key education objectives 5. The goal of this component was in line with the PDO’s sub-objectives of- planning, monitoring, and delivery of basic education services in deprived districts. At Appraisal, this component was to provide annual non-salary resources to 57 deprived districts as a supplement to existing resource flows, support districts’ annual programs of work and government strategic priorities under the ESP and AESOP. This included district grants to fund activities related to the four key policy areas of (i) equitable access and participation, (ii) gender, (iii) teaching and learning quality, and (iv) management of education service delivery, as well as the upgrading of 5,000-8,000 untrained teachers in those districts. To complement this component and harmonize with GPEG, DfID provided parallel grant funding of US$10 million for girls’ education packages (scholarships for JHS students, girls network/mentoring, capacity building for district girl education officers) in the same GPEG districts. Under the second restructuring, additional funding was allocated to the district sub-grants from unallocated funds (contingencies) as well as from exchange rate savings and a reduced number of teacher trainees in the UTDBE program. The additional funding increased district grant support and in particular, start up expenditures for the newly created districts. Tables 1 and 2 highlight achievements of results and outputs under 39 Component 1.1 and 1.2, respectively, organized by the sub-objectives of improved planning, monitoring, and education service delivery. Table 1: Achievements of Results and Outputs under Component 1.1 – District Sub- Grants Results and Outputs Comment on Indicators Intermediate PDO-level indicator indicator Planning 75 districts drafted, finalized, and implemented APWs. 100% of deprived districts disbursed more than 75% X At appraisal, the end target was 80% of their district education grants as planned in their PDO-level indicator exceeded APW DEDs collating SRCs from 100% of schools within X At appraisal, the end target was 70% the district Intermediate indicator exceeded Average total disbursement of district education grants as planned at 98.6% GH125,945,572 spent on APW activities by deprived districts Monitoring 5,993 head teachers trained on school report cards X At appraisal, the end target was 6,930 1,094 Circuit supervisors trained on school report head teachers and Circuit cards Supervisors trained on school report cards Intermediate indicator exceeded 1,568 officers trained on ADEOP and school report X At appraisal, the end target was 500 cards district education officers trained 308 regional and district officers trained in reporting, under the project monitoring and evaluation 329 regional and district officers trained in internal Intermediate indicator exceeded controls and auditing Service Delivery 1,059 activities (44% of total expenditure) conducted related to equitable access 639 activities (20% of total expenditure) conducted related to bridging the gender gap 1,878 activities (17% of total expenditure) conducted related to improving management of education service delivery 1,407 activities (19% of total expenditure) conducted related to improving the quality of teaching and learning. 40 889 district gender officers trained in guidance and counseling 94,827 INSET teacher trainings (cumulative throughout the Project) 28,056 teachers trained in core INSET Math X At appraisal, the end target was 14,485 teachers trained in non-core INSET 30,000 teachers trained under INSET (Kindergarten, Gender) Intermediate indicator met through proxy calculation Table 2: Achievements of Results and Outputs under Component 1.2 – Teacher Development and Skill Upgrading Results and Outputs Comment on Indicators Intermediate PDO-level indicator indicator Planning More cost effective teacher training program implemented Service Delivery 75.7% of UTDBE-trained teachers obtained a X At appraisal, the end target was 50% satisfactory rating or higher in the LOS PDO-level indicator exceeded 6,480 unqualified teachers upgraded their skill sets X At appraisal, the end target was 5,000 through the UTDBE training program UTDBE teachers trained under the project Intermediate indicator exceeded Percentage of trained teachers in deprived districts increased between 10-18 percent in KG, Primary, and JHS between 2012-2013 and 2015-2016. 6. Achievement of Component 1 PDO Indicators:  The project successfully met the target of this PDO indicator – Deprived districts disbursing 75% or more of their district education grants as planned in their APW. At project closing, 100% of deprived districts had disbursed 75% or more against planned activities, exceeding the end target of 80%. In fact, the average total disbursement was 99% against planned activities in the APW.  The project also successfully met the target of this PDO indicator – Teachers trained under the project in deprived districts who obtain satisfactory rating or higher in the SBI/CBI lesson observation sheet for lesson planning, teaching methodology, and classroom organization and management. While midline scores were higher overall than endline scores, endline scores were still improvements from both pre- project scores and the project baseline scores (conducted in year 2) and higher than the target of 50%. 41  While not listed as an indicator in the results framework, the increase in percentage of trained teachers in the deprived districts is a natural PDO-level indicator under this component. The school and district grant IE provides some key figures on this. The percentage of trained teachers in deprived districts increased between 10-18 percent in kindergarten, primary, and junior high-levels between 2012-2013 and 2015-2016. While the percentage of trained teachers also increased in non-GPEG districts, the GPEG districts saw greater gains (3-5 percentage points higher) allowing deprived districts to narrow the gap in this indicator. In addition, the percentage of trained junior high school teachers jumped from 72 to 82 percent in GPEG districts, compared with 87 to 92 percent in non-GPEG districts. 7. Table 3 below shows the most popular activities executed within a year by count. The ten most popular activities account for 51.9% of the total number of activities selected across all districts. Note that many of these were mandatory activities. Table 3: APW Most Popular Activities Executed by Count Code Activity Count Proportion A2 Provide schools with essential supplies - chalkboards, mono desks, dual 337 7.1% desks, teacher’s tables, chairs, cupboards, KG furniture etc. Q9 Provide teaching and learning materials, learning kits etc. 335 7.0% M11 Monitor and support school grants planning and expenditure 306 6.4% A4 Provide gender friendly sanitary facilities in basic schools (Urinals and 301 6.3% toilets) Q1 Organize INSET – Mathematics, Science & Lit 278 5.8% M27 Provide operational expenses 267 5.6% M3 Conduct training for Head teachers on the use of school grants 211 4.4% Q15 Organize School Appraisal Meetings (SPAMs) at the district level 155 3.3% M5 Train district officers in Excel forecasting etc. 141 3.0% M16 Conduct regular school inspection and disseminate reports in a timely 138 2.9% manner TOTAL 2,469 51.9% TOTAL FOR ALL ACTIVITIES 4,758 100% 8. Table 4 below presents the most popular activities executed within a year by expenditure. The top ten most popular activities by expenditure account for 79.32% of the overall budget, with the top three alone accounting for nearly 50% of the overall expenditure in a given year. All of the top three activities were procurement related and hence it is reasonable to expect higher costs for these activities. Other activities that were popular were the mandatory activities including INSETs and trainings for Head teachers. 42 Table 4: APW Most Popular Activities Executed by Expenditure (Ghana Cedis) Code Activity Expenditure Proportion A2 Provide schools with essential supplies - chalkboards, mono 24,236,945 20.10% desks, dual desks, teacher’s tables, chairs, cupboards, KG furniture etc. A4 Provide gender friendly sanitary facilities in basic schools 21,730,847 18.02% (Urinals and toilets) A13 Provide material support to children who are orphaned in order 12,388,831 10.28% for them to complete their education Q1 Organize INSET – Mathematics, Science & Lit 8,181,657 6.79% Q9 Provide teaching and learning materials, learning kits etc. 8,155,606 6.76% M11 Monitor and support school grants planning and expenditure 7,051,981 5.85% A1 Rehabilitate school buildings – minor repairs 4,461,588 3.70% M31 Start Up Costs 3,805,586 3.16% M3 Conduct training for Head teachers on the use of school grants 3,033,697 2.52% Q15 Organize School Appraisal Meetings (SPAMs) at the district 2,586,374 2.15% level TOTAL 95,633,111 79.32% TOTAL FOR ALL ACTIVITIES 120,571,004 100.00% 9. Achievement of intermediate Indicators under Sub-Component 1.1: There were three intermediate indicators linked to the sub-grants to deprived districts which were all met and surpassed: (i) Head teachers and circuit supervisors trained in the use of School Report Cards; (ii) Regional and District Education officers trained under the project; and (iii) DEDs collating School Report Cards from at least 70% of schools within the district. At project closing, 5,993 Head Teachers and 1,094 Circuit Supervisors were trained, exceeding the target of 6,930. A total of 2,457 Regional and District officers were trained under the project, far surpassing the end target of 500. And 100% of DEDs had collated SRCs, exceeding the target of 70%. A fourth intermediate indicator, number of teachers trained in INSET, is linked to Sub-Component 1.1. While this indicator was not properly tracked throughout Project implementation (trainings rather than persons trained were counted), proxy measures of the number of Mathematics INSETs conducted (28,056) allows for some assurance that the indicator was close to being met, if not met and exceeded. 10. Achievement of Intermediate indicators under Sub-Component 1.2: At project closing, 6,480 teachers (of 7,800 initially enrolled) completed the UTDBE training, surpassing the minimum target of 5,000. Component 2: School Sub-Grants 11. The goal of this component was in line with the PDO sub-objectives of improving planning, monitoring, and delivery of basic education services in deprived districts. At Appraisal, this component was expected to improve teaching and learning through supplements to the capitation grants at all basic education schools in deprived districts. The component would improve the management and accountability of resources at the school level and strengthen the SPIP process and community engagement. Component 2 funded training on the school grant process, monitoring visits, communication campaigns, and the 43 school grant itself. Table 5 highlights the achievements of results and outputs under Component 2 within the sub-objectives of improved monitoring and education service delivery. Table 5: Achievements of Results and Outputs under Component 2 – School Sub- Grants Results and Outputs Comment on Indicators Intermediate PDO-level indicator indicator Monitoring 98% of schools in deprived districts with up- X At appraisal, the target was for to-date SRCs 75% of schools to have updated SRCs PDO-level indicator exceeded 93% of schools received and displayed the X At appraisal, the target was for most recent SRC on their notice board 75% of schools to have displayed their most recent SRC on their notice board. Intermediate indicator exceeded Service Delivery Approximately 98% of basic schools in X At appraisal, the target for this deprived districts participating in all INSET indicator was 65% core courses Intermediate indicator exceeded 1,991,030 estimated beneficiaries reached, of X At appraisal, the end target was which 48% are female. 1.6 million direct project beneficiaries, of which 40% are female. PDO-level indicator exceeded 16% of P3 students achieving proficiency in X At appraisal, the end targets English were 15% in P3 English and 13% of P3 students achieving proficiency in 11% in P3 Mathematics Mathematics PDO-level indicator using midline data. 12. Achievement of Component 2 PDO Indicators: The project successfully exceeded the targets of these PDO indicators::  The PDO indicator –Public basic schools in deprived districts with up-to-date SRCs. At project closing, 98% of schools had an up-to-date report card according to School Grant Reports, exceeding the end target of 75%.  The PDO indicator – P3 Students achieving improved proficiency in English and Math in deprived districts was achieved. The 2013 NEA results listed P3 English scores at 16%, exceeding the end target of 15%, and P3 Mathematics scores at 13%, above the end target of 11%. Just prior to project closing, the 2016 NEA had switched to collecting proficiency scores for P4 Math and English, for which there 44 were no project targets set for baseline and closing. EGRA and EGMA for P2 students in GPEG districts between 2013 and 2015 confirmed that there were statistically significant improvements in number identification, letter sounds in English, oral reading in English, and letter sounds for local languages in the GPEG deprived districts.28  The PDO indicator – direct project beneficiaries reached was exceeded with an estimated 1.9 million beneficiaries at project closing, 300,000 more beneficiaries than the project target of 1.6 million. It is estimated that the project beneficiaries comprised of roughly 912,000 females, or 48% of the total, exceeding the project target of 40%.  The intermediate indicator – basic schools in deprived districts participating in all INSET courses supported under the project was achieved with an estimated 98% of the required INSET carried out. While this is a proxy calculation, it still exceeds the project target of 65%.  The intermediate indicator – schools that have received and displayed the most recent SRC on their notice board – exceeded its end target of 75% with an estimated 93% of schools displaying their SRC. Component 3: Project Management and Institutional Strengthening 13. The goal of this component was in line with the PDO’s sub-objectives of improved planning and monitoring in deprived districts. The purpose of this component was to strengthen government systems for the implementation and supervision of decentralized education services in the deprived districts. This component funded consultancy services, training and operational costs, materials, IEs, and external and internal audits. Table 6 displays the achievements of results and outputs under Component 3 within the sub- objectives of improved planning and monitoring. 28 GPEG District and School Grant Impact Evaluation, 2016. 45 Table 6: Achievements of Results and Outputs under Component 3—Project Management and Institutional Strengthening Results and Outputs Comment on Indicators Intermediate PDO-level indicator indicator Planning 100% of schools in deprived districts with X At appraisal, the end target was SPIPs approved by SMCs 75% of schools with SPIPs approved by SMCs Intermediate target exceeded by 25% Monitoring On average schools were visited 7.8 times and no school was visited fewer than 2 times per year. 100% of schools visited by Circuit X At appraisal, the end target was Supervisors at least two times in a year 100% of schools visited by Circuit Supervisors at least two times in a year Intermediate indicator met 100% of Regional Education Directors X At appraisal, the end target was submitting acceptable annual reports, 100% of REDs submitting incorporating data from all districts in the acceptable annual reports region Intermediate indicator met Impact Evaluation on UTDBE teachers conducted Impact Evaluation on district and school grants conducted External and internal audits carried out 14. Achievement of Component 3 Indicators: The project successfully met the targets of all indicators, as outlined below:  At project closing, 100% of Regional Education Directors submitted acceptable annual reports, incorporating data form all districts in the region – meeting the target for this intermediate indicator.  The intermediate indicator – Schools visited by Circuit Supervisor at least two times in a year – was met as 100% of schools were reported as doing more than two times per year.  The intermediate indicator – Schools in deprived districts with completed SPIPs approved by SMCs using the revised template – exceeded the project target of 75% with 100% of schools in compliance by project closing. 46 Additional findings from Fiduciary Review and Procurement Review 15. In depth fiduciary reviews were conducted to assess performance at central and local levels with respect to financial management and procurement. The findings from the various fiduciary reviews confirmed the following strengths: a) Strong technical competences of the financial controller (GES) and the accounts team at head office; b) Qualified and capable financial officers at the district levels; c) Quarterly financial management supervision visits by HQ Accounts team of all the participating districts to review their systems and validate expenditure; d) Regular capacity building on fiduciary related topics to address fiduciary capacity challenges of the districts and also to mitigate issues such as transfers of key staff, lack of full complement of internal audit staff etc; e) Strong management and supervisory oversight by the technical staff of GES and those responsible for their respective component in ensuring that budgets are adhered to and costs are reasonably controlled; f) A good and timely reporting mechanism in place channeling up from districts to regions and HQ. The Project Financial management reviews by the World Bank were conducted in 2013 and 2015. Implementation Manual also included specific templates for schools to report on the grants; and g) Release of funds only when conditions spelled out in the PIM were met. 16. Although FM performance was very steady, the fiduciary reviews also noted some room for improvement. The strict procedures required that district finance and administration had to submit hard copy of all claims made by the districts in order to receive payment (for procurement related activities).This often required travel to Accra which delayed payments and increased transaction costs for district staff. Electronic submission would be a preferred method. The FM reviews also recommended a standard template for internal auditor reporting at the district level, and more training at this level as capacity seemed to prevent the district auditors from supervising school heads more effectively. In some of the new districts, internal auditors had not been hired and this contributed to poor supervision. Filing systems in some districts were weak and similarly at some schools the filing systems were quite poor. 17. The most recent audit was completed in June 2016. In instances in which minor issues were identified, corrective actions were enacted by the Bank and the Government. FM reviews by the World Bank were also conducted in 2013 and 2015. Safeguards 18. Safeguards compliance was not well monitored until the second year when District Education Directors were trained and briefed about their responsibility and role in ensuring safeguard compliance throughout project implementation. Moreover, the project developed 47 a simple checklist that was shared with the Government, included in the PIM and all minor civil works bidding and contract documents going forward. The most recent safeguards review in 2016 found minor impacts from the rehabilitation and minor construction works as well as the adherence to the applicable guidelines provided in the Safeguards Instruments especially the mitigation measures provided in the checklist for construction and rehabilitation activities. 19. The applicable Safeguards instruments were prepared and approved for the project but proved to be a large and unwieldly document. The ESMF from previous projects was used with additional information provided related to the GPEG. The document may have provided adequate guidelines for the rehabilitation of classroom blocks and construction of basic sanitary facilities for the beneficiary schools, but the quality of the document could have been improved. Use of the Environmental Management Plan (EMP) checklist has ensured safeguards compliance and holding contractors accountable for following EMP for minor civil works projects. 48 Annex 3. Economic and Financial Analysis 1. Following the economic and financial analysis presented in the Project Appraisal Document, the Economic and Financial Analysis is similarly divided into two parts: (a) District and School Grants; and (b) teacher upgrading (Untrained Teachers Diploma in Basic Education - UTDBE). For both sections, the economic and financial analysis presents the following sections: (i) efficiency of implementation; and (ii) financial sustainability. 2. The overall findings that are detailed below indicate that GPEG benefitted from exchange rate gains and cost effective implementation of activities, allowing the project to achieve and exceed the project targets. 3. Component level expenditure was in line with the assessments made during appraisal and cost savings from currency depreciation were redirected towards increased allocations for district grants. The table below compares the component level project costs as budgeted in the PAD with the actual expenditure (in US dollars). The Ghanaian Cedi (GHc) depreciated by 115% against the US dollar during the Project Period29 and savings generated from depreciation of the Ghanaian currency were redirected towards Component 1. 30 Also, 67.2% of the funds allocated to Physical and Price Contingencies were directed towards additional support for districts and expansion of the in-service teacher training program. The remaining contingency amount was directed towards Project Management and Institutional Strengthening. Due to exchange rate gains, actual expenditure for school grants was lower than projected even though the base grant and per capita amounts were increased over the project period. Table 1. Project Costs and Financing by Component (Budgeted in PAD and Actual Expenditure) in US$ Project Costs and Financing Budgeted Actual Component 1 District Grants, Teacher Development & Skills Upgrading 42,360,000 48,407,527 Component 2 School Grants 22,060,000 20,382,339 Component 3 Project Management and Institutional Strengthening 4,580,000 6,710,134 Physical and Price Contingencies 6,500,000 Total 75,500,000 75,500,000 Source: Author’s calculations based on Financial Management Reports, Ghana Education Service 29 The ICR uses 3.05 as the exchange rate for the GH¢ to the US dollar. This is average exchange rate over the lifetime of the project weighted by the amounts at each withdrawal. The exchange rate during the drafting of the PAD was 1.90. 30 An additional supplementary grant was provided to districts in 2016. 49 1.1 District and School Grants Efficiency 4. In terms of efficiency of district and school grants the analysis found that district grants were allocated based on relative deprivation. Two district level activities (accounting for 24.81 percent of overall district level expenditure) – construction of urinals and toilets and in-service teacher training - were analyzed and both activities were found to be relatively cost effective in comparison with government and other donor led initiatives in Ghana. At the school level, school grants disbursements were found to be highly correlated with improved planning (through school performance improvement plans) and monitoring (through school report cards). A difference in differences analysis showed that in line with the available literature on district and school grants, districts that received grants witnessed substantial improvements in gross and age-appropriate enrolment (as compared to non GPEG districts over the same period). However, not all the improvements in access can be attributed to the project as other programs were operating in deprived districts during the same period. 5. District grants were allocated based on relative deprivation among the GPEG districts maintaining a strong equity focus. Resources to districts were allocated based on a consistent, transparent and deprivation based allocation formula that provided additional funding to more deprived districts31. The model was updated annually using the latest EMIS data. Thus, funds were equitably allocated using a resource allocation model developed during project preparation. Districts were also provided with summary statistics highlighting their performance on key indicators (GERs, NERs, BECE pass rates etc.). Since then, the model has become a part of the GES planning toolkit and is used to also disburse government funds to non GPEG districts. 6. District grant budgeting and expenditure was tracked. As outlined in the PAD, to receive district grants, each district submitted an Annual Programme of Work (APW) which detailed their planned activities. Districts planned for activities along four cross cutting thematic areas: Access, Quality, Gender, and Management found in a pre-approved Menu of Activities. At the end of each year, districts reported on the progress in implementing their activities within their APW, and were reported on actual expenditure, timeframe and quantity. The analysis of the executed APWs reveals that districts conducted 4,983 activities over the project period. These comprised of 76 unique activities. 31 The allocation formula consists of a base grant component and additional allocation was based on the number of schools, students, teachers and the level of deprivation ( lower retention rates, lower share of girls’ enrolment, lower share of trained teachers and lower pass rates in BECE Mathematics and English). 50 Table 2. District level Expenditure (in GH¢) Cross Cutting Pillar Count of Activities Expenditure % Of Expenditure Access 1,059 65,100,932.86 44.16% Gender 639 30,069,127.40 20.40% Management 1,878 24,392,645.01 16.55% Quality 1,407 27,850,547.05 18.89% Total 4,983 147,413,252.32 100.00% Source: Annual Program of Work Expenditure data. The pillar level allocations include GPEG and DFID’s Girls PASS Program. 7. As seen from the table below five activities accounted for more than 60% of the expenditure for district grants. Among the five, the unit cost of two activities (construction of gender friendly toilets and urinals and provision of INSET) are subsequently examined in greater detail32. Table 3. District level activities with the highest level of expenditure (in GH¢) Activity % Count Expenditure Expenditure Provide schools with essential supplies - desks, tables chairs 337 24,236,945 20.10% Construct gender friendly toilets and urinals 301 21,730,847 18.02% Provide material support to children in need 89 12,388,831 10.28% Organize INSET – Mathematics, Science & Lit 278 8,181,657 6.79% Provide teaching and learning materials, learning kits etc. 335 8,155,606 6.76% 8. Unit cost of construction for toilets and urinals funded under the GPEG district grants was lower than both the government’s own procurement unit and UNICEF. The Ministry of Education mandates that all toilets and urinals constructed at the school level follow a standard design33 allowing for a cost comparison of identical units. Using GPEG funds, districts constructed 871 2- unit urinals and 609 4-unit Kumasi Ventilated Improved Pit-Latrines (KVIPs) at an approximate cost of US$7.12 million dollars. As the table below shows, the average cost of construction for both the toilets and the urinals were lower than the costs of 300 toilets being constructed with UNICEF funding and 41 newly constructed toilets under the government’s Funds and Procurement Management Unit of Ghana (FPMU) in the same period. Thus, the decentralized GPEG financed toilets were more cost efficient than other construction at the school level. However, the standards and the cost of construction for school toilets is higher than UNICEF’s mud walled community based toilets 34 . A relaxation of strict construction 32 These items are selected because information regarding quantities for the highest expenditure item of providing schools with essential supplies (disaggregated by type of furniture) was not available. 33 Kumasi Ventilated Improved Pit-Latrines (KVIPs) 34 Community based toilets have a unit cost of Gh¢8,490 51 norms in the design of toilets at the school level may lead to significant generation of cost savings (cost ratio of 4:1). Table 4. Unit Cost of Construction (in GH¢) GPEG UNICEF FPMU Toilet (4 Unit KVIP) 32,801 36,083 40,100 Urinal (2 Unit) 5,665 6,368 5,950 Source: Author’s calculations based on GPEG Annual Programs of Work and data provided by the MOE and UNICEF 9. Unit cost of providing in-service teacher training under GPEG compares favorably to non-GPEG districts. In-service teacher training in core subjects (Math, Science and Literacy) accounted for 6.79% percent of the district grants budget. As part of the reporting requirements, districts submitted training reports on all the training activities conducted during each data collection cycle. Delivery of modules through GPEG compared favorably with the delivery of identical modules in non GPEG districts. For example, delivering a module in Mathematics under GPEG cost the project GH¢ 39.48 per beneficiary (approximately US$13) as compared to GH¢ 45.8 (approximately US$15) in a non GPEG district. Table 5. Unit Cost of core INSET (in GH¢) GPEG districts Non GPEG districts Mathematics 39.48 45.8 Science 34.41 48.8 Literacy 37.57 42.7 Source: Author’s calculations based on INSET training reports for 2013, 2015 and 2016 and reports from non-GPEG districts from GES. Disaggregated data for INSET isn’t available for 2014 as not all districts submitted reports for that year. School Grants 10. School Grant allocations were based on annual EMIS data and were open and transparent. The joint monitoring visits conducted by MOE/PBME and GES found that funds transferred to bank accounts of each school were displayed on a notice board in the district education office. An impact evaluation of the district and school grant found that only 3% of the sampled Head Teachers reported that they had not received the full grant entitlement. 11. Schools primarily used the grants for facilitating of SMC meetings, small repairs, building of toilets and the overall improvement of the school environment35. A fiduciary assessment conducted in a sample of 105 schools found that projects undertaken by the school were inspected by the district office and the expenditure of school grant funds were tracked at the district level. The execution rate of the funds within each 35 Endline Impact Assessment 52 fiscal year increased as the project progressed from 72.3% in 2013, to 88.0% in 2014, and finally 92.4% in 2015/16. 12. Improvements were seen in planning and monitoring at the school level. On average, across the duration of the project, 86.9% of schools displayed the School Report Cards36 on their noticeboard. This proportion has been increasing steadily across the years: in 2013, the figure was 79.0%, which increased to 87.6% in 2014 and noted a further improvement in 2015/16 at 93.0%. Thus, improvements in planning and monitoring at the school level (as envisaged in the PAD) were evident. Though the data to analyse if the investments made using school grants fund were cost effective is not available. 13. There has been a substantial improvement in gross and age appropriate enrolment in deprived districts over the project period. Though non GPEG (the remaining 141 districts that did not receive GPE funding) and GPEG districts have seen improvements in enrolment over the project period, improvements in districts receiving school and district grants have been appreciably higher than non GPEG districts across all sub levels of basic education. 14. As Table 6 shows, at the Kindergarten and Primary level, GPEG and non GPEG districts had comparable Gross Enrolment Rates (GER), Net Enrolment Rates (NER), Gross Admissions Rate (GAR) and Net Admission Rates (NAR) at baseline. For example, in 2012, GAR in non GPEG districts was 107.9 as compared to 107.7 in GPEG districts. However, by endline, GAR in non GPEG districts had risen only to 115.4 while in non GPEG districts, GAR had risen to 124.4 (a 16.7 percentage point increase and a 9.2 percentage point improvement above non GPEG districts). The differences are particularly pronounced at the KG level wherein improvements in GER and NER over the project period are 24.2 and 16.6 percentage points higher than improvements in non GPEG districts (see Figure 1). 36 The School Report Card (SRC) collects information from schools concerning pupil and teacher attendance and student test performance 53 Table 6. Difference in differences between GPEG and non GPEG districts Non GPEG Districts GPEG districts Change Difference Change in from Baseline Endline from Baseline Endline differences baseline (2012) (2016) baseline (2012) (2016) (6-3) to to endline endline (1) (2) (3) (4) (5) (6) (7) Kindergarten GER (%) 114.8 117.8 3.0 111.6 138.8 27.2 24.2 NER (%) 74.5 74.5 0.0 75.6 92.2 16.6 16.6 Primary GER (%) 107.4 111.2 3.8 98.8 111.4 12.6 8.8 NER (%) 85.2 90.7 5.5 81.1 93.7 12.6 7.1 GAR (%) 107.9 115.4 7.5 107.7 124.4 16.7 9.2 NAR (%) 79.3 80.6 1.3 79.3 84 4.7 3.4 Junior High School GER (%) 86.6 92.0 5.4 68.9 76.5 7.6 2.2 NER (%) 52.1 52.7 0.6 34.8 43.2 8.4 7.8 GAR (%) 93.5 101.6 8.1 67.5 85.4 17.9 9.8 NAR (%) 46.1 47.3 1.2 27.2 40.2 13 11.8 Source: EMIS (2012 – 2016) Figure 1. Kindergarten Gross Enrolment Rates (2012 – 2016) KG GER(%) 140 138.8 135 130 125 120 117.8 114.8 115 114.6 111.6 110 Baseline (2012) Endline (2016) Non GPEG districts GPEG districts GPEG (assumed prior trajectory) Source: EMIS (2012 – 2016) 54 15. However, DFID’s GPASS program (at the JHS level) and the government’s Complementary Basic Education program that supports mainstreaming of out-of-school children also operated in GPEG districts during the project period and so it would be difficult to isolate the improvements in access attributable only to the provision of district and school grants. Fiscal Sustainability 16. The financial cost of continuing the school and district grants in the deprived districts is a recurrent expenditure of US$19.3 million annually. During project implementation, the fiscal impact of the district and school grants was limited to the opportunity cost of staff time involved in planning, implementing and monitoring both district and school grants. The financial cost to the Government of continuing the school and district grants to the deprived districts is US$19.3 million a year (higher than the US$16 million a year estimated in the PAD as the allocation for district and school grants increased over this period). However, with limited sector financing for capitation grants and other education investments because of fiscal constraints, no school grants have been disbursed in 2016 apart from the preexisting capitation grant (of approximately US$0.49 per student per semester). So far, the government has no plans to increase the capitation grants provided to schools or increase district grants, leading to no recurrent fiscal implications for the government. 17. Donor partners plan to provide gap financing to complement existing public recurrent education spending for basic schools. USAID is committed to expanding the district grants while DFID will continue to fund girls’ scholarships to all GPEG districts. UNICEF and USAID also plan to introduce more efficient mobile monitoring tools to help track local level expenditure. 1.2 Untrained Teachers Diploma in Basic Education (UTDBE) Efficiency 18. In line with the PAD, the Project’s four year in-service Untrained Teachers Diploma in Basic Education (UTDBE) program is compared to Ghana’s conventional Diploma in Basic Education (DBE) model. The DBE model uses the same curriculum for training teachers as UTDBE through a three year pre – service residential program and was the prevailing program of choice to upgrade teacher skills prior to GPEG. 19. In terms of efficiency, the following was determined: UTDBE and the DBE model produce teachers with similar skillsets though UTDBE is more cost-effective with a cost ratio of 1.6. The program has led to a convergence in the percentage of trained teachers within GPEG and non GPEG districts though the upgrade of teacher skills has not translated into vastly improved completion or transition rates to date. 55 20. Following participation in training, UTDBE- and DBE-trained teachers have comparable skillsets. In the UTDBE impact evaluation (IE) study comparing the two programs, 400 of the UTDBE trainees and 185 of the DBE graduates were observed in a classroom setting and graded on lesson planning and preparation, classroom methodology and delivery and class management and organization. The UTDBE trainees showed mean scores within the same range as DBE teachers on 14 out of 17 skills in lesson planning/ preparation, teaching methodology and classroom organization and management 37 . However, UTDBE and DBE trainees were also tested on content knowledge. DBE teachers displayed a better understanding of Integrated Science, a result which is statistically significant at the 1% level. UTDBE trainees showed better results on Classroom Management techniques. Overall, the assessment concluded that following their trainings, UTDBE and DBE teachers have a comparable level of skills. 21. UTDBE is more cost-effective than DBE with an efficiency ratio of 1.6. The Project Appraisal Document estimated the cost of the UTDBE program to be US$2,268 per trainee. However, the IE study estimated the final cost of the program to be US$4,278 (for all four years) per trainee even after the depreciation of the Ghanaian currency. After cost sharing with the student, the cost of the training to the program was US$2,130 (102% higher than the estimate of US$1,052 in the PAD). However, the attrition rate of 20.43% was lower than the modelled 40% and similar to the attrition rate of the DBE program. 22. During the time of project conception, the cost of training a DBE teacher used to be significantly higher because it included a stipend for each student. In 2012, Murphy found the cost of training a DBE trainee to be GHc 15,350 (US$5,033)38. However, this stipend has recently been abolished and the cost of a DBE student has reduced to US$3,409 per trainee (a 32% decline)39. With the new government assuming office in January 2017, the stipend may be reintroduced. Table 7. Estimated Government and Project Costs for training UTDBE and DBE trainees UTDBE DBE Cost to government per year (GHc) 1,624 3,466 Cost to government per year (US$) 532 1,136 Duration of program (years) 4 3 Total Cost to government (GHc) 6,496 10,398 Total Cost to government (US$) 2,130 3,409 Source: “Impact Assessment of the Untrained Teacher Diploma in Basic Education (UTDBE) Trainees: Endline Study.” by Associates for Change (2016). 37 “Impact Assessment of the Untrained Teacher Diploma in Basic Education (UTDBE) Trainees: Endline Study.” by Associates for Change (2016). 38 Murphy, P (2012) ‘The Untrained Teacher Diploma in Basic Education in Ghana: Analysis and Recommendations: Second Draft’ 39 “Impact Assessment of the Untrained Teacher Diploma in Basic Education (UTDBE) Trainees: Endline Study.” by Associates for Change (2016). 56 23. At endline, the cost to the government of providing training for one teacher under UTDBE was US$2,130 as compared to US$3,409 under the DBE model (which continues to remain in use in Ghana). Effectively, training a student using the conventional mode of training costs the government 60% more than the cost of training a teacher using the UTDBE modality. This is a cost ratio of 1:1.6 or conversely, for every US$100 spent on the UTDBE program, US$160 would need to be spent on the DBE program (or higher if the allowance is reinstituted). This cost ratio of 1.6 is lower than the one projected in the PAD (the estimate was 4.3 primarily due to the inclusion of the stipend and the underestimation of the cost of the UTDBE program). However, the UTDBE program remains more cost effective while training teachers to achieve the same skills and certificate at similar levels of attrition. 24. Training UTDBE teachers increased the percentage of trained teachers and the PTTR in deprived districts. As the difference in differences table below shows, while the proportion of trained teachers has increased nationwide during the duration of the project, the increase in GPEG districts has outpaced that of non-GPEG districts across all levels of basic education40. The rate of increase in GPEG districts is 17 percentage points higher than the non-GPEG districts for KG and 12 percentage points higher for Primary and JHS from baseline to endline. The subsequent improvement in PTTR is much higher for GPEG districts as compared to non-GPEG districts for all levels. At the KG level, the rate of change at the KG level is particularly noteworthy where the PTTR decreased by 62 more pupils to a trained teacher in GPEG districts compared to non GPEG districts. 25. The convergence in the percentage of trained teachers between GPEG and non GPEG districts is likely to remain persistent. Most UTDBE teachers had been teaching in GPEG districts for several years prior to being selected for the program and have roots within the communities in which they reside.41 Moreover, UTDBE teachers are bonded to stay within the same school in the deprived areas for at least another two years after the successful completion of their training. Thus, the reduction in disparity between GPEG and non GPEG districts in terms of percentage of trained teachers is likely to persist, especially if all UTDBE trained teachers are put on the GES payroll. 26. However, so far, these improvements have not yet translated into substantial improvements in indicators of internal efficiency. Though GPEG districts have shown improvements in completion rates (for example completion rate at JHS rose from 59.9% to 65.1%), improvements in non GPEG districts outpaced GPEG districts in primary completion rates and transition rates from primary to Junior High School (JHS) at endline (both the levels for which data is available). However, given the fact that the UTDBE teachers hadn’t graduated from the program while the data on transition and completion rates was collected, the difference in differences between GPEG and non GPEG districts 40 The absolute number of untrained teachers has risen over the project period in both GPEG and non GPEG districts allaying concerns that the shift in percentage of trained teachers may be partially credited to net migration of untrained teachers from GPEG districts to non GPEG districts. 41 “Impact Assessment of the Untrained Teacher Diploma in Basic Education (UTDBE) Trainees: Endline Study.” by Associates for Change (2016). 57 in terms of repetition, transition and completion rates should be observed over the next few years as the impact of the increased proportion of qualified teachers may take longer to materialize in terms of results in completion. Table 8. Difference in differences between GPEG and non GPEG districts (improvements in percentage of teachers trained, Pupil Trained Teacher Ratio (PTTR), completion rates and transition rates) Non GPEG Districts GPEG districts Difference Change Change in Baseline Endline from Baseline Endline from differences (2012) (2016) baseline (2012) (2016) baseline (6-3) to endline to endline (1) (2) (3) (4) (5) (6) (7) Kindergarten % Trained Teachers 57.00 71 14 34 65 31 17 PTTR 56 39 -16 159 81 -78 -62 Primary % Trained Teachers 77 85 8 50 70 20 12 PTTR 41 37 -4 78 57 -21 -17 Junior High School (JHS) % Trained Teachers 87 92 5 72 89 17 12 PTTR 17 16 -1 27 22 -5 -4 Completion rate (%) 73.2 80.0 6.8 59.9 65.1 5.2 -1.6 P6 to JHS1 transition 97.8 98.7 0.9 83.5 83.2 -0.3 -1.2 (%) Source: EMIS (2015 – 16) and authors' estimates from the final number of successful UTDBE graduates as EMIS (2015 – 16) was conducted prior to the graduation of UTDBE teachers. Fiscal Sustainability 27. The upgrading of skills for UTDBE teachers has a recurrent fiscal implication of US$12.48 million. The PAD estimated the fiscal impact of upgrading up to 8,000 teachers to be around US$20 million per year given current salary levels.' The significant depreciation of the Cedi over the lifetime of the project and a revision in Ghana’s civil service salary structure has led to a substantial decline in the recurrent fiscal implications to an annual recurrent expenditure of US$12.48 million.42 28. However, these higher costs are already foreseen in the cost projections prepared by the Ministry of Education for the annual budget hearings. In meetings 42 Prior to their graduation, UTDBE teachers were drawing a salary of US$158 per month, which rose to US$322 per month (an increase of 103%) as of November 2016. 58 with the Ministry of Finance, the MoE has asked for an increased allocation for the 2017- 2018 financial year based on the additional recurrent cost. 59 Annex 4. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending/Grant Preparation Deborah Newitter Mikesell Senior Operations Officer AFTED Team Lead Qaiser M. Khan Sector Lead Economist AFTSE Sector Lead Economist Eunice Yaa Brimfah Ackwerh Senior Education Specialist AFTED Education Specialist Peter Darvas Senior Education Econ. AFTED Education Economist Kirsten Majgaard Education Economist AFTED Economist Patrick D. Murphy Consultant GED13 Teacher Education Daniela Anna B. D. Junqueira Legal Counsel LEGAM Counsel Robert Wallace DeGraft- Hanson Financial Management Specialist AFTME Financial Management Shwetlena Sabarwal Economist AFTED Economist Adu-Gyamfi Abunyewa Senior Procurement Specialist AFTPE Procurement Anders Jensen M&E Specialist AFTOS Results Framework Bernardo da Cruz Vasconcellos Education Specialist AFTED Monitoring/ Evaluation Luis M. Schwarz Sr. Finance Officer CTRLD Disbursements Rose-Claire Pakabomba Program Assistant AFTED Program Assistant Beatrix Allah-Mensah Social Development Specialist AFTSD Safeguards Robert West Consultant AFTED ESP /Grant application Jessica Dodoo Team Assistant AFCW1 Team Assistant Supervision/ICR Deborah Newitter Mikesell Senior Education Specialist GED13 Team leader Charles John Aryee Ashong Senior Procurement Specialist GGOO1 Procurement Robert Wallance DeGraft-Hanson Sr. Financial Management Specialist GGO31 Financial Management Eunice Yaa Brimfah Ackwerh Senior Education Specialist GED13 Education Specialist Peter Darvas Senior Economist GED 13 Economist Christin McConnell ICR consultant GED13 ICR author Patrick D. Murphy Consultant GED13 Teacher Education Juvenia Lidia Mapilele Cohen Finance Analyst CAFE5 Financial Management Lydia Sam Procurement Assistant AFCW1 Procurement Assistant Franklin Kuma Kwasi Gavu Environmental Specialist GEN01 Safeguards Janet Adebo Program Assistant GED13 Program Assistant Stephen Kwaku Tettevie Team Assistant AFCW1 Team Assistant Kabira Namit Consultant GED13 Economic/Fin Analysis Kathleen Beegle Program Leader AFCW1 Impact Evaluation GPE Country Leads Kouassi Soman Sr. Operations Officer DFPGE GPE Country lead Sven Baeten Sr. Education Specialist DFPGE GPE Country lead Prema Clarke Sr. Education Specialist DFPGE GPE Country lead Josephine Lutta Kiyenje Sr. Education Specialist DFPGE GPE country lead 60 (b) Staff Time and Cost Staff Time and Cost (Bank Budget and TF) Stage of Project Cycle USD (including travel and No. of staff weeks consultant costs) Lending BB Total 10 76,436 Trust Fund Total 37 309,489 Supervision/ICR BB Total 19 131,494 Trust Fund Total 84 503,504 61 Annex 5. Beneficiary Survey Results Summary 1. A beneficiary assessment survey was carried out in June 2016 by the Ghana Education Service (GES) to help identify successes and issues arising from the Ghana Partnership Education Grant (GPEG) project implementation and collect recommendations from stakeholders. The main objectives of the GPEG beneficiary assessment were to: (i) collect recommendations and lessons learned from stakeholders and; (ii) produce recommendations to sustain the gains made by GPEG in the future and address the challenges identified moving forward. 2. Fieldwork was undertaken in June 2016 and a total of 16 districts across all GPEG regions were visited. In total, 377 GPEG beneficiaries were interviewed at central, regional, district and school levels. Two schools were visited in each of the 16 districts.43 3. Overall findings indicated that (i) beneficiaries were highly familiar with GPEG at project initiation; (ii) expectations from the GPEG were broadly met and consistent across respondent groups and regions; (iii) GPEG was found to be a success with 82% of the sample rating the project 8 or above on a scale from 1 to 10 (where 1 is ‘very unfamiliar’ and 10 is ‘very familiar’); (iv) GPEG was deemed very necessary and timely by respondents at all levels- on a scale from 1 to 10, respondents on average ranked 9.8 when asked about the necessity of GPEG; (v) trainings held under GPEG helped in the effective execution of the project; and (vi) GPEG reduced the financial burden on parents of having to pay additional fees for miscellaneous expenditure to schools. 4. Closer study of the qualitative data by each respondent group level revealed several key lessons including the optimism that: (i) decentralization can be achieved under the right circumstances and with the right amount of training; (ii) targeting as a resource allocation tool is effective and the base grant component is an equitable way of getting funds to schools; (iii) leadership, good record keeping and accountability mechanisms are essential for successful outcomes and; (iv) capacity building at several levels simultaneously was a key component to the success of GPEG. 5. In the absence of further GPEG funding, respondents highlighted the need to continue empowering districts to plan and prioritize their needs, ensuring community and stakeholder engagement in school management structures as well as facilitating continuous teacher skills upgrading. A minimum amount of funding would be necessary for re-training officers and keeping institutional structures in place. These activities could be sustained through capitation grants and dedicated leadership in the future. 43 Focus groups were held with JHS3 pupils (and where possible included at least one GPASS beneficiary). In primary/KG schools, the focus group was held with upper primary (P4-P6) pupils. In addition, where possible, at least one UTDBE teacher was interviewed per district. 62 Figure 1: GPEG Success Ranking by Respondent Group GPEG Success 10 9.45 8.71 8.62 9 8.33 8 7.57 7 Success Scale 6 5 4 3 2 1 0 Total average HQ Average REO Average District School Average (143) (7) (15) Average (92) (48) 6. Closer study of the qualitative data by each respondent group level (central, regional, district and school) revealed several key lessons from Project implementation challenges including:  Frequent staff turnover – particularly at district level – hampered the implementation as institutional memory and transferred skills were lost. Moving forward, a new system addressing the frequency of transferals may need to be elaborated.  Lack of coordination between regional offices and districts delayed some activities.  Difficulty in providing effective oversight of the implementation at the district and school levels due to inadequate human capacity in the most deprived districts and schools.  Lack of sound and timely report writing at regional and district level.  Short project implementation period which in some cases seemed to rush implementation.  Key challenges under UTDBE noted by teachers included the late payment of fees thereby delaying some training courses. 7. Key lessons learned emerging from a review of the beneficiary assessment data included:  Decentralization can be achieved. Giving officers training and funds to manage leads to tangible results and shows that decentralization is possible under the right circumstances and with the right amount of training. At school level, where there is adequate and regular funding, teaching and learning will improve and enrolment figures will be enhanced.  Targeting as a resource allocation tool is effective and the base grant component is an equitable way of getting funds to schools. 63  Leadership, good record keeping and accountability mechanisms are essential for successful outcomes. Capacity building in particular led to improved record-keeping and accountability through strengthened monitoring activities.  Capacity training at several levels simultaneously was a key component to the success of GPEG. Involving various stakeholders in decision making increases supervision at school level. Head teachers, SMCs and PTAs should therefore be well sensitized and orientated before any new programs in the future.  Most teachers strongly agreed that participants in the UTDBE program had improved their lesson delivery. Over 85% of teachers estimated that their students’ performance had improved as a result of their UTDBE training. 8. In the absence of further GPEG funding, respondents added the following lessons and activities which could be sustained through capitation grants and dedicated leadership in the future:  Empowering districts to plan and prioritize their needs by applying and transferring skills learned in terms of planning and executing activities at district and school level. A minimum amount of funding would be necessary for re-training officers and keeping institutional structures in place.  Ensuring community and stakeholder engagement in school management. Continuous stakeholder involvement in decision making would improve supervision and monitoring at low or no cost.  Teacher skills upgrading through INSETs could be envisioned with minimal funding. 64 Annex 6. Stakeholder Workshop Report and Results A half-day workshop was held on February 8, 2016 among the key implementers of GPEG to discuss lessons learned, sustainability, and arrangements for the future. The workshop was attended by District Directors of Education, Regional Directors of Education, various implementers at the MOE/GES level, and Development Partners. Some of the key recommendations for future sustainability are highlighted below by indicating the project activities that are to be continued with minimal to no incremental funding requirements as well as those activities that would require additional budget allocations. 1. INSET Funding not required The following processes / procedures do not require funding to be institutionalised:  Promotion and performance appraisal for Heads should be reviewed and new guidelines issued. Part of the appraisal guidelines should involve whether an INSET has been organised and the relevance of the INSET.  More school and cluster based INSETs should be held. 2. Trained teachers, teacher retention, and UTDBE Funding not required  Transfer/posting policy should be reviewed to ensure that, especially in the case of interventions, transfers/postings are maintained for a minimum of three years. This is to counteract the loss of knowledge that results from frequent turnover of staff following training/implementation experience.  The UTDBE programme should be continued to ensure that those teachers who are willing to work in deprived communities can be provided with opportunities to become certified teachers. However, it was agreed that this should no longer be funded by the GOG as there is an excess of unemployed trained teachers who need to apply for vacancies in deprived communities.  An analysis should be conducted on trained teachers in the system (employed and unemployed) and compared to how many teachers are needed to reduce the PTTR level to GES acceptable norms.  New teachers in schools should have mentors from within the school who are selected (and trained in mentoring).  There should be a clear set of guidelines on the conditions required for a school to be established (i.e. teacher accommodation, etc.). This can increase teacher retention. Funding required  It is essential that UTDBE trained teachers who are not on Government payroll are prioritised to be placed on payroll. 65 3. Community involvement and participation Funding not required  It is important to empower SMCs by recognizing local champions, engaging them in school activities, and sensitising them on their responsibilities. This should be the responsibility of the DEO, particularly the Circuit Supervisor and the School Heads and should be incorporated in their guidelines. 4. School supervision Funding not required  In order to ensure that findings from the Circuit Supervisors are addressed, the format for Circuit Supervisors to report should be simplified and a concise template should be designed. These findings can then be communicated to the DDE in a digestible format for further follow-up. Funding required  The mobile School Report Card being funded by UNICEF should be expanded to include additional districts, which will eliminate the need for printed information. This will help ensure more reliable and timely production of SRC.  Supervision of schools is necessary to ensure that they are performing and adhering to standards. Hence, it is essential that DEOs receive money for supervision duties. 5. SPIPs, SPAMs, SRCs Funding not required  To ensure that SPIPs are functional, it is essential to enforce preparation of them regardless of funding. It was found that many schools prepare SPIPs only when they are aware of the amount of money they will be receiving. Instead, the SPIP should be regarded as a plan for the school that should be completed regardless of funding. When the funding becomes available, the school can then draw up an action plan based on their SPIPs to prioritise the activities captured within the allocated budget.  To ensure that SPAMs are functional, it should be enforced that BECE results for JHS and EGRA/EGMA/NEA results for KG/Primary are the basis for holding SPAMs.  SPIP guidelines should be reinforced to ensure that schools are preparing one SPIP per academic year.  It is important to ensure vertical linkages in planning throughout the system. SPIPs should be linked to the ADEOP which should feed into regional plans and subsequently national level plans. 6. Girls PASS Funding not required  There should be sensitisation of the DGEOs in deprived districts on the scholarship opportunities for GPASS girls to attend SHS. Possible scholarships include the 66 Secondary Education Improvement Project (SEIP) scholarship, the Northern scholarship, etc.  A selection committee for scholarships should be retained at the district level so that if other scholarship opportunities arise (outside of GPASS), DEO staff have the requisite skills to select the neediest students.  More research should be conducted on better targeting scholarships including tracking the current GPASS beneficiaries through SHS. Funding required  Continued targeted scholarships are necessary to ensure that those who are the neediest have access to education. 7. Other Funding not required  Posting of staff needs to be reviewed to ensure that HR staff at the DEOs have the requisite background to perform their duties. It has been observed that teachers often become HR staff at the DEOs and may not have the appropriate training.  It is important to recognise hard working teachers and high performing pupils to motivate them to continue performing.  It should be the responsibility of the Circuit Supervisor to encourage the use of local TLMs developed by teachers. This can also feature as part of the teacher appraisal for promotion.  Recognising the lack of funding flowing from the Government to DEOs, it is important that districts are incentivised to generate funds internally (IGF) so that education outcomes are not compromised. Potential use of these funds will help to address inadequacy of school resources, support needy students, etc.  It was found that where there was exchange of information and ideas across DEOs, more innovative approaches and advanced project implementation was observed. It is thus encouraged that there be a network of communication across DEOs apart from the frequent training which brought DEOs together.  To ensure that capacity building activities are coordinated, it is necessary to have a yearly plan for GES on training activities that is coordinated with MOE. It is also necessary for the activities of NGOs working in the education sector be coordinated.  To address the problem of high attrition, information should be placed on DEO noticeboards and regular meetings should be held with the DEO staff to ensure that knowledge is being transferred.  It is important that data is used to make decisions and is shared and communicated appropriately. Funding required  Continuing to ensure that toilets in basic schools are gender friendly and potable water is available is essential for safeguarding student health and continuing to improve access. 67  Districts should administer standardised exams for their district. The policy of creating item banks at the district level should be enforced to ensure the sustainability of initiatives like EGRA/EGMA/NEA.  The capitation grant amount should be reviewed regularly to ensure the maintenance of the economic value of the grant.  The structure of school grants should be changed to include a base and capitation grant so that even schools with low enrolment have some resources available.  Consistent ICT training for DEO staff in Microsoft Office is essential to ensure that reporting is of a high quality. This training should also include training on the SRCs.  It is important for schools to receive additional capacity building for financial management and proper filing. It was found that record keeping at the school level was not always adequate; hence, providing them with training on cash books and recording information is necessary.  It is essential that Heads are provided with orientation prior to assignment and leadership training to ensure a conducive learning environment and help bring about improvements in education results. 68 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR Aim of the Report and Project Background 1. The Ghana Partnership for Education Grant (GPEG) closed on August 31st, 2016. The following Project Completion Report (PCR) assesses the GPEG objectives, design, implementation and operational experience from the perspective of the Government of Ghana (GoG). It provides an evaluation of the program throughout its implementation and identifies lessons. Background at Project Initiation 2. Country context: Ghana as a strong performer in sub-Saharan Africa reached the status of lower middle income economy in 2011 after years of sustained economic growth with ambitions for further rapid growth and development in the coming years. However, high public expenditure arrears remained a challenge even with new oil revenues. Despite the overall reduction in poverty, wide regional disparities in poverty and human development indicators remained, mainly between the northern Savannah regions and the rest of the country. Since 1988, Ghana had put in place comprehensive local government and decentralization reform. Districts reported to regional offices which played a coordinating role. District Education Directorates (DED) which would eventually become part of the Metropolitan / Municipal / District Assemblies (MMDAs), oversee the operation of the basic schools. 3. Education Sector context: Public spending on education was around 6 percent of Ghana’s GDP indicating a high budget priority for education. Districts mainly concentrated in the north had weaker education outcomes than other parts of the country. Children from the three Northern Regions, particularly girls, had lower rates of school participation and weaker learning outcomes. With unsatisfactory learning outcomes in basic schools, improving learning was definitely a Government priority. The percentage of trained teachers had declined in public basic schools from 65 percent to 48 percent between 2001 and 2009. As a result, teacher upgrading had to be accelerated to meet Ghana’s goal of 95 percent. 4. Government introduced the Untrained Teachers Diploma in Basic Education (UTDBE) program in 2004, to upgrade eligible untrained teachers in schools in rural communities and there was the need to increase the number of teachers participating in this program. There was also the need to strengthen School-based management approaches to promote transparency and accountability for education service delivery. These included School Management Committees (SMCs), Parent Teacher Associations (PTAs) and local traditional authorities. School Report Cards (SRC) and School Performance Appraisal Meetings (SPAM). 5. Project Rationale: The GPEG project was designed to improve the planning, implementation and monitoring capacity of ongoing government efforts to decentralize education resources to the district and school levels. The project sought to catalyze 69 funding for education service delivery in the deprived areas in Ghana, which were lagging behind the rest of the country in terms of educational outcomes. Project Development Objectives and Results Indicators 6. The PDO of the GPEG project was to improve the planning, monitoring and delivery of basic education services in deprived districts in Ghana. This was aligned with the Education Strategic Plan 2010-2020. Progress towards the achievement of GPEG’s development objective was measured by the following results indicators: Improved Planning - Deprived districts disbursing 75% or more of their districts’ grants as planned in their APW; Improved Monitoring - Public basic schools in deprived districts with up-to-date School Report Cards; Improved service delivery - P3 students achieving proficiency in English and Math (National Education Assessment results for deprived districts), Teachers trained under the project in deprived districts who obtain a satisfactory rating or higher in the SBI/CBI Lesson Observation Sheet for (a) lesson planning (b) teaching methodology (c) classroom organization and management; and direct project beneficiaries (disaggregated by female beneficiaries). 7. Main Beneficiaries: The project was expected to benefit 75 deprived districts, approximately 6,600 schools and 1.7 million pupils (47% female), over 5000 untrained and 40,000 teachers. District, Regional and Headquarters Education Directorates, National Inspectorate Board (NIB), officers in the MOE as well as the Colleges of Education responsible for the UTDBE program. Ultimately, the beneficiaries were the pupils in public basic schools (over 7,000) in deprived districts who benefitted from an improved and equitable learning environment. 8. GPEG was prepared in very close partnership between the Government and the Local Education Group (LEG) and was aligned with the Education Strategic Plan 2010-2020 (ESP) which had a strong emphasis on the expansion of educational opportunities to disadvantaged populations as well as the enhancement of the quality of education and students’ learning outcomes. Out of the 170 districts in Ghana, 57 (approximately a third) were selected from 8 regions with a selection criterion which included district- level poverty index (weighted at one-third) and some aggregated education indicators (weighted at two-thirds). In the course of the project, the GPEG beneficiary districts increased to 75 from 57 in June 2012 after re-delineation of districts to 216. 9. In collaboration with the World Bank, the MOE and GES developed a Project Implementation Manual (PIM), an Environmental and Social Management Framework (ESMF) and a social assessment. Project Components 10. Component 1: Sub-Grants to Deprived Districts to support key education objectives: This was annual non-salary resources to deprived districts provided as a supplement to existing resource flows, to support districts APW and government strategic priorities identified under the ESP. 70 11. Sub component 1.1: District Sub-Grants: This was provided annually to the districts to implement their APW. Activities in the APWs were focused on local priority needs but aligned with government strategic priorities of improving access, equity, quality and management of basic education. Activities financed under the grant included: in- service training, capacity building, monitoring and evaluation and continuous sensitization to maintain social accountability. DFID provided additional district grant funding of GBP 7 million for girls’ education scholarships packages to the deprived districts under their Girls Participatory Approach to Student Success (GPASS) program. 12. Sub-component 1.2: Teacher Development and Skill Upgrading: The objectives of this sub-component was to support the upgrading through UTDBE of approximately 5,000 and no more than 8,000 untrained teachers in deprived districts. . 13. Component 2: School Sub-Grants: Under GPEG, school grants were provided annually to all public basic schools in the 75 districts to finance activities identified in School Performance Improvement Plans (SPIPs) prepared jointly by parents, teachers, and head teachers. A base grant formula in addition to the per capita grant formula was used to ensure that smaller schools were able to leverage financial resources. 14. Component 3: Project Management and Institutional Strengthening : This component provided the necessary resources for management, monitoring and evaluation of GPEG activities at the headquarters, regional, district and school levels. The objective was to strengthen Government systems for the implementation and supervision of decentralized education services in the deprived districts. Major activities included consultancy and capacity building among others. Girls Participatory Approach to Student Success (GPASS) 15. The Girls Participatory Approach to Student Success (GPASS) Scholarship program is a GBP 7 million project funded by the Department for International Development (DFID). GPASS is not a component of GPEG, but is aligned with the GPEG APWs and uses the same structures and also targets the GPEG target districts. With an overall objective of increasing the enrolment, retention, completion and performance of girls at Junior High School (JHS) level, the GPASS provides material support in a form of scholarships to girls and support to enhance the capacity of the Girls Education Unit (GEU) Head Office as well as District and Regional education officers. Agencies and Responsibilities 16. GPEG was overseen by the Ministry of Education (MOE) and actual implementation was by GES at all its levels of management with responsibilities for: - Planning Budget Monitoring and Evaluation (PBME) Division of the MOE (Project monitoring), - Project Coordinator (Overall coordination), - Director of Basic Education (reporting on Project Implementation and Operational) - Financial Controller (financial management reporting), The Director of the Teacher Education (TED) (management and reporting on UTDBE and INSET). The project avoided establishment of parallel structures and integrated all operations with existing structures based on country systems. The World 71 Bank was the supervising entity while the LEG provided external oversight on behalf of the GPE and Project Steering Committee (PSC) provided oversight on implementation. 17. Project restructuring: Two project restructurings were undertaken. The first introduced works as a category to allow for minor rehabilitation under the District Grant component. The second restructuring extended the closing date from October 31, 2015 to August 31, 2016 and reallocated funds as shown below. Table 1: Re-allocated Project proceeds Categories Budget (USD) Re-Allocation (USD) CATEGORY 1 (HQ + 19,700,000.00 14,936,759.52 UTDBE) CATEGORY 2 (District 27,300,000.00 38,941.702.00 Grant) CATEGORY 3 (School 22,100,000.00 21,621,538.48 Grant) UNALLOCATED 6,400,000.00 TOTAL 75,500,000.00 75,500,000.00 Project Implementation Performance Table 2: Overview of Project Financing as of December 30, 2016 Details Project Budget Withdrawals Expenditure Budget Funding USD $ USD $ USD $ USD $ Execution 75,500,000.00 2013 22,685,456.66 22,684,769.62 18,232,577.00 80% 2014 23,815,833.94 23,913,975.38 25,261,151.00 106% 2015 24,100,000.00 23,887,951.82 23,455,773.60 97% 2016 4,898,709.40 5,013,303.18 8,550,498.40 175% Total 75,500,000.00 75,500,000.00 75,500,000.00 75,500,000.00 100% % of Project Funding 100% 100% 100% Detailed Disbursement Table 3: Project Disbursement Status, 2013-2016 School Grant District Grant UTDBE HQ Total Year USD$ USD$ USD$ USD$ USD$ 2013 6,686,797.71 8,000,037.78 2,113,469.62 1,432,271.88 18,232,577.00 2014 6,200,521.85 14,475,526.99 2,521,510.51 2,063,591.65 25,261,151.00 2015 8,734,218.44 10,861,579.47 2,021,485.09 1,838,490.61 23,455,773.60 2016 - 5,604,557.76 1,728,354.92 1,217,585.72 8,550,498.40 TOTAL 21,621,538.00 38,941,702.00 8,384,820.14 6,551,939.86 75,500,000.00 72 Audits 18. Annual audits were conducted to express independent opinion on the financial statements prepared by GES. The exercise reported on the extent of compliance with various financial requirements and project guidelines. The Ghana Audit Service was appointed the independent external auditor for the project. Three annual audits were conducted for the fiscal years 2013, 2014 and 2015. The 2016 audit will be conducted by June 2017 with funds escrowed under the project. 19. As part of the World Bank’s Financial Management Supervision, two comprehensive Financial Management (FM) reviews were conducted by the World Bank supervision team in 2013 and 2015 and rated the GPEG’s FM’s systems highly satisfactory and satisfactory respectively. Independent Fiduciary Review: During the course of the project, two independent fiduciary reviews were undertaken in 2014 and 2016. 20. 2014 Fiduciary Review: Observations and Conclusions. Adequate capacity existed at the GES headquarters. At all levels of implementation; record keeping, documentation and reconciliations were up to date, there was clear evidence of regular inspection and pre-auditing. Proper authorization and controls were exercised. However, there were lapses in some schools that were brought to the attention of Management. To strengthen the fiduciary environment, the review suggested more training in basic bookkeeping for School Heads and there should be clearly defined basis for distributing items procured. With regard to procurement, there were strong controls in place and implementation at central level followed all applicable guidelines. 21. 2016 Fiduciary Review. The use of country systems for project management as well as the development of project implementation manuals to guide project execution was a key strength of the GPEG/GPASS project. Weaknesses included variations in documentation of monitoring at the district-level, delayed processing of claims since district supporting documents often required in-person delivery to Accra. Procurement 22. Most procurement activities were conducted at district level and were in accordance with Ghana’s country systems as well as the World Bank procurement guidelines. Procurement trainings were conducted for participants from all districts. There was periodic monitoring of district procurement activities. All planned procurements under goods, works (district level) and consultancy services earmarked under the grant facility were executed to completion. Again the Bank provided the required supervision and offered constructive suggestions during implementation support missions. Procurement under GPEG was rated as satisfactory by the World Bank. The project undertook an independent procurement audit with the objective to review the procurement, contracting, and implementation processes which have been followed for contracts in the project to confirm consistency with the Legal Agreement. 23. The main findings of the In-depth Procurement Review are as follows:  Compliance – most districts had approved procurement plans for all the years. While all the entities reviewed and adhered to advertising requirements for 73 applicable contracts. Districts conformed to standard bidding document for goods, works and services. 65 percent of districts visited showed evidence of complying will all evaluation procedures and award of contract.  Contract Administration and Performance – 78 percent of District’s goods in the sample were supplied according to specifications. Similarly, physical works constructed were executed in accordance with bills of quantities, as evidenced from the goods delivery notes and certificates of payments reviewed under the contracts.  Record-keeping - Filing system for most of the Districts visited was unsatisfactory. Separate procurement files were not kept for each contract reviewed. 52 percent of procurement files were incomplete and lacked proper and sequential arrangements depicting the procurement cycle. 24. The Procurement Review noted the following recommendations:  Publications - Contract awards should be published on PPA Website in addition to placing copies on the Notice Board of the Districts.  Awards - Communicating to unsuccessful bidders should be made by the Districts who are not fulfilling this requirement under the Public Procurement Act 2003.  Implementation - Copies of goods delivery note and certificate of completion of work should be kept in the procurement file as an evidence of supply of goods or contractor completing the contract in accordance with the specifications in the contract agreement.  Compliance - Appropriate procurement procedures as stipulated in the World Bank’s guidelines and the PPA rules and regulations should be strictly adhered to by all the Districts. The Districts should also comply fully with the terms and provisions in the contract agreement signed with contractors providing various works for the Districts.  Record-keeping - There should be systematic filing of all documents under each contract in separate files to give a clear audit trail of procurement processes and documentation. All documents at every stage of the procurement cycle should be kept in the procurement file.  Capacity Assessment - All the Districts should train and encourage the procurement officers to be responsible for undertaking and coordinating all detailed procurement activities within the Districts. APW Analysis 25. To receive district grants, each district submitted an Annual Program of Work (APW) which detailed their planned activities along four thematic areas: Access, Quality, Gender, and Management. Districts reported on the progress on implementing their activities, actual expenditure and timeframe twice a year. District grants were allocated based on a resource allocation model which is updated every year and takes into consideration a number of education indicators weighted with disadvantage criteria to ensure equity in the allocation of budget ceilings to districts. 74 26. Average total expenditure for the districts across all years is 95.7%. This increased through the years from 89 percent in 2013 to 94 percent in 2014 to 99 percent in 2015- 2016.Regional expenditure covered mostly capacity building, monitoring, and operational activities. Average total expenditure for the regions was 89.4 percent across all years with 2015-2016 recording over 100 percent due to rolled over activities from 2014. Table 4 details the count of activities along the thematic areas that were executed under project. Table 4: Count of Activities Executed in APW by Thematic Area 2013 2014 2015/16 TOTAL Count Count Count Count Proportion Access 181 283 596 1,059 22.3% Quality 317 448 639 1,407 29.5% Gender 146 126 142 414 8.7% Management 340 651 889 1,878 39.5% TOTAL 984 1,508 2,266 4,758 100.0% 27. In terms actual expenditure, the largest amount was spent on access, at 54 percent with the least amount on gender activities at 2.7 percent. (Excluding expenditure on UTDBE and GPASS). 23.1 percent and 20.2percent of the district budget were spent on quality and management thematic areas respectively. Most of the access activities were gender oriented especially the provision of gender friendly sanitation facilities. Untrained Teacher Diploma in Basic Education 28. The UTDBE activities commenced in August 2012 and ended in August 2016 with 6,480 trainees (from the original 7,027) who are bonded to teach in the districts for at least 3 years. GPEG supported Capacity Building for the selected institutions delivering modules, distance learning courses and in-school support and mentoring. A consultant was engaged to support writers from the COEs and the Universities to write field support modules and a handbook for Tutor Counselors, Mentors and Coordinators. In-Service Training 29. GPEG supported the institutionalization of In-service training of teachers for continuous professional development and capacity building for school management committees. A core activity for all districts for funding under the AWP was INSET and over 96,000 trainings over the project period. Some of the trainings included, INSETs in Literacy, Mathematics and Science, Braille,and ICT, as well as trainings for head teachers and SMC members. The project also funded capacity building for the key officers at the districts managing the conduct and monitoring of INSET and also funded TA for the development of INSET manuals and teacher mentorship guides. 75 30. Girls Participatory Approach to Student Success (GPASS) included the following activities:  Scholarship Package: A total of 60,000 girls have received the scholarship package which included: exam fees, school uniform, and school supplies and stationery; in some cases this also included a bicycle and solar lamp.  Training, Capacity Building and Review: A series of trainings were held for District Education Officials and School Based Facilitators. Districts and GEU headquarters were provided with ICT packages along with ICT training. Furthermore, a budget was provided for administration, communication and maintenance at all levels. Annual reviews discussed implementation and provided feedback which helped to facilitate smoother implementation.  Community Sensitization: This was done to help adults better appreciate the importance of girl’s education and support the program. In some cases, the outreach activities encouraged out of school girls to return to school.  Monitoring: This occurred at all levels and focused on the distribution of scholarship packages, performance and attendance. Reports were submitted to the GEU head office, shared with GES management and DFID, and were also incorporated in GPEG annual reports. School Grants 31. The school grant comprised a base and a per-capita grant formula paid in two tranches. See Table 5 for information on school grant allocations throughout the project period. A total of 7,465 public basic schools were benefiting from the grant as of the end of the project covering a total of 1,930,878 students. Over the project period, an average of 85 percent execution rate was recorded in school grant utilization, improving from 72 percent in 2013 to 92 percent by project closing. Table 5: School grant allocation (per school) 2013 2014 GHC 2015 GHC Type of Grant GHC Base Grant KG 650 800 1150 Prim 1000 1200 1650 JHS 800 1000 1350 Per-Capita 3.62 4.00 4.50 32. In order to receive in School Grants, schools performed the following:  SPIPs: School Performance Improvement Plan (SPIP) was the basis of spending for the school grants. It was developed by the schools in collaboration with the SMC informed by the SRC.  SPAMs: The School Performance Appraisal Meeting was used as an accountability mechanism to relay school performance back to stakeholders and feeds into SPIP preparation. 76  School Report Cards (SRC): The SRC collected information from schools concerning pupils, teachers and management at the school level. The school communicated the analysis through the SPAM, and used it in the preparation of the SPIP. GPEG made funds available to collect the information for SRCs, generate reports, and feed them back to schools. GPEG monitored that SPAMs were being held regularly to discuss the data with the community. The percentage of school with up-to-date SRCs increased from 95 percent in 2013 to 98 percent in 2015-2016. The percentage of schools with SRCs displayed on their notice board rose from 79 percent in 2013 to 93 percent in 2016. 33. Circuit Supervisor Visits: The average number of school visits by circuit supervisors was 7.1 per school/year across all districts over the project implementation period. There has been steady improvement across the years. 2013 recorded an average of 5.9 circuit supervisor visits per school, 2014 at 7.4 and 2015/16 at 7.8. Monitoring and Data Collection for Reporting 34. Monitoring by PBME: Seven monitoring trips were undertaken during the project period. The visits confirmed: the status of implementation and spending performance on APWs and SPIPs; the approval process and challenges with SPIP; the collaboration between Districts and Colleges of Education; and status of a sampled on-going projects. Some of the main findings at the district level included: reporting had improved over time; SRCs were not always distributed on time to the schools; Districts were not always able to obtain regular reports from COEs. At the school level: Improvements in school management; supply of TLMs; Non-availability of reputable vendors in most localities; Lack of capacity in book-keeping; and problems in understanding and interpreting SRCs by school staff. 35. Monitoring Activities by Districts: In an effort to provide standardized feedback from DEOs, a template for a Performance and Monitoring Report was introduced. The reports indicated that district officials ensured judicious use of school grants and contractors on site used quality materials and completed their work on schedule. Supervision improved in all GPEG districts. Data Collection: Headquarters met district officers twice a year, to collect data on APW, financial, procurement, SGRs, and training reports using predetermined templates. As the years went on, reporting as well as cash book entries at the district level improved substantially. Project Management 36. The project management, monitoring and evaluation was carried out through collaboration between the GPEG technical implementation team made up of GES Finance, Basic Education, Supply and Logistics, Teacher Education, Girls Education Unit, and the PBME Division of the MoE. GES management was responsible for the overall project administration supported by a project coordinator. The central Ministry executed Component 3 of the project and prepared an annual work plan covering four major areas: (i) operations - to cater for the day-to-day overhead cost for running the project; (ii) monitoring- checking and following up on project execution to provide feedback into implementation; (iii) training- providing capacity building to support 77 project implementation; and (iv) consultancy- technical assistance to support capacity for key areas and also independent external reviews and impact evaluation. 37. Steering Committee Meetings: A Project Steering Committee chaired by the Chief Director of the MoE met five times to review annual plans and implementation status. The PSC also had LEG and civil society representation. 38. Mid-Term Review: A project Mid-Term Review (MTR) involving all stakeholders was conducted from February 23-March 6, 2015. The MTR report and the MTR Mission concluded that: Project implementation was progressing; disbursements were in line with projections; project indicator targets were being met; GPEG had strengthened the participation of stakeholders in planning and management; enrolment had increased while intensified monitoring and evaluation had improved both teacher and pupil attendance; teaching and learning materials were provided to basic schools and INSET in key areas were available to teachers. Areas for improvement included the timely allocation and implementation of school and district grants, strengthening SMCs, providing more timely collated reporting on school and district activities and improving support for UTDBE trainees. Achievement of Project Development Objectives and Impact 39. The PDO was to “improve the planning, monitoring and delivery of basic education services in deprived districts” in Ghana. The Results Framework tracked progress made against indicators on an annual basis. The PDO was achieved as all indicators were met. 40. UTDBE Quality Assurance: An independent training and research institution, Center for Continuing Education, University of Education, at Winneba was contracted to provide quality assurance on the delivery of the UTDBE program. This consultancy provided feedback and recommendations for improvement to the Colleges of Education delivering the UTDBE program and the Teacher Education Division over the program implementation period. The main reasons for dropout (majority for which occurred in the first year) were, death, withdrawal due to underachievement and leaving the teaching profession for other professions. Many of the coordinators did not keep up to date records of the trainees. Heads of the schools offered both academic and professional support to the trainees. These included in-service training, lesson notes preparation, lesson observations and demonstration lessons. However, there remained inadequate resources to engage adequate facilitators for school-based INSET. The District Coordinators were not able to support student teachers with the organization of cluster meetings and tutorials due to inadequate allowance paid to teachers for cluster meeting, poor attendance of tutors, ineffective communication channels and late releases of results from the examining body. 41. UTDBE Impact Assessment and Lesson Observation Survey: An impact evaluation was carried out to assess the extent to which the distance training program had achieved its objective of upgrading the quality of teaching skills among untrained teachers in the deprived districts and the extent to which this had impacted on learning outcomes. A comparison at baseline and endline was also made in terms of cost efficiency. The 78 assessment also explored whether training while teaching results in teacher retention in deprived districts. 42. Field work: The impact evaluation was carried out over a two-year period between 2014 and 2016. The baseline, midline and endline reports present findings on the content knowledge and skills gained by UTDBE trainees compared to the trained DBE teachers across the core areas of lesson planning/preparation, teaching methodology, classroom organization and management. The endline assessment also explored the retention status of teachers in rural deprived areas. The study used both qualitative and quantitative approaches and included single face-to-face interviews and questionnaires. However, the main data collection instrument was a well-tested instrument - the Classroom Lesson Observation Schedule (LOS). 43. The UTDBE IE had the following key findings:  Performance of UTDBE Trainees: It was affirmed that the trainees have had their knowledge and experience deepened as a result of the UTDBE program. The baseline and endline comparison reveals that the UTDBE trainees had improved remarkably in performance over that of the baseline. However, when the endline is compared to the midline there was some decline in performance.  Comparative Analysis of UTDBE and DBE Teachers: The comparison of examination results showed that, while the DBE trainees performed better in the core subject area of Integrated Science, the UTDBE trainees performed better in the professional field course of Trends in Education and School Management. The endline findings showed that more DBEs were ranked satisfactorily compared to the UTDBEs in relation to the use of generic skills; however, the UTDBE trainees scored better in relation to the use of teaching and learning materials.  Cost Efficiency and Effectiveness: The endline study revealed that the UTDBE program is more cost effective relative to the DBE model of training even without allowance for conventional trainees. Thus, training a teacher on the conventional mode (DBE) cost the government 60 percent more than the cost of training a teacher using the UTDBE modality  Retention in Deprived Districts: The endline findings suggested that a higher proportion of UTDBE trainees compared to DBE teachers are willing to stay in rural deprived areas due to community affiliation. 44. As a result of the UTDBE IE, the following key recommendations were made on the UTDBE program:  Efforts should be made to reduce the high incidence of drop out;  UTDBE program course content should be reviewed to ensure that basic literacy and numeracy content meet current needs of the basic school teachers and be made more relevant to the levels of education where they are expected to teach;  More involvement of Colleges in district cluster meetings;  Circuit Supervisors and Head teachers should be empowered to intensify their mentoring, coaching, monitoring and supervision of UTDBE trainees; 79  Government should adopt the UTDBE training mode to increase teacher retention and assume a less expensive approach;  CoEs need to prepare well for the face to face sessions with UTDBE trainees;  The trainees should be on the Government payroll once they graduate. Project Impact Evaluation: 45. Edburgh Consultants, based in the Netherlands, were contracted for the overall impact evaluation of the GPEG intervention. Their work can be divided into two main tasks: Task 1 covers impact evaluation of higher order outcomes whiles Task 2 covers Survey at the school and district levels in all 75 GPEG districts. This section gives an overview of the results. 46. Impact evaluation of higher order outcomes: This involves a desk review of higher order outcomes specifically related to BECE and EGRA/EGMA examination results, Primary to JHS transition rates, dropout rates, Net Admission Rate, Net Enrolment Rate, Gender Parity Index, etc. It relies on the collection of data from secondary sources such as EMIS, BECE examination results, EGRA/EGMA results (latest year: 2014/15), and SRCs. Outcomes in GPEG districts were compared to a subset of 75 non-GPEG districts that were closest to the cut-off ranking for selection as a GPEG district. The results were combined with the results of Task 2 to prepare the final evaluation report. 47. The main findings from task 1included:  Access: The GPEG had a positive impact on access and participation in GPEG districts in comparison to comparable non-GPEG districts at all levels in relation to NAR,NER, proportion of disabled children enrolled and Primary to JHS transition. However, GPEG districts did not record an improvement in overall dropout rates at any of the levels compared to comparable non-GPEG districts.  Gender: Dropout rates of girls and the dropout GPI at all levels as well as GPI at the JHS level improved substantially more for GPEG districts compared to comparable non-GPEG districts.  Quality: While it is too early to measure the full impact of GPEG on BECE, it is worth stating that it has encouraged a higher number of children to take the exam, and more are passing now than before. However, relative to comparable non-GPEG districts, BECE pass rates in GPEG districts have declined while the gender gap in BECE pass rates have widened. Poorer performance in BECE results in GPEG districts could be attributed to negative effects of increased access which could reverse in future as a result of the benefit from improvements in teaching and education management that have been brought about by the GPEG. GPEG districts registered statistically significant improvements in some areas of EGRA/EGMA between 2013 and 2015, with no decline in performance noted. 80 48. The survey at the school and district level in all 75 GPEG districts looked deeper into the findings in Task 1 to understand the impact of GPEG in the deprived districts. The survey took place in all 75 districts and 353 schools representatively sampled, with the support of questionnaires responded to by District Education Officers, Circuit Supervisors, Head Teachers and School Management Committees. The survey found that in general, there has been improved management at the district level with DDEs and Circuit Supervisors frequently using management tools such as the SRCs, SPIPs, etc. Respondents at the district level were satisfied with the training provided during the project; however, the frequent retirement and transfers of DDEs remains a concern. SMCs and Head teachers require more training in the production and use of SRCs. School level management has also improved as head teachers seem to be satisfied with the circuit supervisors visits in terms of frequency and feedback provided, and the involvement of parents and communities in school matters has increased. Additionally, teacher motivation has increased and teachers spent more time teaching, and the study found that INSET trainings were appreciated by the teacher trainees. Update of district rankings 49. At the end of GPEG, the selection criteria to all of Ghana’s 216 districts was reapplied to see whether the deprived districts have moved up in ranking. The poverty headcount was obtained from an updated ranking that combines the Population and Housing Census (PHC) 2010 and the Ghana Living Standards Survey (GLSS) 6 conducted in 2012. Education indicators were obtained from the 2015/16 EMIS data. The new rankings show that, 57 of the 75 GPEG districts remain in the top 75 most deprived districts while 18 are no longer in them. The most deprived district is Gushiegu in the Northern region. Juaboso in the Western region has made the most progress and now ranks 152 out of the 216 in terms of being deprived. Girls PASS 50. GPASS data from schools is collected through Excel templates by district officers District databases showed the following positive results: a. Overall, the GPASS girls are ranked in the top 43% of their class and are achieving aggregate 36 in BECEs. b. Non-GPASS students are performing slightly better than PASS girls, as expected due to the type of student selected for GPASS. c. On average GPASS girls are attending school 91 percent of the time with 4 percent higher average attendance than non-GPASS students. Monitoring trips also consistently show high attendance rates for the GPASS girls. d. District level data shows that the transition rate to SHS or TVET for GPASS girls is on average 78 percent, compared to a national average of 69 percent (EMIS, 2014/15). 51. The following areas remain challenges for the GPASS program: a. Dropouts due to pregnancy and marriage. The current dropout rate of GPASS full phase is 0.83 percent (394 cases), with 33 percent of those dropouts caused by pregnancy. Marriage led to 51 dropouts. Results also showed that 73 percent 81 of all pregnancy cases on GPASS lead to dropout. Only 14% of teachers in the GPASS schools were female, and 48% of SBFs were female. b. Girl-friendly infrastructure. A recent monitoring visit suggests there has been an improvement in child-friendly environments in the GPASS schools over time with 19 out of 21 schools having separate toilets for boys and girls (90%), and the majority having access to hand washing. However only 6 out of 22 schools had a bin for sanitary pads, and only 6 out of 21 had flushing facilities for the toilets. Results show that 20 out of 23 schools had access to electricity (87%), and 19 out of 22 had access to water (86%). GPEG Lessons Learned and Sustainability 52. The data revealed key lessons and project implementation challenges including: frequent staff turnover, lack of coordination between regional offices and districts, difficulty in providing effective oversight of the implementation at the district and school levels, and lack of sound and timely report writing at regional and district level. 53. Key lessons learned from the Beneficiary Assessment are that decentralization can be achieved, and targeting as a resource allocation tool is effective. Leadership, good record keeping and accountability mechanisms are essential for successful outcomes, capacity training at several levels was a key component to the success of GPEG. Empowering districts to plan and prioritize their needs, ensuring community and stakeholder engagement in school management and teacher skills upgrading through INSETs could be achieved with minimal funding. 54. Key recommendations on lessons learned, sustainability, and future arrangements:  INSET: New guidelines issued to include organization of relevant INSET as a mandatory activity.  Trained teachers, teacher retention, and UTDBE: Transfer/posting policy should be reviewed. The UTDBE program should be continued but should no longer be funded by the GOG. Again an analysis should be conducted to determine how many teachers are needed to reduce the PTTR level to GES acceptable norms and new teachers should have trained mentors. UTDBE trainees who are not on Government payroll should be prioritized in recruitment of teachers.  Community involvement and participation: Local champions should be recognized and be empowered to engage in school activities.  School supervision: A concise template should be designed to simplify the format Circuit Supervisors use to report. The mobile School Report Card should also be expanded to include all districts.  SPIPs, SPAMs, SRCs: SPIP must be prepared regardless of funding to make it functional. BECE results for JHS and EGRA/EGMA/NEA results for KG/Primary should be the basis for holding SPAMs. It is important to ensure vertical linkages in planning throughout the system between SPIPs, ADEOP regional plans and national level plans. 82  Girls PASS: A selection committee for scholarships should be retained at the district level. More research should be conducted on better targeting scholarships including tracking the current GPASS beneficiaries through SHS. Scholarships must be targeted to the neediest.  Staffing and Capacity Building: HR staff at the DEOs must have the requisite background, hardworking teachers and high performing pupils should be motivated, there should be a network of communication across DEOs, capacity building activities must be coordinated, DEO staff should be given consistent ICT training, Heads should be provided with leadership training.  Planning: data should be the bases of decisions and this must be shared and communicated appropriately.  Testing/Assessment: Districts should administer standardized exams for their district. The policy of creating item banks at the district level should be enforced to ensure the sustainability of initiatives like EGRA/EGMA/NEA.  School Financing: the capitation grant amount should be reviewed regularly and must include a base grant. Stakeholders Assessment of Project LEG’s Assessment of GPEG Implementation 55. The GPE grant has played a significant role in expanding quality access and participation especially where and for whom it matters most:  Targeting: The focus on 75 districts primarily in the deprived districts located in the northern Ghana ensured the availability of adequate and predictable funds for those most in need and demonstrated the utility of targeting in making education accessible especially for girls and boys in poor communities.  Better alignment and coordination: The targeted focus has also influenced development partners to invest in the deprived districts; and also align and coordinate their activities. For instance the alignment between DfID supported Girls Pass scholarship program and the GPE grant in the 75 districts has allowed for effective coordination, capacity-building, implementation and monitoring by the Girls Education Unit at both the national and district level.  Systems Strengthening: The use of GPE funds to improve on planning, monitoring and delivery of basic education services in districts also highlights the importance of systemic change and the need to strengthen the entire education systems and not just individual elements; and empowering districts to choose and lead on their own pathways to progress;  Introducing new ideas and innovation: For example the district grants and school grants, the latter as a combination of a base grant as well as a per-capita school grant so as not to short-change smaller and remote schools; & practice of school report cards at scale for increased management accountability, and which UNICEF is now supporting the expansion through mobile School Report Cards. 83 Government Assessment of the Supervising Entity’s Performance 56. The World Bank served as the supervising entity for the GPEG project. The Government greatly benefitted from the World Bank’s support in financial management, procurement and understanding World Bank guidelines. Additionally, the Government commends the bank on granting approvals with minimal delays. The Government found that support missions were extremely useful in ironing out challenges. The Bank’s procedures and systems lent itself to the use of country systems which made project implementation smoother. 57. The Government were particularly appreciative of the World Bank’s support in reviewing documentation by the impact evaluation firm, Edburgh, and assisting with technical support for the evaluation as well as advice. 84 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A 85 Annex 9. List of Supporting Documents Aide Memoires and Implementation Status Reports (April 28, 2013 –August 31, 2016). Basic Education Beyond the MDGs in Ghana, 2014- World Bank, Darvas and Balwanz. Beneficiary Assessment, MOE/GES October 2016. DFID annual reviews of Girls’ PASS program 2014-2016. Documentaries on MOE and GPE websites 2016. Ruhainatu dreams of becoming a nurse. https://www.youtube.com/watch?v=qBA3X7NQDNM GPE helps strengthen Ghana’s education system (short). https://www.youtube.com/watch?v=epkJ9j-uKFo Empowered to Learn – Learn to be Empowered (long video). https://www.youtube.com/watch?v=X39v_sFNETo Education Sector Performance Reports, MOE (ESPRs 2011-2016). EGRA/EGMA reports 2013, 2015. Final Main Procurement Audit Report, February 2017. Benning, Anang and Partners. Ghana Project Completion Report, GES/MOE December 2016. GPEG Project Implementation Manual, January 2013 Ghana Education Service/Ministry of Education. GPEG Government status reports 2013, 2014, 2015, and 2016. GPEG Fiduciary Reviews 2014 and 2016. GPEG Procurement Audit 2016. GPEG MTR reports February 2015. GPEG PAD, October 2012. GPEG application to GPE June 2012. GPE Tracking by the Social Enterprise Development Foundation of West Africa (SEND) 2015. 86 ICR for EdSeP 2013 World Bank. ICR for EFA-FTI 2012 World Bank. “Impact Assessment of the Untrained Teacher Diploma in Basic Education (UTDBE) Program.” Baseline, Midline and Endline Surveys, Prepared by Associates for Change, 2014, 2015 and 2016. “Impact evaluation and monitoring of the District Grant and School Grant component of the Ghana Partnership for Education Grant.” by Edburgh consultants and Innovative Services consortium August 2016. Impact Evaluation of School Based Management Committees, 2010. M-H Cloutier, World Bank. Murphy, Paud (2012) “The Untrained Teacher Diploma in Basic Education in Ghana: Analysis and Recommendations: Second Draft”. National Education Assessment (NEA) reports. NDPC Citizen’s Assessment Survey on Capitation Grant Policy, NDPC and UNICEF June 2015. SABER on Student Assessment- World Bank November 2013 “Transparency and Accountability Processes in the Utilization of Grant Support Under GPEG Project Tracking Survey” Ghana National Education Campaign Coalition (GNECC) July 2014. 87 MAP of Ghana – IBRD 33411 88