Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Angola Learning for All Project (P122700) AFRICA | Angola | Education Global Practice | IBRD/IDA | Investment Project Financing | FY 2014 | Seq No: 9 | ARCHIVED on 28-Dec-2017 | ISR30664 | Implementing Agencies: Ministry of Education, Government of Angola Key Dates Key Project Dates Bank Approval Date:26-Sep-2013 Effectiveness Date:16-Jun-2014 Planned Mid Term Review Date:28-Mar-2016 Actual Mid-Term Review Date:28-Mar-2016 Original Closing Date:30-Apr-2018 Revised Closing Date:28-Feb-2020 Project Development Objectives Project Development Objective (from Project Appraisal Document) The proposed Project Development Objectives are to improve teachers’ skills and knowledge as well as school management in project designated areas; and to develop a system for systematic student assessment. Has the Project Development Objective been changed since Board Approval of the Project Objective? No PHRPDODEL Components Name Component 1: Improving Teachers’ Skills and Knowledge and School Management in P:(Cost $51551820.00 M) Component 2: Establishing a System for Student Assessment:(Cost $7890000.00 M) Component 3. Project Management:(Cost $6558180.00 M) Contingencies Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO  Moderately Satisfactory  Moderately Satisfactory Overall Implementation Progress (IP)  Moderately Satisfactory  Moderately Satisfactory Overall Risk Rating  Substantial  Substantial 12/28/2017 Page 1 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Implementation Status and Key Decisions The Angola Learning for All Project became effective on June 16, 2014. During the first phase of implementation, the project faced a number of challenges and significant delays in implementation. After the Mid-Term Review (MTR) of the project in May 2016, a decision was made to restructure the project to enable the Ministry to successfully achieve the project development objectives. This restructuring included (i) an extension of 22 months; (ii) a revised implementation schedule; (iii) changes to component costs due to exchange rate differences and budget constraints in Angola; and (iv) a revised results framework to account for realities faced on the ground. This restructuring was approved in late January 2017. Over the past year, the Learning for All Project in Angola has made considerable progress. Despite challenging first years of implementation between 2014 and 2016, the Project is now on track to achieve its development objectives. The concentrated efforts by Ministry of Education leaders, increased capacity of the Project Management Team (PMT), and the successful recruitment of key service providers and technical experts have enabled the Project to begin making a noticeable difference in Angola’s education sector. Results from the first phase of the in-service teacher training program show measurable improvements in teacher capacity; data on the performance of the education system and student learning outcomes from the Early Grade Reading Assessment (EGRA) and Service Delivery Indicator Survey (SDI) are being used to inform policy and address issues of education quality; and capacity of Ministry of Education staff to effectively manage the education sector at both the central and Provincial level has improved. The development of Zones of Pedagogical Influence (ZIPs) to improve the management of schools and strengthen the connection between the school and the community is also starting to take hold. Some highlights of achievements under each component are described below. Component 1: Improving Teachers’ Skills/Knowledge and School Management Zones of Pedagogical Influence (ZIPs): • A national policy and manual was developed and all 167 ZIPs supported by the Project were established in each municipality of the country. • 30 Facilitators have been recruited to work in each Province to establish and activate school councils and ZIP councils in Project targeted areas. The number of active ZIP Councils is now 41% • School literacy kits for all 842 schools which fall under the 167 ZIPs supported by the Project were procured and sent to each Province for dissemination at the school level. Due to issues surrounding dissemination (some schools received more than one kit while others received none; and some books remained in central level warehouses), the Ministry is working on elaborating a more effective dissemination strategy. • Equipment and materials for all 167 ZIPs have been procured. While 143 of these ZIPs are ready to receive the materials (, as of September 2017, 24 ZIPs had not set-up the required storage space to receive the materials. These ZIPs have until March 2018 to secure proper storage or the materials will be sent to alternative ZIPs. In-service Teacher Training: • Overall Progress: The in-service teacher training program was divided into 3 phases. Phase 1 took place from October 2016 to August 2017, and included: developing training modules 1 and 2 (Portuguese and Mathematics), developing and administering pre-and post-tests, training the Provincial level trainers/supervisors, Municipal/ZIP level trainers, and the teachers. As planned, these training sessions took place in November, January, May and August. The results to date have been very positive. The Ministry effectively reached the expected number of participants, attendance and participation rates were high, and feedback on the training program very encouraging. Early tests results show measurable improvements in teacher content knowledge in Portuguese and Mathematics, and lessons learned from Phase 1 are being identified to adjust and improve Phase 2 of the training. • Supervisors: In addition to the original 72 trainers/supervisors selected in Phase 1, 43 more trainers from the teacher training schools have been identified to reinforce efforts in Provinces with many Municipalities and large geographic distances between Municipalities. All replacements must be completed by end of December, in time for Phase 2 of the training of trainers session which is scheduled for January 2018. School Education Projects (Projectos Educativos da Escola – PEEs), School Grants, and School Report Cards: • School Report Cards: A new school report card that focuses more on quality and targets parents and members of the community is being developed under the Project. The Ministry is working on developing template that is very visual and concise (per best practices from other countries), and that tracks key indicators identified in the PEEs (see below). The template is expected to be finalized by early 2018 and implemented along with the work to develop PEEs. • PEEs and School Grants: A manual and template to create school improvement plans and a manual to guide the school grants program have been drafted. While drafts had been finalized in July 2017, it was agreed that the PEE manual should be simplified to account for limited capacity at the school level, and the grants manual had to be further clarified and include information on the scale of the grants. Following a workshop with the Bank in September, a finalized version of these documents were agreed upon. The schools participating in the first phase are expected to be trained in February 2018 at the beginning of the new school year. While the school grants are limited to the 842 schools benefitting from the Project, the Ministry intends to implement the PEEs at a national scale. Component 2: Establishing a System for Student Assessment 12/28/2017 Page 2 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) • The Project includes an impact evaluation to determine the degree to which the interventions on teacher training and school management are able to improve education quality and learning outcomes. The design of the IE was finalized in November 2015. • Two survey instruments – EGRA and SDI – were successfully implemented in 2016 and baseline data collected for the Project. During a workshop in March 2017 to discuss an initial draft of the results, it was agreed that additional work was needed to verify the accuracy of some results and improve the structure and format of the report. To assist the Ministry, the Bank has recruited additional implementation support to help analyze the datasets and redraft the report; the final report is expected to be completed in January 2018. A detailed dissemination plan to share EGRA/SDI results more broadly was also agreed upon. • Activities for the development of a National Learning Assessment, National Exams and a 5-year Assessment Strategy are delayed. Although the Project made efforts to recruit technical assistance to develop the detailed terms of reference required to hire various firms and consultants needed to implement these activities, no available candidates with the requisite skill sets were identified both times advertised. Given the urgency to advance, the Bank is directly recruiting additional implementation support to assist with these activities. Component 3: Project Management • For more than a year, all key staff positions have been filled by both Ministry of Education and external experts, including a Deputy Project Manager, a Financial Manager, 2 Accountants, a Procurement Specialist, 2 Procurement Assistants, a Communications Specialist, and a Senior Monitoring and Evaluation Specialist. Staff working on the Project now possess the required training and skills, and quarterly reports received are timely, detailed and of quality. • Training for over 300 staff from Provincial Education Departments and Teacher Training Institutions has taken place, Bank accounts opened in all 18 Provinces, and transfers of funds for training sessions executed in all Provinces. • A firm has been recruited to further develop the Education Management Information System (SIGE) and include a specific module to organize data collected on various indicators tracked by the Project; progress has been made on hiring a technical expert to review and improve existing questionnaires to collect school level data; and training sessions at the Provincial and Municipal levels have taken place to reinforce capacity of education staff and collect missing data. Risks Systematic Operations Risk-rating Tool Risk Category Rating at Approval Previous Rating Current Rating Political and Governance  --  Substantial  High Macroeconomic  --  High  High Sector Strategies and Policies  --  Low  Low Technical Design of Project or Program  --  Substantial  Substantial Institutional Capacity for Implementation and Sustainability  --  Substantial  Substantial Fiduciary  --  Moderate  Moderate Environment and Social  --  Low  Low Stakeholders  --  Moderate  Moderate Other  --  --  -- Overall  --  Substantial  Substantial Results Project Development Objective Indicators 12/28/2017 Page 3 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDPDOTBL  Demonstrated improvement in trained teachers’ skills and knowledge as a result of the training (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 80.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 PHINDPDOTBL  Percentage of active school councils (schools targeted by the project only) (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 41.90 41.30 100.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments While more ZIP School Councils have been established - 105 by the time of the supervision mission - the number of "active" school councils slightly decreased leading to a slight decline in the overall percentage. The project is now working to ensure that activity is also maintained and the overall percentage increases in order to eventually reach the expected 100%. PHINDPDOTBL  Number of sample-based assessment exercises carried out (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 1.00 2.00 2.00 3.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Overall Comments Significant progress has been achieved on the PDO indicators. While the results for the teacher training still show as zero percent, this is because the analysis of the post-tests from the last training session in August 2017 had yet to be completed at the time of the mission. During the four teacher training sessions implemented in November 2016, January 2017, May 2017 and August 2017 to over 15,000 teacher trainers and teachers in project-supported schools, pre-and post tests were administered to determine if the training is having a positive impact on improving teachers' knowledge of Portuguese and Mathematics. Preliminary results from the tests for Modules 1 and 2 are promising, as they indicate that more than 90% of teachers demonstrated improvements in both Mathematics and Portuguese. At the time of the mission, the total number of ZIP School Councils established was 105 out of 167 (an increase from 83 in the last ISR). However, the number of active ZIP School Councils (defined as meeting at least once a trimester) was only 69. To ensure these ZIP School Councils are not just created but are active, the project is focusing its supervision to target ZIPs where additional support is needed to boost 12/28/2017 Page 4 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) school council activity. As part of the Impact Evaluation, the Ministry implemented both EGRA and SDI to collect the baseline data for the Project. A draft report with the results of these two exercises was prepared by the Ministry and is currently being revised. In addition to helping to revise and finalize the draft report, a detailed dissemination plan to share EGRA/SDI results more broadly was reviewed with the Bank’s team during the last supervision mission. Implementation of these dissemination activities is expected in early 2018. Intermediate Results Indicators PHINDIRITBL  Direct project beneficiaries (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 3,259.00 18,137.00 19,576.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments These beneficiaries include teachers participating in teacher training, ZIP supported schools, teacher training institutions, Ministry of Education staff - central, Provincial Education Departments (including accountants, ZIP Provincial Coordinators, etc.), and Municipal Education Departments (ZIP Coordinators). According to the data provided, 58% of these beneficiaries are female due to a high proportion of female teachers in the primary schools targeted by the Project. PHINDIRITBL  Female beneficiaries (Percentage, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 50.00 46.00 58.00 50.00 PHINDIRITBL  ZIPs set up and active (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 167.00 167.00 167.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 12/28/2017 Page 5 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDIRITBL  ZIP facilitators recruited and trained (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 30.00 30.00 30.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments In fact, to date, the Project has recruited 33 Facilitators to work in each Province to establish and activate school councils and ZIP councils in Project targeted areas. (Of the original group, 2 were replaced due to poor performance and another one passed away.) Out of the total 30 Facilitators, 11 of these are also School Inspectors as was agreed at the time of the MTR. These School Inspectors are filling the role as additional Facilitators to reinforce the original team in Provinces with a higher number of Municipalities and ensure greater sustainability. PHINDIRITBL  Set of school report cards produced and disseminated (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 842.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments There are already two report cards produced by the Ministry. The Project is supporting the creation of a new one that will focus more on education quality and that targets parents and members of the community. A draft template of the school report card template has been developed and is expected to be finalized by early 2018. The new school report cards will be implemented along with the first phase of PEEs in February 2018. It is expected that of the total 842 schools, nearly half of these will begin producing these report cards in 2018 and the other half will begin in 2019. PHINDIRITBL  In-service teacher training modules aligned with learning goals developed (six modules) (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Value No No 2 out of 6 Yes Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments The in-service teacher training program is being developed in 3 phases, and contain a total of 6 modules. The first 2 modules (Portuguese and Mathematics) were designed, aligned with learning goals developed, and piloted in November 2016 and January 2017. They were then 12/28/2017 Page 6 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) finalized and implemented during the training program with 15,000 teachers in May and August 2017. Modules 3 and 4 have been developed and analyzed to confirm their alignment with the learning goals; they are now being finalized and printed. The development of modules 5 and 6 has not started yet. PHINDIRITBL  Teachers recruited or trained (Number, Corporate) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 15,043.00 15,000.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments The first rounds of training at the school level took place in May and August 2017.. Because of the Presidential Elections in August, participation during the August session was slightly lower than expected; however the training was repeated in Provinces where attendance was an issue, and the final total number of teachers reached was 15,043. These same teachers will participate in the training program over the next 3 years. PHINDIRITBL  Teacher training schools equipped (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 13.00 18.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments Out of the 18 teacher training schools to be supported by the Project with equipment, 13 have already received equipment. The equipment for the remaining 5 schools is expected to arrive to the schools in early 2018. 12/28/2017 Page 7 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDIRITBL  ZIP center schools equipped (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 167.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments The 167 ZIPs to be supported by the Project have been identified. The list of equipment to be purchased has also been agreed upon. Procurement of this equipment was finalized, and the equipment is ready to be disseminated. Based on information provided during the mission, out of the total 167 ZIPs, 143 were deemed ready to receive the materials, while 24 of them have not yet established a secure space where this equipment will be stored. Distribution of materials to the 143 ZIPs that are ready is expected by end of 2017; the remaining 24 schools will have until March 2018 to demonstrate they have set up a secure space to receive the materials. PHINDIRITBL  Number of school kits produced and distributed (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 842.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments 842 school kits have been procured and distribution all of materials is expected to be finalized in early 2018. These kits include a small library of 56 books, a geometry kit for teachers, a kit of games for stimulating reading and creative writing, and pedagogical materials for teachers. Procurement of these kits was finalized and they are now in Luanda ready to be distributed. (The school library books were procured earlier and already disseminated although issues arose in several Provinces where dissemination of these materials was an issue. Based on these lessons, the Project has been paying careful attention to dissemination plans for this second round of materials and to ensure all targeted schools receive the full kits.) PHINDIRITBL  Number of education projects approved (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 520.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments The "Projecto Educativo da Escola" (PEE) manual has been developed and is being finalized. The PEEs and school grants program have 12/28/2017 Page 8 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) been divided into two phases. The training on the guidelines for Municipalities, School Directors, and School Councils participating in the first phase is expected to take place in February 2018, at the beginning of the school year. The second phase is expected at the beginning of the school year in February 2019. PHINDIRITBL  Number of grants disbursed (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 0.00 520.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments The school grants will only happen after schools have developed their "Projectos Educativos". The first grants aren't expected until mid 2018. PHINDIRITBL  Assessment framework reviewed (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N N N Y Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 PHINDIRITBL  Minimum standards defined as well as bank of items and standardized tests constructed (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value N N N Y Date 01-Nov-2012 01-Dec-2016 20-Oct-2017 28-Feb-2020 12/28/2017 Page 9 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDIRITBL  Impact evaluation phases completed (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Follow-Up Data Baseline Report Field Visits to Monitor Value Design Analysis, and Drafted Implementation IE report Date 01-Nov-2013 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments The report with the baseline data for the IE has been drafted. It is undergoing a second round of revisions. PHINDIRITBL  System for learning assessment at the primary level (Yes/No, Custom) Baseline Actual (Previous) Actual (Current) End Target Value Y Y Y Y Date 01-Nov-2012 20-Mar-2017 06-Dec-2017 28-Feb-2020 Comments The Ministry developed a dissemination plan to share the outcomes of the EGRA and SDI surveys, target audiences were identified, and an agreement was reached on the different activities to be financed under the Project. This is to ensure the utility of the learning assessment system and use of data to improve education quality. PHINDIRITBL  Utility of the learning assessment system (Number, Custom Supplement) Baseline Actual (Previous) Actual (Current) End Target Value 1.00 1.00 1.00 3.00 12/28/2017 Page 10 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) PHINDIRITBL  Number of provincial, municipal and teacher training schools officers trained in management around the project (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 266.00 304.00 254.00 Date 01-Nov-2012 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments A number of trainings took place in 2016 and 2017 to launch activities that are expected to take place at the Provincial and Municipal levels. These included training for Provincial accountants (approx. two from each Province - 35) in FM and Procurement, training for teacher training school and other training Supervisors (87), training in Projectos Educativos and school grants (54), and another 128 staff Provincial and Municipal staff in basic project management. To reinforce capacity, additional rounds of training are envisioned in 2018. PHINDIRITBL  Number of teacher kits produced and distributed (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 0.00 15,043.00 15,000.00 Date 01-Dec-2016 20-Mar-2017 20-Oct-2017 28-Feb-2020 Comments Materials for modules 1 and 2 of the teacher training program (Math and Portuguese) were already distributed to all 15,043 teachers trained. Modules for the upcoming teacher trainings (modules 3-6) are yet to be distributed. Overall Comments Data on Financial Performance Disbursements (by loan) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed Disbursed P122700 IDA-52540 Effective USD 75.00 75.00 0.00 27.10 41.72 36% 12/28/2017 Page 11 of 12 Public Disclosure Copy Public Disclosure Copy The World Bank Implementation Status & Results Report Angola Learning for All Project (P122700) Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P122700 IDA-52540 Effective 26-Sep-2013 16-Jan-2014 16-Jun-2014 30-Apr-2018 28-Feb-2020 Cumulative Disbursements Restructuring History Level 2 Approved on 29-Jan-2017 Related Project(s) There are no related projects. 12/28/2017 Page 12 of 12 Public Disclosure Copy