FOR OFFICIAL USE ONLY Report No: ICR00005696 INTERNATIONAL DEVELOPMENT ASSOCIATION IMPLEMENTATION COMPLETION AND RESULTS REPORT IDA52090, IDA60900, TF14107, TF14815, TFA4689, TFA4705 ON A CREDIT IN THE AMOUNT OF SDR 65.1 MILLION (US$100 MILLION EQUIVALENT) AND A GRANT OF US$20 MILLION FROM THE HEALTH RESULTS INNOVATION TRUST FUND AND ADDITIONAL FINANCING IN THE AMOUNT OF SDR 110.6 MILLION (US$150 MILLION EQUIVALENT) A GRANT OF US$60 MILLION FROM THE GLOBAL FINANCING FACILITY A GRANT OF US$20 MILLION FROM THE POWER OF NUTRITION TRUST FUND TO THE FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA FOR THE HEALTH SUSTAINABLE DEVELOPMENT GOALS PROGRAM-FOR-RESULTS December 23, 2022 Health, Nutrition and Population Global Practice Eastern and Southern Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2022) Ethiopian Birr Currency Unit = (ETB) ETB52.1500= US$1 US$1.0 = SDR 0.75313681 FISCAL YEAR July 8 – July 7 Regional Vice President: Victoria Kwakwa Country Director: Ousmane Dar Regional Director: Amit Dar Practice Manager: Ernest E. Massiah Task Team Leaders: Roman Tesfaye, Enias Baganizi ICR Main Contributor: Miriam Schneidman ABBREVIATIONS AND ACRONYMS ANC Antenatal Care AF Additional Financing CBHI Community-Based Health Insurance JCCC Joint Core Coordinating Committee COVID-19 Coronavirus Disease 2019 CPS/CPF Country Partnership Strategy/Country Partnership Framework CPR Contraceptive Prevalence Rate CRI Corporate Results Indicator CRVS Civil Registration and Vital Statistics DHIS2 District Health Information System DLl Disbursement Linked Indicator DP Development Partner DPO Development Policy Operation EA Environmental Assessment E&S Environmental and Social EDHS Ethiopia Demographic and Health Survey EFY Ethiopian Fiscal Year EmONC Emergency Obstetric and Newborn Care ESSA Environmental and Social System Assessment ETB Ethiopian Birr ESS Ethiopian Statistics Service FCV Fragility, Conflict, and Violence FEACC Federal Ethics and Anti-Corruption Commission FM Financial Management FSA Fiduciary Systems Assessment GAVI Global Alliance for Vaccines and Immunization GBV Gender-Based Violence GDP Gross Domestic Product GTP Growth and Transformation Plan GFF Global Financing Facility GMP Growth Monitoring and Promotion GRS Grievance Redress Service HCI Human Capital Index HCP Human Capital Project HIV Human Immunodeficiency Virus HMIS Health Management Information System HNP Health, Nutrition, and Population HP Health Post HRC High Risk of Conflict HSTP Health Sector Transformation Plan HUMC Health Unit Management Committee IBRD International Bank for Reconstruction and Development ICR Implementation Completion Report ICS Immigration and Citizenship Service IDA International Development Association IDP Internally Displaced Person IFA Iron Folic Acid IFC International Finance Corporation IFR Interim Unaudited Financial Report IHP International Health Partnership IOM International Organization for Migration IPF Investment Project Financing JCF Joint Consultative Forum KRA Key Results Area M&E Monitoring and Evaluation MDG Millennium Development Goals MDGPF Millennium Development Goals Performance Fund MOH Ministry of Health MOU Memorandum of Understanding MS Moderately Satisfactory MU Moderately Unsatisfactory NHCAP National Health Climate Adaptation Plan NCD Non-Communicable Disease NGO Non-Governmental Organization OHCHR Office of United Nations High Commissioner for Human Rights OOP Out-of-Pocket PAD Program Appraisal Document PAP Program Action Plan PDO Program Development Objective PFM Public Financial Management PforR Program for Results PFSA Pharmaceuticals Fund and Supply Agency PHC Primary Health Care PHCU Primary Health Care Unit PNC Postnatal Care RA Results Area RHB Regional Health Bureau RMNCAH+N Reproductive, Maternal, Neonatal, Child, and Adolescent Health Plus Nutrition SARA Service Availability and Readiness Assessment SDG Sustainable Development Goal SDGPF Sustainable Development Goals Performance Fund SEA/SH Sexual Exploitation and Abuse/Sexual Harassment SHI Social Health Insurance SNNPR Southern Nations, Nationalities, and Peoples’ Region SPA+ Service Provision Assessment Plus TA Technical Assistance TTLs Task Team Leaders UHC Universal Health Coverage UK-FCDO UK- Foreign, Commonwealth & Development Office UN United Nations UNICEF United Nation Children’s Fund UNFPA United Nations Population Fund UN-OCHA United Nations Office for the Coordination of Humanitarian Affairs VAS Vitamin A supplementation WASH Water, Sanitation, and Hygiene WHO World Health Organization TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROGRAM CONTEXT AND DEVELOPMENT OBJECTIVES .................................................... 6 A. CONTEXT AT APPRAISAL AND THEORY OF CHANGE .................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 11 II. OUTCOME .................................................................................................................... 15 A. RELEVANCE .......................................................................................................................... 15 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 16 C. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 23 D. OTHER OUTCOMES AND IMPACTS ........................................................................................ 24 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 24 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 26 A. QUALITY OF MONITORING AND EVALUATION ....................................................................... 26 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 28 C. BANK PERFORMANCE ........................................................................................................... 29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 31 ANNEX 1. RESULTS FRAMEWORK, DISBURSEMENT LINKED INDICATORS, AND PROGRAM ACTION PLAN ...................................................................................................................... 34 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 71 ANNEX 3. PROGRAM EXPENDITURE SUMMARY ................................................................... 74 ANNEX 4. BORROWER’S COMMENTS ................................................................................... 75 ANNEX 5. SUMMARY OF REVISED OUTCOMES AND DLIS ...................................................... 76 ANNEX 6. SUMMARY OF ACHIEVEMENTS: DISBURSEMENT LINKED INDICATORS .................. 78 ANNEX 7. SUPPORTING DOCUMENTS – BORROWER’S REPORT ............................................ 81 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) DATA SHEET BASIC INFORMATION Product Information Program ID Program Name Financing Instrument Health Sustainable Development Goals P123531 Program-for-Results Financing Program-for-Results Country IPF Component Ethiopia Yes Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Immigration and Citizenship Service (ICS), Ministry of FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA Health (MOH) Program Development Objective (PDO) Original PDO To improve the delivery and use of a comprehensive package of maternal and child health services. Page 1 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) FINANCING FINANCE_TBL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Administered Financing 100,000,000 100,000,000 94,346,690 IDA-52090 20,000,000 20,000,000 20,000,000 TF-14107 150,000,000 150,000,000 155,870,882 IDA-60900 20,000,000 20,000,000 20,000,000 TF-A4705 60,000,000 60,000,000 59,265,730 TF-A4689 Total 350,000,000 350,000,000 349,483,302 Non-World Bank Administered Financing Borrower/Recipient 0 0 0 Total 0 0 0 Total Program Cost 350,000,000 350,000,000 349,483,303 KEY DATES Program Approval Effectiveness MTR Review Original Closing Actual Closing P123531 28-Feb-2013 29-Mar-2013 18-Jan-2016 30-Jun-2018 30-Jun-2022 Page 2 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 20-Apr-2020 246.99 Change in Implementing Agency Change in Results Framework Reallocation between and/or Change in DLI Change in Disbursements Arrangements Change in Institutional Arrangements Other Change(s) 13-Apr-2021 311.95 Change in Loan Closing Date(s) Change in Implementation Schedule 13-Oct-2021 315.34 Reallocation between and/or Change in DLI 13-Dec-2021 340.37 Change in Results Framework Change in Loan Closing Date(s) 29-Mar-2022 340.37 Change in Loan Closing Date(s) Change in Implementation Schedule KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROGRAM PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 22-May-2013 Satisfactory Satisfactory 0 02 29-Dec-2013 Satisfactory Moderately Satisfactory 35.39 03 21-Jun-2014 Satisfactory Moderately Satisfactory 35.47 04 22-Dec-2014 Satisfactory Moderately Satisfactory 69.99 05 22-Jun-2015 Satisfactory Moderately Satisfactory 69.99 06 23-Dec-2015 Satisfactory Moderately Satisfactory 70.12 07 23-Feb-2016 Satisfactory Moderately Satisfactory 70.12 08 20-May-2016 Satisfactory Moderately Satisfactory 73.79 09 18-Nov-2016 Satisfactory Moderately Satisfactory 75.52 Page 3 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) 10 28-Jun-2017 Satisfactory Moderately Satisfactory 75.62 11 08-Dec-2017 Satisfactory Moderately Satisfactory 82.16 12 14-Jun-2018 Moderately Satisfactory Moderately Satisfactory 156.20 13 11-Dec-2018 Moderately Satisfactory Moderately Satisfactory 166.91 14 16-May-2019 Moderately Satisfactory Moderately Satisfactory 195.73 15 28-Jun-2019 Moderately Satisfactory Moderately Satisfactory 196.32 16 30-Oct-2019 Moderately Satisfactory Moderately Satisfactory 196.32 17 08-Apr-2020 Moderately Satisfactory Moderately Satisfactory 247.39 18 05-Oct-2020 Moderately Satisfactory Moderately Satisfactory 312.35 19 15-Mar-2021 Moderately Satisfactory Moderately Satisfactory 312.35 20 02-Sep-2021 Moderately Satisfactory Moderately Satisfactory 315.33 21 21-Apr-2022 Moderately Satisfactory Moderately Satisfactory 340.77 22 29-Jun-2022 Moderately Satisfactory Moderately Satisfactory 343.16 SECTORS AND THEMES Sectors Major Sector/Sector (%) Health 100 Health 100 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100 Health Systems and Policies 100 Health System Strengthening 100 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Victoria Kwakwa Page 4 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Country Director: Guangzhe Chen Ousmane Dione Director: Ritva S. Reinikka Amit Dar Practice Manager: Olusoji Owolabi Adeyi Ernest E. Massiah Task Team Leader(s): Gandham N.V. Ramana Roman Tesfaye, Enias Baganizi ICR Contributing Author: Miriam Schneidman Page 5 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) I. PROGRAM CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL AND THEORY OF CHANGE Context 1. At the time of appraisal of the Health Millennium Development Goals (MDG) Program for Results (PforR) in early 2013, Ethiopia had experienced a decade of strong economic growth, progress on poverty reduction, and significant improvements in human development. The country had a long-standing strong track record of macroeconomic management with an economic growth rate of 10.6 percent, nearly double the average for the sub- Saharan Africa region. While nearly 40 percent of Ethiopians lived in extreme poverty in 2004, five years later this dropped to about 30 percent. Commensurate improvements were noted in expanding access to primary education, clean water, and basic health services. The 2011 Demographic and Health Survey (DHS) found a rapid decline in several key outcome indicators, including infant and under-five mortality and childhood stunting and a modest drop in fertility. Ethiopia was considered on track to reach several MDG targets (i.e., child health, HIV/AIDS, and malaria). 2. Despite progress, Ethiopia continued to face persistent gaps and disparities in access and utilization of quality maternal and child health services and lagging health outcomes. The maternal mortality ratio (MMR) had not changed significantly during the previous five years, remaining stubbornly high at 676 per 100,000 live births (2011). Use of maternal health services, such as antenatal care (i.e., 43 percent of pregnant women received at least one antenatal care visit) and skilled attendance at birth (i.e., a meager 10 percent of women received skilled care at birth) remained considerably below regional averages. Utilization of contraceptives nearly doubled during the previous five years, but the Contraceptive Prevalence Rate (CPR) remained low at about 27 percent. The overall financing for health was relatively modest (US$16 per capita) with external assistance and out-of-pocket expenditures contributing a significant portion of overall financing. External assistance to the health sector focused on a few diseases, leaving critical gaps in the delivery of maternal and child health services. 3. Ethiopia benefited from a strong enabling policy and institutional environment with staunch commitment to the MDGs. Ethiopia’s Growth and Transformation Plan (GTP) 2011-2015 gave a high priority to human development and placed a strong focus on the MDGs. The Health Sector Development Program (HSDP) was a key component of the GTP and the main vehicle for achieving the government’s targets. The HSDP benefitted from harmonized donor support with Ethiopia being the first country to sign the International Health Partnership (IHP+) country compact. Ethiopia had well established institutional mechanisms, and procedures for pooling health resources (e.g., MDG Performance Fund, MDGPF, block grants to decentralized structures in the devolved health system). The MDGPF was considered an effective mechanism for pooling resources to address gaps in maternal and child health services. 4. There was growing recognition in the country of the need to shift focus on results, enhance accountability, and leverage ongoing initiatives and reforms. In a rapidly evolving federal health system, accelerating on-going efforts to institutionalize monitoring and evaluation systems and provide reliable and timely information on Program results was expected to bolster accountability and strengthen management. Evidence-based planning was to be strengthened through improved tracking and monitoring of Program results. Reforms were to also build and leverage on successful programs, such as the country’s flagship health extension worker program. High-level government officials, including the Prime Minister of Ethiopia, challenged the World Bank team to come up with a results-focused instrument to channel funds to the health sector that would be better suited to the national context. The World Bank responded favorably, designing an innovative PforR, which was the first both in the Ethiopia portfolio and in the Health, Nutrition and Population (HNP) Global Practice. Page 6 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Theory of Change (Results Chain) 5. The Program Appraisal Document (PAD) included a table on the scope and results supported under the Program through the MDGPF, which included most of the elements needed for the Theory of Change. 1 For this Implementation Completion Report (ICR), a Theory of Change was constructed ex-post and has been retrofitted to include the Results Framework for both the original Program and the Additional Financing (AF), as well as the long- term goals to which the Program was contributing (Table 1). Table 1: Theory of Change PRIORITY ACTIVITIES OUTPUTS/ OUTCOMES LONG-TERM AREAS INTERMEDIATE RESULTS GOALS PDO: “To improve the delivery and use of a comprehensive package of maternal and child health services”, as measured by: Accelerate -Supply equipment and -Health centers offer basic emergency -Deliveries attended by skilled -Reduction in progress commodities for obstetric care birth provider (PDO1) maternal towards provision of emergency -Woredas have functional ambulance -Deliveries attended by skilled mortality and maternal obstetric care services birth provider for bottom three fertility health MDGs -Supply contraceptives -Midwives receive in-service training performing regions (PDO1a) -Provide ambulances to -Health officers trained in emergency -Pregnant women receiving at all Woredas surgical and obstetric care least one antenatal care visit -Conduct in-service (PDO3) training of midwives -Pregnant women receiving at and Health Officers in least four antenatal care visits Emergency Surgical and (PDO3a) Obstetric skills -Contraceptive Prevalence Rate -Conduct capacity (PDO4) building of health - Contraceptive Prevalence Rate extension workers in in rural areas (PDO4a) clean and safe delivery - Pregnant women taking iron and folic acid tablets (PDO5) 1According to the Operations Policy and Country Services guidelines, including the Theory of Change became mandatory for PADs as of May 2018. The PforR was approved by the World Bank’s Board of Directors in July 2013. Page 7 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Sustain the -Strengthen cold chain -Health centers have functional cold -Children 12-23 months Reduction in gains made in systems chain equipment immunized with Pentavalent 3 infant and child health -Supply vaccines -Outreach campaigns held vaccine (PDO2) under-five MDG -Hold immunization -Long lasting insecticidal nets - Woredas in non-emerging mortality campaigns distributed regions delivering vitamin A -Supply bed nets -Growth Monitoring and Promotion supplements to children (PDO7) activities (IO 8) -Children 6-59 months receiving -Woredas in emerging regions Vitamin A supplements (PDO6) transitioning from Enhanced Outreach Services to Community Health Days (IO9) -Immunization campaigns at IDP/refugee camps for missed vaccinations; and targeted catch up immunization campaigns in four regions (IO17) -Catch up campaigns to increase uptake of essential health and nutrition services (IO18) Health centers reporting HMIS data in -Improved Strengthen -Constructing health time (IO1) health system health centers -Development and implementation of performance for systems -Supplying essential Balanced Score Card to assess enhanced medical products and performance and related institutional- delivery of equipment incentives (IO2)-dropped quality -Validating HMIS semi- -Development and implementation of maternal and annually Annual Rapid Facility Assessment (IO3) child health -Undertaking surveys -Improve transparency of PFSA services and studies Procurement Processes (IO4) -Introduce performance indicators (IO5); automate PFSA business fiduciary system, (IO6) (dropped); submit backlog audit reports, (IO7) (dropped) -Ensure Primary Health Care (PHC) facilities have all drugs on MOH essential drug list (IO10) -Develop and implement postnatal care directive to improve quality of care (IO11) -Improve quality of adolescent health services (IO12) -Woredas with functional Community Based Health Insurance schemes (IO13); undertake CBHI review every two years (IO14) -Devise and implement a mechanism for documenting consultations when communal/private land is used for construction of health facilities (IO15) - Develop and implement a Health Sector Community Score Card (IO16) Page 8 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Rationale for PforR Support, and Program Scope and Boundaries 6. The PforR instrument was a good fit for supporting the Government’s HSDP which used evidence -based policies and strategies and had a good track record of results. Hence, the Government decided to use the health sector to test the PforR instrument in Ethiopia. The PAD provided a strong and clear rationale for use of the PforR instrument, namely it would: (i) support the Government to adopt a greater focus on results; (ii) allow the World Bank to effectively harmonize its support with other development partners; (iii) use and strengthen country systems; and (iv) motivate program managers to find solutions for addressing bottlenecks and inequities. 7. The scope of the PforR operation was fully aligned with the overarching framework of the Government’s HSDP IV. The HSDP IV envisioned a strong client-centric approach, ensuring timeliness, quality, safety, and responsiveness. The HSDP IV had a pro-poor focus, targeting four underserved regions; supported key health financing reforms to strengthen financial protection; and promoted citizen accountability. The boundaries of the PforR operation were set clearly to contribute to the HSDP IV objectives by disbursing against achievement of a targeted set of results while supporting key activities under the MDGPF except for high-value procurement. 2 The PforR operation complemented and reinforced International Development Association (IDA) support of block grants for health extension workers and essential medical products and cash transfers to sub-national governments through the social protection sector. The MDGPF was considered an effective mechanism for supporting financing gaps in maternal (e.g., equipment and commodities for emergency obstetric care, ambulances, and contraceptives) and child (e.g., cold chain strengthening, vaccine acquisition and campaigns) health and strengthening health systems (e.g., procurement of medical equipment; improving the Health Management Information System (HMIS); conducting surveys). Given that the MDGPF was fully under the control of the Ministry of Health (MOH) this provided a unique opportunity to disburse funds directly to the sector for enhanced performance. Alignment and harmonization of donor support was to be done using the principles of ‘One Plan’, ‘One Budget’, ‘One Program’ following country systems, and ‘One set of reporting requirements’ with common platforms and mechanisms, leveraging existing arrangements and mechanisms and minimizing the transaction costs for the Government. 3 Program Development Objectives (PDOs) 8. The Program Development Objective as stated in both the Program Appraisal Document and Financing Agreement, was to improve the delivery and use of a comprehensive package of maternal and child health services.4 2 As noted in the PAD, the maximum value of a single contract committed using the MDGPF was about US$12 million for the Pharmaceutical Fund and Supply Agency and US$20 million for the MOH for procurement of ambulances. To address the risk of financing contracts exceeding PforR high value contract exclusion thresholds, the IDA credit and Health Results Innovation Trust Fund grant funds were placed into a sub- account within the MDGPF, which provided assurances that World Bank funds would be used for intended purposes while allowing funds to be fully harmonized. 3 The MDGPF used an annual Joint Consultative Forum involving all key stakeholders to discuss and agree on priorities to improve health outcomes. The Joint Financing Agreement set out the governance and reporting requirements for the MDGPF which was supported by multiple development partners (e.g., UK Department of International Development, Spanish Cooperation, Australian AID, Italian Cooperation, Irish AID, UNFPA, UNICEF, WHO and the Netherlands Government). 4 The PDO was one of the strategic objectives of the HSDP IV that focused on improving access to health services and promoting evidence-based decision making. Page 9 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Key Expected Outcomes and Outcome Indicators 9. Progress toward achieving the PDO would be measured by four outcome indicators which were used as proxies for measuring the delivery and use of a comprehensive package of maternal and child health services. In addition, the Program included four Intermediate Outcome Indicators (IOs) related to health system strengthening which would contribute to the achievement of the PDOs (Table 2). The outcome indicators were: • Deliveries attended by skilled birth providers (%) • Pregnant women receiving at least one antenatal care visit (%) • Contraceptive Prevalence Rate (%) • Children 12-23 months immunized with Pentavalent 3 vaccine (%) Program Results Areas and DLIs 10. The PforR operation aimed to support Ethiopia to make progress towards the achievement of the maternal and child health MDGs. The PforR operation was financed through a US$100 million IDA credit and a US$20 million grant under the Health Results Innovation Trust Fund. Both supported the achievement of Disbursement Linked Indicators (DLIs) focused on agreed service delivery outcomes. In line with the PDO, the PforR had three priority areas that could be achieved within the timeframe of the operation: (i) accelerating progress towards the maternal health MDG; (ii) sustaining gains in the child health MDG; and (iii) strengthening health systems. These three priority areas were well articulated, relevant, and timely, given the sluggish progress on the maternal health MDG and the strong progress on the child health MDG while needing to bolster health system capacity to attain and sustain these results. The health systems DLIs aimed to support the government to strengthen its capacity to manage a large, complex, devolved health system, focusing on introducing new tools for monitoring and evaluation and enhancing transparency of the Pharmaceuticals Fund and Supply Agency (PFSA) procurement process. The design was simple, focused, and targeted, ambitious in its vision, and yet practical and operational in terms of expected outcomes, allowing the government to test the utility of the instrument and draw lessons for its replicability. 11. The Program included eight DLIs. For the ICR report, the DLIs have been attributed to the results areas they contributed to, as articulated in the PDO, and as summarized in Table 2. The first four DLIs captured improvements in both the delivery and utilization of key maternal and child services. The four DLIs were effectively used as proxies to measure improved access to maternal and child services. These are standardized, globally accepted core indicators which are monitored and tracked regularly by all countries, including Ethiopia. Hence, measurement was easily done with benchmarking over time. The DLIs were well selected to address areas where Ethiopia was lagging (e.g., skilled attendance at birth, antenatal care) or needed to accelerate progress (i.e., contraceptive use).5 The targets for the first four DLIs were established based on global experience with annual rates of change, and what was deemed to be achievable by virtue of activities to be undertaken by the Government under the HSDP IV, as noted in both the PAD and Financing Agreement. The targets established were deemed challenging but achievable based on the systematic analysis conducted at appraisal. Even though quality was not explicitly mentioned, with the enhanced access to staff training, and the provision of equipment, drugs, and supplies supported under the Program, the DLIs also contributed to the quality of care, such as DLI1 (skilled attendance at birth). The health systems DLIs contributed 5 For example, at Program inception, births attended by skilled provider was a meager 10 percent in Ethiopia in contrast to the sub-Saharan regional average of 51.4 percent; similarly, only 43 percent of women received antenatal care in Ethiopia in comparison to the SSA average of nearly 74 percent; while the CPR was slightly higher in Ethiopia (about 27 percent) compared to the regional average (about 24 percent), it still remained too low to accelerate progress towards the demographic transition. Page 10 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) to the delivery and utilization of these services as well as to overall efficiency, given the enhanced access to routine data, improved monitoring and tracking of results, and strengthened management of the system which allowed policy makers and practitioners to identify bottlenecks and propose remedial actions. Program Action Plan 12. The DLIs were complemented by a Program Action Plan (PAP) with 17 actions which aimed to address gaps identified in the technical, fiduciary, and environmental and social (E&S) assessments (Annex I). The PAP focused on two key operational gaps: (i) using the balanced scorecard approach for performance monitoring (DLI6); and (ii) putting in place an information mechanism to assess facility readiness to deliver essential services (DLI7). The PFSA fiduciary reform was another key area with a focus on improving transparency, contract management and complaint redress (DLI8). 6 In addition, the PAP included several actions to strengthen E&S safeguards (i.e., infection prevention, appropriate hazardous waste disposal, patient service committees, outreach activities for vulnerable persons). Table 2: Program development indicators and disbursement-linked indicators by areas of the PDO Type of Priority Results Disbursement-linked indicators (DLIs) indicator area area DLI 1. Deliveries attended by skilled birth provider (%) PDO Maternal Health MDG Delivery, Utilization, Quality DLI 2. Children 12-23 months immunized with PDO Child Health MDG Delivery, Utilization, Quality Pentavalent 3 vaccine (%) DLI 3. Pregnant women receiving at least one antenatal PDO Delivery, Utilization Maternal Health MDG care visit (%) DLI 4. Contraceptive prevalence rate (%) PDO Maternal Health MDG Delivery, Utilization DLI 5. Health centers reporting HMIS data on time (%) IO DLI 6. Development and implementation of a balanced IO score card approach Strengthened Delivery, Utilization, DLI 7. Development and implementation of Annual IO Health Systems Efficiency Rapid Facility Assessment DLI 8. Transparency of Pharmaceutical Fund and Supply IO Agency procurement process B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs Outcome Targets, Result Areas, and DLIs Additional Financing (2017) 13. Building on the successful performance of the original Program, Additional Financing was mobilized to expand the scope and scale and maximize impact. To this end, AF of US$230 million (including US$150 million, IDA; US$60 million grant, Global Financing Facility (GFF), and US$20 million grant, Power of Nutrition) was approved on April 20, 2017. The PDO remained unchanged. The PforR boundary also remained unchanged with the Program continuing to support results under the Sustainable Development Goals Performance Fund (SDGPF), formerly called MDGPF. To enhance development impact, the AF continued to use DLIs to reflect the Government’s focus on equity/quality, 6The focus on PFSA was considered important as a significant portion of the MDGPF was spent on procurement of drugs and supplies; while MDGPF related audits were up to date, audits of government-financed program activities were way behind. Page 11 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) information revolution, and Woreda transformation, in line with the Country Partnership Framework (CPF) FY18-22. Building on the successful four-year track record of the original operation, outcome targets and DLIs were refined to: (i) scale up DLI targets which had been met and set higher targets and more ambitious objectives; (ii) accelerate progress on addressing persistent socio-economic and geographic inequities in maternal and child health services; (iii) include a focus on community health insurance for rural households; (iv) add a focus on nutrition, in line with the CPF FY18-22; and (v) strengthen select fiduciary and system targets. 7 Building on the original PAP, new and incremental assessments were conducted.8 The technical assessment was broadened to include several new areas: citizen engagement and social accountability; nutrition; and gender. Details of changes to the PDO/IO/DLI indicators are provided in Annex 5. 14. Program Action Plan. Several actions were dropped from the PAP, including: (i) developing and implementing balanced score card approach to assess facility performance and related institutional incentives as the tool was already rolled out nationwide (Action #1); (ii) exchange rate to be applied on the basis of actual rate of transfer while reporting expenditures to minimize reported exchange losses, as the World Bank confirmed that there was no gain/loss arising from PFSA transactions since all payments and transfers were made in foreign currency (Action #7); and (iii) FMOH through appropriate consultation to consider establishing fiduciary subcommittee of Joint Core Coordinating Committee (JCCC) to monitor budget performance of SDGPF, advances, reporting, procurement and audit issues as consultations with partners deemed this unnecessary. 15. New Components. With the approval of AF an Investment Project Financing (IPF) component (US$21 million) was added for the first time. The IPF component aimed to address two critical areas: (i) strengthening the Civil Registration and Vital Statistics System (CRVS) which is essential to providing key maternal and child health information; monitoring underage marriages; developing a modern system of identification; and (ii) providing complementary capacity building and technical assistance to further strengthen Program implementation. The IPF component had three sub-components: (i) Sub-Component 1: the CRVS (US$15 million) at the Federal Vital Events Registration Agency to build the CRVS system through technical assistance, capacity building and procurement of equipment; (ii) Sub- Component 2: Technical Assistance and Capacity Building for Nutrition (US$5 million) for the Government’s National Nutrition Program II Agenda in various capacity building areas including multisectoral coordination, evaluation and lesson learning; and (iii) Sub-Component 3: Unallocated amount (US$1.0 million) was reserved to support various areas such as fiduciary management at the PFSA and MOH; implementation of the national Health Care Financing Strategy; and data and management information systems and surveys. Program Restructuring (2020-2022) 16. The Program was restructured five times as summarized in Table 3. The World Bank team worked closely with government to systematically address emerging issues. The first restructuring (April 2020) was necessitated by the transfer of the CRVS program to a new institution and by a delay in providing GFF financing. The second restructuring (April 2021) involved a change in the data verification method for DLI2 (children 12-23 months immunized with Pentavalent vaccine) and a 6-month extension of the closing date. The third restructuring (October 2021) involved substantial changes to DLIs and verification sources and reallocation of funds, as Ethiopia, like all other countries, was 7 In total, for the PDO indicators/DLIs, 4 were revised/scaled and 3 were added. For the Intermediate Results indicators/DLIs, 3 were revised/scaled; 12 were added; and 1 was dropped. With the approval of the AF, the PforR operation now included 22 DLIs, in comparison to 8 DLIs at appraisal. 8 The technical assessment and the environmental and social assessment prepared for the original Program were updated and a new Integrated Fiduciary Assessment was conducted. Page 12 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) unable to carry out population-based surveys during the COVID-19 pandemic which was compounded by the deteriorating security situation in the country. Likewise, several fiduciary and safeguard DLIs were not met and needed to be revised. In total, 8 out of 22 DLIs were revised and/or data verification sources changed to allow the Government to effectively use all resources prior to Program completion. The fourth restructuring (December 2021) was triggered by difficulties to conduct catch up vaccination campaigns and to carry out an assessment on the integration of youth friendly health services, resulting in a three-month extension of the closing date. The fifth restructuring (March 2022) was done due to the security situation to allow more time to complete activities under component 1 and to revise related end of Program targets, as well as provide another three-month extension of the closing date. In total, there were four extensions of the closing date (including the one approved with the AF) as described in detail in Table 3 and in Annex 5 (Summary of Revised Outcomes and DLIs). Table 3: Summary of changes during implementation of the Ethiopia PforR Changes WB Objectives/Rationale Financing Approval Agreement Revision to -Changes to procedures for conducting financial audits of PFSA Revised Financing Agreement January 15, 2014 Additional Financing April 20, 2017 -Provide AF Revised Total US$230m AF -Amend DLIs May 18, 2017 US$150million IDA; -Add IPF component US$40millionGFF; -Extend closing date to June 30, 2021 US$20million PoN Restructuring April 21, 2020 -Change in implementation arrangements for sub-component 1 Revised First from VERA to INVEA - Changes in activities and budget for Sub-Component 1 (ICT related activities aimed to support the national electronic registration system were dropped; technical assistance and capacity building at lower administrative levels were added) -Inclusion of lagged financing of US$40 million (GFF) Restructuring April 15, 2021 -Change in the data source for verification of DL2 (children 12-23 NA Second--Level II months immunized with Pentavalent 3 Vaccine) to the DHS, as the Household Cluster Survey could not be conducted due to the pandemic, and the HRITIF grant was due to close on June 30, 2021 - Change in the closing date from June 30, 2021, to December 31, 2021, to avoid an implementation gap until second AF is approved Restructuring October 18, -Difficulties in validating several 2020 results as 2020 DHS and NA Third--Level II 2021 Service Availability Readiness Survey were not conducted due to the COVID-19 pandemic and the deteriorating security situation -Several fiduciary and safeguard indicators could also not be met due to institutional changes -DLIs restructured and/or verification sources revised along with reallocated amounts to allow Government to use all resources before Program closure Restructuring December 13, -Due to the security situation, the MOH was unable to report on NA Fourth--Level II 2021 missed vaccinations at camps for IDPs and to conduct an assessment of youth friendly health service integration into PHC -Closing date extended from December 30, 2021, to March 30, 2022 Restructuring March 28, 2022 -Activities under IPF sub-component 1 required additional time to NA Fifth--Level II be fully implemented and disbursed -End of Project targets revised for some indicators -Closing date extended from March 30, 2022, to June 30, 2022 Page 13 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Rationale for Changes and their Implication for the Original Theory of Change 17. The main rationale for changes made was to leverage on the initial successful track record of the PforR to add new DLIs and activities, while taking advantage of the availability of additional IDA and grant resources to address financing gaps in the Government’s program. With the approval of the AF (2017) the World Bank’s total contribution rose to US$US350 million with other partners funding about US$1.3 billion during the EFY 2009-2022 (Annex 3). During the initial four years (2013-2017) there was sound progress with all PDO/DLIs targets surpassed. This included significant progress in expanding access to antenatal care, skilled attendance at birth, and contraceptive use. The changes introduced with the AF appropriately focused on increasing targets, setting more ambitious objectives (e.g., shifting to four antenatal care visits which is the gold standard; and focusing on contraceptive use by women in rural areas); and adding a focus on nutrition, and health financing, including financial protection for rural households. 18. Changes undertaken during the restructurings were necessitated by some implementation challenges experienced primarily due to the deteriorating security situation following the approval of the AF, and later by the COVID-19 pandemic which hindered the Government’s ability to deliver essential health services and to conduct surveys. According to an analysis of the impact of COVID-19 on essential health services conducted by the World Bank, Ethiopia experienced moderate disruptions during the pandemic compared to pre-pandemic trends with large and persistent subnational disparities. The largest service declines occurred in April 2020 (e.g., outpatient consultations: 25 percent; antenatal care 1 visit: 16 percent; pentavalent 3 vaccine administration: 8 percent) with hospitals (24 percent) and health centers (14 percent) still reporting lower than expected service volumes in 2021.9 Delays also stemmed from difficulties in carrying out some fiduciary measures and managing the expanded number of activities. 19. The original Theory of Change remained highly relevant, with the Program continuing to support improvements in the delivery and use of a comprehensive package of maternal and child services. The expanded scope of activities and the scaled DLIs remained well aligned with the PDO. The increased focus on nutrition with the introduction of two PDOs/DLIs was also consistent with the Theory of Change, as improvements in the nutritional status of women and children are an integral part of a comprehensive package of services and contribute to the long-term goals of reducing maternal, infant and child mortality. The addition of two IOs/DLIs on financial protection was consistent with growing recognition of the importance of expanding the Government’s Community Based Health Insurance (CBHI) scheme to lower out-of-pocket spending, especially for rural households.10 The new DLIs and additional activities fit in generally well with the Program’s three priority areas. 9Disruptions throughout 2021 may be underestimated due to lower completeness rates in many health bureaus. 10Out-of-pocket spending as a share of total health spending, which oscillated between 52 percent (1996) and 38 percent (2008) remained high, representing about a third of total health spending. Page 14 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) II. OUTCOME A. RELEVANCE Relevance of PDO 20. The relevance of the PDO is rated High. The PforR was fully aligned with the World Bank Group’s Country Partnership Strategy (CPS) for Ethiopia (2012–2017) and remains relevant and aligned with the current CPF for Ethiopia (FY18-22). The 2012-17 CPS was strongly anchored within the Government’s GTP, built on achievements, and supported key areas where there was strong government ownership and commitment. The operation was fully aligned with pillar two of the CPS which aimed to enhance resilience and reduce vulnerabilities through improved delivery of social services, including access to health services. The operation was also consistent with the governance agenda of the CPS which focused on improving public service performance management; enhancing citizen participation; and strengthening transparency and accountability in public financial management and procurement. The PforR operation remains relevant to the current CPS for FY18-22, which is aligned to the Government’s GTP II and to the second Health Sector Transformation Plan (HSTP II), builds on the excellent progress in improving social services, and mobilizing additional health resources, with the overall goal of attaining lower middle-income status by 2025. The CPF for FY18-22 addresses two overarching challenges, the need for sustainable financing for growth and feedback mechanisms for citizen engagement. The CPF program focuses, inter alia, on providing more equitable access to quality services at the district level (woredas), building resilience and inclusiveness (including gender equality), and supporting institutional accountability, with the PforR operation contributing to these overriding goals. Two of the CPF targets (i.e., boosting the CPR for rural women; reducing childhood stunting) were aligned with areas of focus of the health Program. Relevance of DLIs 21. Relevance of the DLIs is rated High. The DLIs were well aligned with the PDO with each contributing to the three priority areas of focus (i.e., accelerating progress towards maternal health MDG; sustaining gain in the health MDG; strengthening health systems). The DLIs were also aligned with the Government’s Monitoring and Evaluation Plan of the HSTP. The PDO/DLIs were highly relevant and fully aligned with global evidence and practices for measuring progress towards the maternal and child health MDGs. The DLIs on antenatal care, skilled deliveries, contraceptive use, childhood immunization, and nutrition were highly appropriate as they represent key components of a comprehensive package of maternal and child health services with Ethiopia lagging regional averages for several key maternal health indicators. Moreover, the DLIs were mutually reinforcing recognizing that maternal health contributed to improved child health, as enhanced antenatal care, improved nutrition, skilled attendance at birth and greater birth spacing improve infant and child survival. All PDO/DLIs were well articulated with a clear source of data for verifying results. The PDO/DLIs were progressively revised as they were attained to set more ambitious targets/objectives and to focus greater attention on equity and quality. The IO/DLIs were also relevant to the PDO, focusing initially on improving availability of data, and strengthening transparency in the procurement process, and subsequently with the AF, adding a wider range of activities and objectives (e.g., monitoring drug availability; developing mechanisms for community engagement; improving quality of adolescent services; and improving the functionality of the community health insurance scheme).11 The number of IO/DLIs rose substantially with the AF, as 11The functionality of the CBHI scheme was defined as “Woredas that established a CBHI scheme, registered households, entered a contract with health facilities, and household started accessing health services based on their membership”. While this indicator could have been more clearly articulated, it was meant to serve as a proxy for household use of services based on membership. Page 15 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) the scope of the PforR was broadened and financing nearly doubled (i.e., from US$120 to US$230 million), generating some implementation challenges in an increasingly complex country context. Rating of Overall Relevance 22. The overall relevance of the operation is rated as High. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective or Outcome 23. The Program contributed to improving the delivery and use of a comprehensive package of maternal and child health services with virtually all PDOs/DLIs targets surpassed by 2019 (as discussed in detail in Annex 6), with Ethiopia continuing to make progress in reducing maternal, infant and under-five mortality.12 As depicted in Figure 1, the PforR included a comprehensive package of evidence-based interventions which are key to enhanced maternal and child health. The next section reviews progress in improving the delivery and use of these maternal and child health services. Figure 1: Key Elements of Maternal and Child Health Antenatal Care Skilled birth Nutrition attendance KEY ELEMENTS OF MATERNAL AND CHILD HEALTH Child Family Immunization planning Vitamin A and Iron & Folic Acid Supplementation 24. There was considerable progress on all PDO/DLI indicators for tracking improvements in the delivery and use of maternal health services with women from the lower socio-economic groups and those from rural areas benefitting the most. Deliveries attended by a skilled provider increased five-fold while other indicators (i.e., antenatal care, contraceptive prevalence, IFA supplementation) improved by 25-50 percent, as discussed in Box 1. 12Assessment of progress on some PDOs was done comparing the baseline estimates in the Results Framework from the original PAD to the 2019 DHS results and for indicators added with the AF comparing data from the 2016 and 2019 DHS. Page 16 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) 25. Complementary data from the 2016 Ethiopia Emergency Obstetric and Newborn Care (EmONC) Assessment are broadly consistent with these trends, highlighting substantial improvements in the availability of EmONC services but persistent gaps.13 The rise in the utilization of these critical services stemmed from a combination of strategies, namely: (i) adopting a pro-poor focus by progressively targeting women from lower socio-economic groups and underserved regions (PforR DLIs); (ii) improving the availability of critical commodities and drugs through the SDGPF to enable states to provide these services; (iii) conducting maternal death assessments to identify root causes and adopt remedial actions (e.g., establishing maternity waiting homes to allow high-risk women to arrive early and stay following delivery); (iv) developing and implementing postnatal care service directive to improve quality and uptake (PforR DLI); (v) using the Maternal and Perinatal Death Surveillance and Response system to capture underlying causes and skill gaps and providing clinical mentorship to improve knowledge, attitudes, and skills; and (vi) using community health extension workers to conduct outreach activities. The improved uptake of quality maternal health services contributed to a 40 percent reduction in maternal mortality (i.e., from 676 (2011) to 412 (2016) per 100,000 live births). Box 1: Key Achievements on Maternal Health - Progress on expanding deliveries attended by skilled providers was outstanding, rising from a meager 10 percent (2011) to 50 percent (2019), surpassing the end of Program target of 40 percent, and approaching the regional average of 61 percent (Figure 2). -Women from the lower socio-economic groups experienced the most rapid rise in skilled attendance at birth (13-15-fold increase) even though their overall utilization rate remained modest. -Deliveries attended by skilled birth provider for the bottom three performing regions also rose progressively from 19 percent (2016) to 33.4 percent (2019), surpassing the end of Program target of 28 percent. -Progress on delivery and use of antenatal care was also strong with the percentage of pregnant women receiving at least one antenatal care visit rising from 43 percent (2011) to nearly 63 percent (2016), substantially surpassing the end of Program target of 56 percent, and reaching about 75 percent by 2019 (Figure 3). -Pregnant women receiving at least four antenatal care visits also rose, from 32 percent (2016) to 43 percent (2019), exceeding the end of Program target of 38 percent. -Exceptional improvements in contraceptive use were noted with the CPR increasing from about 27.0 percent (2011) to nearly 36 percent (2016), against the end of Program target of 35 percent; and rising to over 41 percent (2019), surpassing the regional average of about 33 percent (2019). -The CPR in rural areas increased from 32 percent (2016) to close to 37.5 percent (2019) meeting the end of Program target of 38 percent. -The percentage of pregnant women taking iron and folic acid (IFA) supplements rose steeply from 42.1 percent (2016) to 60 percent (2019), exceeding the end of Program target of 54.1 percent. 13 Key findings from the 2016 EmONC include: (i) availability of EmONC facilities rose from 11 to 40 percent; (i) readiness to provide services improved with substantial investments in infrastructure, human resources and drugs, but the overall score remained at 46 percent with only 17 percent of facilities having functional ambulances and with persistent shortages in staffing; (ii) institutional deliveries reached 66 percent (higher than DHS estimate) but only 18 percent of health facilities could provide obstetric care with complications referred to hospitals and with only 53 percent having a maternity waiting home to manage high-risk births; and (iv) financial barriers declined with the introduction of free maternal and child health services and roll out of the CBHI scheme. Page 17 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Figure 2: Deliveries attended by skilled birth Figure 3: Pregnant women receiving at least one provider, 2005-2019 antenatal care visit, 2005-2019 SSA: 82% 60 SSA: 80 61% 70 50 60 SSA: 74% 40 50 SSA: 30 51.4% 40 20 30 20 10 10 0 0 2005 2011 2014 2016 2019 2005 2011 2014 2016 2019 Note: The figures above include 2005 data to provide a longer-term perspective. 26. There was also solid progress on the PDO/IO/DLI indicators related to sustaining gains in child health and nutrition. There were notable improvements in strengthening the delivery of key child services with all targets met or surpassed (Box 2). With the increased focus on addressing inequities in child health services and intensifying the delivery of key interventions, progress was made in underserved regions. The pandemic and conflict limited what could be done with adaptations introduced in the final years of the Program. From 2020 onwards, COVID-19 social distancing measures and rising insecurity in some areas inhibited GMP provision and monitoring.14 During the pandemic, two DLIs were introduced and achieved by Program completion, namely: (i) conducting immunization campaigns at camps for refugees and Internally Displaced Persons (IDPs) for missed vaccinations and targeted catch up campaigns in four priority regions; and (ii) conducting catch-up campaigns to increase the uptake of essential health and nutrition services, including vitamin A, deworming and nutrition screening in five security-constrained regions. While the DLIs related to child health were limited in number and tended to focus more on intermediate results, they contributed to improving child health. Ethiopia, which was on track to attain the child health MDG at the inception of the Program, continued to make steady progress, with the Under-five Mortality Rate declining more steeply from 88 (2011) to 59 (2019) per 1,000 live births and the Infant Mortality Rate declining more modestly, from 59 (2011) to 47 (2019) per 1,000 live births.15 As seen in other countries, rates of change slowed down in Ethiopia in contrast to the previous period (2005-2011) which experienced steeper drops in infant and child mortality (Figure 3), as it becomes increasingly difficult to make improvements at the lower rates. 14 The MOH used alternative approaches to promote optimal child growth and development to maintain previous gains during the crisis, including a national Social and Behavior Change Communication campaign to promote safe Infant and Young Child Feeding behaviors during the COVID-19 pandemic and integration of nutrition screening into house-to-house COVID awareness campaigns and vaccination catch-up campaigns. 15 These findings are consistent with the results from the 2016 EmNOC survey which found that neonatal mortality remained high with many newborns experiencing difficulties due to breathing problems and the impact of preterm and low birthweight delivery. Page 18 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Box 2: Key Achievements on Child Health - The percent of children 12-23 months immunized with Pentavalent 3 vaccine increased from 65.7 (2011) to 81.3 (2022), surpassing the end of Program target of 76.7 percent. - The percentage of woredas in emerging regions (i.e., Somali, Afar, Benishangul-Gumuz, and Gambella), which are characterized by predominantly pastoralist populations living in extreme poverty and having limited access to health services, transitioning from Enhanced Outreach Services (conducted twice yearly) to Community Health Days (conducted quarterly), rose from zero (2016) to 89 percent (2022), surpassing the end of Program target of 50 percent. These campaign-style events, designed to increase coverage of vitamin A supplementation and one or more child health services, have proven cost effective in delivering services in Ethiopia to a dispersed, predominantly rural population with potential economies of scale when services are bundled. 16 - The percent of woredas in non-emerging regions (i.e., Amhara, Oromia, Harari, Southern Nations Nationalities and People’s Region, Tigray) delivering vitamin A supplements to children increased from 48 percent (2016) to 100 percent (2020), surpassing the end of Program target of 80 percent. - There was also an increase in the percentage of children 0-23 months participating in Growth Monitoring and Promotion (GMP), which improved significantly from 38 percent at baseline (2016) to 54 percent at midline (2019). Figure 3: Trends in Infant and Under-five Mortality, 2005-2019 140.0 123.0 120.0 100.0 88.0 77.0 80.0 67.0 59.0 59.0 60.0 48.0 47.0 40.0 20.0 0.0 2005 2011 2016 2019 Infant Mortality Rate Under five Mortality Rate Note: The figure includes 2005 data to provide a longer-term perspective. 16 Fiedler, J. and Chuko, T., The Cost of Child Health Days: a case study of Ethiopia’s Enhanced Outreach Strategy, April 2008. Page 19 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Institutional Strengthening 27. Institutional strengthening was an integral part of the Program from its inception in line with the PforR approach and contributed to the attainment of the PDO. The initial focus of the health systems strengthening area was on three activities that complemented support provided by other donors, namely, enhancing performance by introducing a balanced score card approach; strengthening monitoring and evaluation through improved routine data and regular facility assessments; and enhancing transparency of a key government institution (PFSA) responsible for procurement of drugs and medical products. With the approval of the AF and the addition of an IPF component substantial additional resources (US$21 million) were explicitly devoted to institutional strengthening in two critical areas for enhanced management of maternal and child health services: (i) building a CRVS system; and (ii) strengthening fiduciary management, health care financing, and HMIS. Overall, progress on institutional strengthening was sound but with some activities experiencing delays. 28. The Program had an important impact on strengthening the capacity of Government institutions. There were notable improvements in both routine data and survey results which assisted policymakers to track progress on maternal and child health and take remedial action, triangulating data from different sources. Health centers reporting HMIS data in time rose from 50 percent (2011) to 84 percent (2022), nearly attaining the end of Program target of 86 percent. The development and implementation of Services Availability and Readiness Assessment (SARA) surveys assisted policymakers to assess the readiness of facilities to provide quality services by generating a comprehensive set of tracer indicators.17 The DLI related to developing and implementing a postnatal care services directive in all PHCs with a monitoring report produced and verified by implementing agencies and an assessment of the effectiveness of the directive conducted was met, which contributed to the delivery of a comprehensive package of maternal services. The percentage of PHC facilities having all drugs from the MOH list of essential drugs available also improved, from 42 percent (2012) to 47 percent (2019) with the end of Program target met. The percentage of woredas with functional CBHI schemes (i.e., CBHI schemes with registered households, entered a contract with health facilities, and household started accessing health services based on their membership) rose from 23 percent (2017) to 65 percent (2022), surpassing the end of Program target of 53 percent with CBHI scheme reviews conducted every two years. While there was an increase in uptake of CBHI, government counterparts acknowledge that more needs to be done to expand and sustain enrollment levels and are working with the World Bank to address these issues. 18 29. Efforts to strengthen institutional accountability and promote community engagement were mixed with the development and implementation of the Health Sector Community Card completed but with delays in devising and implementing a mechanism for documenting consultations when community/private land is used for construction of facilities.19 Likewise, progress in improving the quality of adolescent health services could not be fully verified and was eventually replaced with another indicator which was attained. The fiduciary reforms of the PFSA proved challenging to carry out fully but represented important progress over the situation at Program inception, with the 17 These facility assessments aimed to detect and measure progress in health system strengthening; plan and monitor scale-up of interventions; and generate evidence to feed into country annual health reviews. The facility surveys were conducted regularly in the earlier phase of the Program but proved difficult to carry out during the COVID-19 pandemic. 18 Improved access to CBHI was one of the key strategies for lowering out of pocket spending. The MOH piloted the CBHI in several woredas in 2011/2012 and supported the regions to expand the scheme in subsequent years. While there was a rise in household enrollment in the CBHI, the evidence is mixed on whether enrollment rates were sustained with differences between administrative and survey data. Information from Ethiopian Health Insurance Agency, verified by non-governmental implementing entities, found that enrollment reached 66 percent (2021) while a recent meta review found that CBHI enrollment dropped from 56 percent (2016) to 37 percent (2020). 19 By Program completion a modified mechanism was put in place to reflect the security situation in the country with the results verified and funds disbursed. The mechanism for documenting consultations for land use and acquisition was implemented in 70 percent of woredas (excluding those affected by insecurity). Page 20 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) institution striving to become a premiere operation, benchmarking its performance to similar institutions in neighboring countries, and benefitting from the establishment of a website, enhanced dissemination of procurement related information, and introduction of performance indicators for tracking progress in improving transparency; nevertheless, not all DLIs could be met in a timely fashion, such as clearing up the backlog of all audits and installing a new software system which was dropped. Progress on the IPF components is summarized below. 30. Civil Registration and Vital Statistics: In the years following the approval of the AF, there was a renewed focus on strengthening the CRVS system with enhanced awareness and greater commitment to bolstering registration of vital statistics with the World Bank providing critical funding and technical support. A national comprehensive assessment was conducted; advocacy activities were carried out at the federal, regional, woreda, and kebele level, including for cultural and religious leaders, to create public awareness on the importance of the CRVS system; staff training was conducted; and equipment was provided. Memoranda of Understanding were signed between religious leaders and administrators of the CRVS at both the national and regional levels to strengthen vital events registration system. A 5-year CRVS System Improvement Strategy and Costed Action Plan (2022-2026) was developed. Even though the operationalization of the CRVS system was adversely affected by both the COVID-19 pandemic and the political instability in the country INVEA continued to make excellent progress in supporting the registration of vital events.20 In total, registration of births rose from about 12 percent (2017) to over 28 percent (2022), and deaths increased from about 7 percent to nearly 17 percent in the same period, in comparison to the targets of 40 and 10 percent, respectively. As noted by national stakeholders during the August 2022 ICR mission, on balance there was important progress made in bolstering the CRVS system in Ethiopia as a result of IDA financial and technical support, with a gradual rise in community awareness; improved quality and completeness of information; greater collaboration with other sectors from the federal to the kebele level, and enhanced uptake of services. 31. Technical Assistance and Capacity Building to Support National Nutrition Program: The nutrition activities which were managed by UNICEF involved the design and development of several policy instruments and production of a series of assessments, situation analyses and surveys, that allowed the MOH to strengthen its capacity to manage the national program; identify challenges and shortcomings; and improve performance. While there were some minor delays in firming up the institutional arrangements for this component, the Government agreed to contract UNICEF for this purpose which proved to be a fruitful partnership. Significant progress has been made in strengthening multi-sectoral coordination and conducting operational research to suggest programmatic improvements, with most of the technical support provided during the COVID-19 pandemic, skillfully modifying the working approach to continue to safely carry out planned activities and adapt the IPF programming to emerging needs. Technical assistance for multi-sectoral coordination included deployment of technical specialists to regional levels, an analysis of current experiences and practices, recommendations on improving coordination, and elaboration of a policy for strengthening multi-sectoral coordination at the federal, regional, zonal, woreda and kebele levels. Regional multi-sectoral nutrition plans and budgets were developed; and a Unified Nutrition Information System for Ethiopia was piloted with nutrition-specific and nutrition-sensitive indicators from six sector ministries and was fully integrated into the District Health Information System (DHIS-2) in 16 woredas with additional funding mobilized for expansion to all 50 woredas. 32. Formative research on nutrition-specific services was conducted jointly with Ethiopian researchers to generate recommendations on improving three critical interventions which were areas of focus of the PforR-- IFA supplementation for pregnant women, Vitamin A supplementation (VAS) for children 6-59 months, and GMP for children under two years. The operational research was successfully completed and proved instrumental in 20As a result of the security situation some vital events registration offices were damaged, and motorcycles were looted (Afar, Amhara, Benishangul-Gumuz and Tigray) which impeded registration in those regions. Page 21 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) generating information for enhanced decision making by assessing the quality of implementation of these interventions in different regions (agrarian, pastoral, urban); identifying barriers and opportunities; and proposing strategies for improving coverage, quality, and uptake, using a science of implementation approach with extensive consultations with a broad range of stakeholders and key informants. The final recommendations proved useful to policymakers as they were practical and operational, focusing on improving targeting to reach the most marginalized women and children; strengthening coordination with other ministries; improving delivery strategies and combining interventions to reap synergies. All activities under this component were fully completed by Program completion and greatly appreciated by national stakeholders. 33. Technical assistance and capacity building to strengthen health financing and fiduciary systems in the health sector. By Program completion technical assistance was provided in various areas (i.e., health care financing, resource mobilization, public-private partnerships, financial management) with the allocated amount (US$1 million) fully disbursed. In addition, the World Bank provided technical assistance on health financing supported through GFF (US$1.5 million) and the Gates Foundation (US$3 million) trust funds which included a wide range of activities that generated valuable recommendations for improving health financing. This included several public expenditure reviews; a study of the political economy of social health insurance; a health financing policy guide; multi-stakeholder national consultations; and creation of a pool of national experts on health financing.21 The main health financing reforms stemming from the analytical work were endorsed by government at a 2020 retreat of the SDGPF and included: incentivizing national and regional authorities to expand domestic resource utilization and mobilization through a matching fund mechanism which is one of the key innovations of the follow-up Ethiopia Health PforR (Hybrid) Strengthening Primary Health Care Services (P175167) approved by the Board on December 13, 2022; exploring results-based strategies with a pilot underway; expanding program-based budgeting to ensure resources are allocated to priority programs (which was linked to the Human Capital Project); continuing to strengthen public finance management to ensure that financing of SDGPF activities were well tracked and utilized with notable progress in strengthening the fiduciary capacity of the SDGPF through new tools and procedures for enhanced tracking and reporting; and ensuring that the CBHI scheme had a pro-poor focus. The health financing policy dialogue associated with the 2017 AF helped inform the design of PforR operation with DLIs adapted progressively to incentivize progress. Given the importance of demand-side barriers to the utilization of maternal and child health services, the focus on health financing with the AF complemented well the original design of the PforR that focused on service delivery. 34. Analytical work supported by other partners provided complementary support, such as the assessment of the CBHI scheme program which generated evidence of the impact of the CBHI on increasing health service utilization and decreasing the incidence of catastrophic health spending.22 The assessment found that CBHI household members are poorer compared to non-members, with membership resulting in positive effects on health service utilization, financial risk protection and women’s empowerment. Treatment seeking during illness was only slightly higher among CBHI members (70 percent) relative to non-members from CBHI woredas (67 percent), with the small difference possibly explained by the introduction of the free maternal and child health service policy. CBHI membership resulted in a 28-43 percent reduction in annual out-of-pocket payments compared to non-member households with CBHI member households significantly less likely to incur catastrophic health spending. 21 A notable example is the 2018 Ethiopia Public Health Expenditures Review which found, inter alia, that: (i) total health expenditures remained low in relation to WHO recommendations (US$32 versus US$86) with continued reliance on out of pocket spending; (ii) cost remained an impediment for households with drugs representing one of the largest cost drivers; (iii) donor financing was fragmented and unpredictable; and (iv) limited use of criteria (i.e., population, need) for resource allocation. 22 The Impact of CBHI on Health Service Utilization, Out-of-Pocket Health Expenditure, Women’s Empowerment, a nd Health Equity in Ethiopia (January 2021). Page 22 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Women participating in focus group discussions reported that CBHI membership had an empowering effect on health care Box 1 use (Box 1). The study highlighted the positive effects on It is not because we didn’t need to be healthy that health facilities (i.e., incentivized health personnel to provide we didn’t go to the health facility; rather, it was due to a lack of money. quality services and enhanced financial accountability) and the ongoing challenges (e.g., workloads stemming from We used to get medical services by asking our increased demand; physical inputs not keeping pace with husbands, but now we are getting treatment rising demand; modest premiums compared to services using health insurance. demanded). During the August 2022 ICR mission, key stakeholders at the Ethiopian Health Insurance Agency (EHIA) After I became a CBHI member, I avoided my fear, noted the progress made but acknowledged the continuing and I could get all health services, including family challenges in rolling out the scheme in a strong federal health planning. system (e.g., ensuring appropriate risk pooling and introducing provider/purchaser split mechanisms). Participants in Focus Group Discussions Rating of Overall Efficacy 35. The overall efficacy of the Ethiopia PforR Program is rated High. This reflects the remarkable progress made in improving use and delivery of maternal and child health services despite some shortcomings on the more process- oriented indicators. All PDO/DLI indicators, original and revised, have been reached and virtually all exceeded their end-targets by 2019, as discussed above with details provided in Annex 6. Progress on maternal health was especially strong with Ethiopia catching up to regional averages and experiencing a large drop in maternal mortality. Likewise, most IO/DLIs were achieved or surpassed and those related to institutional strengthening were almost fully achieved. Some indicators were revised to reflect what was feasible in an increasingly complex country context.23 On balance, the scale of the achievements was considerable with the Program contributing to rectifying socio-economic and geographic inequities, putting Ethiopia on a positive trajectory towards reaching the maternal MDG and sustaining the child health MDG. C. JUSTIFICATION OF OVERALL OUTCOME RATING 36. The overall outcome is rated as Satisfactory based on the Program’s continued High relevance, and High efficacy rating. The Satisfactory rating is based on the solid performance described above with only minor shortcomings during the last years of the Program, stemming primarily from exogenous shocks (i.e., mainly the COVID-19 pandemic; deteriorating security) which resulted in service delivery disruptions and inability to conduct national surveys. With the containment of the pandemic, restoration of security, and the recent approval by the World Bank Board of the follow-on operation, Ethiopian authorities will be able to continue addressing remaining gaps and disparities in service coverage and financial protection, building and leveraging on the achievements supported under the Program. 23For example, GMP activities could not be implemented nationwide during the COVID-19 pandemic with funds redirected to conduct immunization catch up campaigns in IDPs/refugee camps and to enhance the uptake of essential health and nutrition services in security constrained regions; and the MOH using alternative approaches to promote optimal growth and development to maintain gains (footnote 16). Page 23 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) D. OTHER OUTCOMES AND IMPACTS Gender 37. The original Program targeted women and children and had an inherent focus on gender issues. The interventions benefitted women from the lower socio-economic groups and the more disadvantaged regions who face multiple constraints to female agency, including lack of economic autonomy, child marriage, low levels of literacy, and high rates of domestic violence and obstetric fistula. By supporting improved access to antenatal care and skilled attendance at birth the Program contributed to addressing the risk of obstetric fistula which affects women widely in Ethiopia, and stems from prolonged and/or obstructed labor and low rates of skilled care during pregnancy and delivery. With the approval of the AF, the Program had an even more explicit focus on gender issues in line with both Government and corporate priorities. The Program continued to focus on health concerns that affect women and children, adding new areas, including nutrition, adolescent health, and Gender Based Violence (GBV) with the design and implementation of a GBV strategy. The heightened attention to maternal nutrition was important to improving birth outcomes and minimizing the risk of intra-generational transmission of malnutrition. The IPF component also brought more attention to gender issues through technical assistance and capacity building. Poverty Reduction and Shared Prosperity 38. The Program had an indirect impact on poverty reduction and shared prosperity by strengthening the provision of maternal and child services which tend to benefit primarily the lower socio-economic groups. By progressively targeting underserved regions and the most vulnerable women and children who face multiple impediments to accessing services the Program may have contributed to a reduction in socio-economic disparities. The improved availability of services combined with efforts to facilitate access to the CBHI scheme (which targeted mainly informal sector workers in rural areas), reduced the risk of impoverishment from high levels of out-of-pocket spending for disadvantaged households which enrolled in the scheme. The increased focus on the nutritional status of women and children with the AF may have contributed to enhancing female productivity and minimizing the risk of intergenerational transmission of malnutrition and poverty. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 39. The Program had a solid design with realistic objectives and targets. While the design was bold in its vision, it was simple and focused. The results areas were appropriate; the DLIs were clearly articulated with realistic but ambitious targets based on global evidence and national experience, and with clear verification protocols. The institutional strengthening DLIs complemented and supported the service utilization DLIs. The eight DLIs ensured simplicity in the design which facilitated implementation in the early years of the Program. The PDO outcome indicators made good use of data from nationally representative population-based surveys that were conducted regularly. The Program had a major focus on generating population-based data to facilitate performance monitoring and to enhance decision making with regular DHS conducted working in collaboration with the World Bank-funded social protection project team and the United States government as well as the design and roll out of national facility-based surveys (Service Provision Assessment, SPA) to monitor the availability and quality of services and overall readiness at all facilities in Ethiopia. 40. The task team took advantage of the favorable environment in country to pilot the PforR instrument. As this was a high visibility first PforR for the government and the HNP network, there was strong support within the World Bank to ensure its success and to generate lessons for scalability. The Program leveraged the government’s high-level Page 24 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) commitment to a results-focused approach mobilizing key stakeholders at the ministries of health and finance; built on innovative initiatives and reforms in the health sector (i.e., such as the health extension worker program that delivered a comprehensive package of health services at the household level to raise awareness and stimulate demand); and relied on existing institutional arrangements and country systems. Strong ownership and active engagement of multiple stakeholders and development partners during the preparation phase was critical to enhancing readiness for implementation. The use of the MDGPF allowed for targeted support for RMNCAH+N (a relatively neglected and underfunded area), and enhanced harmonization of donor assistance by skillfully using the PforR instrument. The use of country platforms enhanced consultations and consensus building around priority areas and later facilitated identification of bottlenecks and remedial actions. Within the World Bank, the task team worked closely with OPCS to adapt the PforR instrument to the complex Ethiopian context, putting in place an operation which was fit for purpose operationally and institutionally. B. KEY FACTORS DURING IMPLEMENTATION Factors subject to the control of government and/or implementing entities 41. The government demonstrated high-level commitment to the Program and to the MDG targets with Ethiopia being the first country to sign the IHP+ compact. The MOH team was heavily engaged in the design and implementation of the Program, demonstrating high-level ownership. Senior management at both the MOH and Ministry of Finance provided leadership and sustained support. The government team established strong platforms for consultations with broad-based participation of a wide range of national stakeholders and development partners. The government’s commitment and ownership were sustained over time as reflected in growing government contributions to the health sector.24 With the enhanced availability of survey data, the government team refined their strategies to focus on emerging issues. For example, during the Annual Health Sector Review Meeting held in October 2015 there was a comprehensive review of annual progress which coincided with the conclusion of 20 years of implementation of Health Sector Development Plans. As a result, the HSTP 2015-2020 was launched to strengthen the country’s path towards Universal Health Coverage. With the recently released results from the 2014 SPA revealing major shortcomings in quality, equity and access the government team prioritized these issues as part of its transformational agenda to be supported under the Program. 42. While there was strong overall commitment and ownership, the implementing entities encountered some difficulties and delays in effectively implementing several activities, particularly in the fiduciary and social development areas. A notable example relates to the reforms to strengthen transparency in the procurement process at the PFSA with some activities successfully completed (i.e., establishing a website; introducing performance indicators; disclosing procurement documents) while others encountered difficulties and some were only partially completed by Program closure (i.e., cleaning up the backlog of audits; automating fiduciary systems). Another example is the measure to devise and implement a mechanism for documenting consultations when communal/private land is used for construction of health facilities which also encountered substantial delays. Despite these difficulties and delays, the government ensured that all Program funds were effectively used by Program completion and worked with the World 24There was a steady rise in government health spending from US$ billion .3 (2011) to US$ billion 1.2 (2020) with the share of total health spending rising from 16 to 32 percent. While per capita health spending reached about US$34 in 2019/2020 (excluding COVID-19 related spending) there was a consensus that more needs to be done to expand government spending on health and to further reduce out-of-pocket expenditures which still represented about 30 percent of total spending in contrast to WHO targets of 15-20 percent (Ethiopia National Health Accounts, 2019/2020). Page 25 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Bank team to modify and adapt some DLIs to what proved feasible in terms of capacity and country context with no major reversals in Program goals and with most activities staying on track, despite the ongoing pandemic and the security situation in Ethiopia. Factors subject to the control of the World Bank 43. The World Bank team was proactive and responsive during implementation, continuing to make optimal use of the PforR instrument. The World Bank team played an increasingly important role in the sector, working collaboratively with other partners to address problems and speak with one voice, and ensured that the Ethiopia Sector Wide Approach remained strong, robust, and resilient over time. The World Bank team ably processed the AF, mobilizing sizable additional grant financing, and ensured the Program stayed on course by providing hands on support and adapting the design to the rapidly evolving context in the country. While the team worked with the government to effectively set more ambitious targets and adopt more ambitious objectives, and to add several new critical areas (i.e., nutrition, health financing) the AF also taxed the capacity of the national institutions in an increasingly difficult country and global context. Factors outside control of government and/or implementing entities 44. The main exogenous factors that impeded implementation following approval of the AF was the deteriorating security situation in Ethiopia and the COVID-19 pandemic. These shocks resulted in difficulties conducting some Program activities, including delivery of services, and delays in carrying out surveys. While the entire country was affected by the pandemic some regions were also adversely impacted by the deteriorating security situation with a few isolated incidents in 2016 (i.e., damage to some health facilities and destroyed ambulances in the affected regions) and more pervasive effects in the last several years. By June 2018, the World Bank team downgraded the PDO rating from Satisfactory to Moderately Satisfactory for the first time due to: (i) political instability which was affecting performance at the central and deconcentrated levels; (ii) limited results officially submitted for notification; and (iii) delays in completing key activities for verification, such as the 2019 mini-DHS that was delayed by nearly one year. By mid-2019, the security situation appears to have worsened with World Bank mission reporting concerns over instability in significant parts of the country, leading to internal displacement as well as looting of health facilities with some healthcare workers unable to fully perform their duties. With the onset of the COVID-19 pandemic, there were disruptions to essential health services, which fueled additional delays in implementation and necessitated modifications to some DLIs, and several extensions of the Program closing date. The modifications involved adapting the DLIs to what was practically feasible, as discussed above. Despite these external shocks, all parties persevered to ensure the 2019 DHS was conducted with the survey used to verify results and disburse funds. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION M&E Design 45. The M&E design was solid. The Program had a major focus on generating and/or utilizing population-based data for enhanced monitoring and evaluation, building on ongoing efforts by the World Bank and other development partners. The PDOs/IOs/DLIs were in line with international good practice and aligned to Government’s M&E plan. Likewise, the target setting process was rigorous, drawing on both international evidence and national experience, setting goals that were ambitious yet realistic. While the service delivery indicators were fully in line with international good practice, some of the targets for the intermediate indicators and dated covenants may have been too ambitious, particularly the one related to improving the transparency of procurement processes at the PFSA (i.e., cleaning up the Page 26 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) audit backlog from a decade earlier).25 There was also strong alignment with indicators in key government documents (i.e., GTP, HSDP) as well as the World Bank’s country assistance strategies for Ethiopia. One shortcoming noted was the articulation of the DLI on financial protection (‘functionality of the CBHI scheme’) which could have been more clearly stated. M&E Implementation 46. M&E Implementation was generally well conducted. Regular DHS were carried out and funded through various sources; annual rapid health facility readiness surveys were integrated with the SPA+ surveys; and a balanced score card was planned but had to be dropped as the scheme was replicated nationwide. There were other surveys and studies conducted during the life of the Program which provided policymakers valuable information for enhanced decision making, such as the nutrition reports which generated national or regional data on a wide range of indicators and recommendations on improving critical interventions supported by the Program (i.e., IFA, VAS, GMP). 47. During the initial phase of implementation Program monitoring went smoothly with consistent data availability, strong compliance with DLIs, and a solid disbursement record. During the phase following the approval of the AF, M&E implementation experienced more difficulties and delays, as the Program covered a wider range of areas and as the COVID-19 pandemic combined with security concerns made it difficult to conduct population-based surveys. On a general note, the MOH/MDGPF/SDGPF produced regular reports to verify compliance with DLIs and enable the World Bank team to disburse funds. While most reports were produced in a timely manner for some activities there were frequent delays, especially on fiduciary issues and during the last few years. The World Bank team held regular consultations with government counterparts to discuss progress, identify key issues, and propose remedial actions to maintain M&E activities on track. The counterpart team demonstrated continued commitment to M&E tracking, working diligently with the World Bank team to propose amendments to the DLIs as the country context evolved, and to ensure all funds were disbursed by Program closing. M&E Utilization 48. The PforR Results Framework and Government Progress Reports as well as results from other nationally representative surveys were used routinely during Joint Health Sector Reviews and World Bank missions. The PforR provided a strong incentive to conduct regular surveys and to utilize the information generated to strengthen M&E reporting and improve performance. As noted throughout various mission Aide Memoires and in World Bank Implementation Support Reports, the data generated was utilized to: (i) take stock of progress on the DLIs and identify remedial actions and modifications; (ii) identify areas which required technical assistance; (iii) hold consultations between national and state authorities to identify strategic priorities and set targets; and (iv) re-orient health sector strategies towards key gaps (i.e., quality, equity, financial protection) and emerging issues (e.g., disruptions in service delivery). The evidence-based Joint Reviews also assisted partners to leverage the comparative advantage of each agency in mobilizing support for the jointly supported Program. National authorities worked with regional and local stakeholders on an annual basis to address emerging issues and bottlenecks. One shortcoming noted is that more could have been done to systematically share results from various surveys with authorities at decentralized levels. Justification of Overall Rating of Quality M&E 49. The overall rating of M&E quality is Substantial. The Program’s M&E system was sufficient to assess achievement of Program objectives, and to inform the direction of the Program. The M&E system was: (i) well designed with a clear 25Soon after the Program got off the ground there was a need to conduct a level 2 restructuring to extend the closing for the PFSA audits for FY2002 and FY2002 as the client was unable to comply in a timely manner (ISR#3). Page 27 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) set of DLIs; (ii) diligently implemented, including adapting it to the evolving context; and (iii) utilized to inform decision making about the overall direction of the Program and the continued relevance of the DLIs. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social Rating: Moderately Satisfactory 50. At appraisal, an Environment and Social Systems Assessment (ESSA) was carried out. The general ESSA finding was that the Program to be implemented through the MDGPF will generate several positive benefits including improved overall health status given the increased access to essential health services, better sanitary conditions, and enhanced availability of skilled health professionals in rural areas. The ESSA rated the overall risks as moderate, with main adverse possible environmental effects likely due to generation of medical waste and improper disposal of expired drugs and other supplies. An action plan was designed to address weaknesses and included measures to enhance capacity, establish infection prevention, set up patient safety committees, improve public and worker safety, and document outreach and actions focused on providing services to vulnerable persons. 51. Performance on E&S safeguards was generally strong. By 2020, there was encouraging progress in the implementation of two key measures in the PAP, requiring All Health facilities to establish and operate infection prevention and patient services committees (PAP #14); and availing appropriate temporary storage facilities for collection of hazardous waste till final disposal is done (PAP #15). However, the World Bank team noted that further improvements in environmental safeguards coordination were needed (ISR #20, February 2020). 52. With the approval of the AF there was a greater focus placed on social development issues, including the preparation of a strategy to tackle GBV and measures to be put in place to ensure broader consultation of the population when land is used for health facility construction. The World Bank provided support to develop a health sector GBV strategy that was translated with orientation and training provided to regional health bureau focal points. By contrast, the DLI on the rollout of mechanisms to document consultations with communities when private or communal land is acquired for health infrastructure development experienced protracted delays throughout the entire period but was eventually complied with. Fiduciary Rating: Moderately Satisfactory 53. The results on fiduciary compliance were moderately satisfactory by Program completion. World Bank missions noted overall satisfaction that funds were used for intended purposes and the government adhered to fiduciary requirements. The Program, which was on the government budget and reporting system, used qualified fiduciary personnel; submitted regular quarterly financial reports; produced internal audit reports; conducted annual financial audits which produced unqualified audit opinions with actions taken on audit findings; produced procurement audits; and submitted semiannual fraud and corruption reports. Although the level of advances under the Program were significant, with engagement of senior management, improvements were noted in terms of following up on these balances to ensure proper documentation. Most importantly, there was substantial progress in improving the transparency of procurement processes at the PFSA (PforR DLI) with a website established, greater disclosure of procurement documents, and introduction of indicators for monitoring progress. These actions were considered important to put in place more robust systems as the organization was expanding. Page 28 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) 54. While the basic fiduciary arrangements were in place, there were significant advances which needed follow-up and delays in complying with some DLIs and dated covenants, particularly with respect to catching up on the entity audit backlog of the PFSA. Moreover, while PFSA entity audits for EFY 2001 and 2002 were eventually produced they raised critical qualification points requiring a detailed action plan (ISR#7). The World Bank team temporarily downrated the rating for fiduciary compliance from MS to MU following the second MTR mission (February 2020 and ISR Seq. no 18), noting, inter alia: (i) limited progress in improving financial management and audit backlog processing at the PFSA; and (ii) delays in regions providing their 50 percent contribution to the large contract for procurement of ambulances, as the Program did not allow the procurement of High Value Contracts. 55. To recognize the concerted efforts made by the Government on fiduciary issues the World Bank team upgraded the rating as improvements were noted to address fiduciary weaknesses and eventually modified several DLIs. In early 2021 the fiduciary rating was reversed from MU to MS, noting substantial progress on key issues (ISR Seq. no 19). Nevertheless, the team noted that remaining fiduciary targets were missed, including automation of EPSA’s fiduciary functions, clearance of audit backlog, submission of procurement audit report for FY2011, and selection of an independent consultant to verify deliverables and strengthen internal audit functions at the MOH and PFSA. Towards the end of the Program, there was recognition that that there was not sufficient time to implement the FM module of the new system to all sites and hence the DLR was amended with the World Bank also agreeing to drop DLI9.3 (PFSA submission of backlog of audits reports and timely quality audit reports thereafter) moving resources to RMNCAH+N DLI/DLR (ISR #20, September 2021). As the Program was nearing completion in 2022, the World Bank team followed up closely with counterparts and ensured that outstanding fiduciary measures were completed.26 C. BANK PERFORMANCE Quality at Entry 56. The World Bank’s performance in ensuring quality at entry is rated Satisfactory. The World Bank team worked collaboratively with counterparts to establish a constructive policy dialogue and to design a path breaking PforR operation that supported the government’s strategic priorities as articulated in the HSDP while ensuring seamless coordination and harmonization with activities supported by other development partners. The task team benefitted from broad-based support from within the World Bank with OPCS providing advice and support to guide the preparation of the first PforR for the HNP network. A solid Quality Enhancement Review was conducted in July 2012 with the team incorporating the recommendations and lessons into the design. The task team was commended for a strong M&E design with clear DLIs that had an inherent pro-poor focus; the chair acknowledged the government’s strong commitment to improving data availability to take informed decisions on service delivery. The chair also noted that given the relatively modest amount of funding being provided to the Program, it will be important to ensure that the PforR instrument will add value with its results-focused approach, will support system efficiency improvements through capacity building, and will generate lessons for use of this instrument. 57. The PforR design involved robust assessments of key areas: technical (e.g., disparities; quality; human resources); macroeconomic (e.g., inflation, foreign reserves); fiduciary (i.e., procurement; internal audits); and environmental and social safeguards; and associated risks with identification of appropriate mitigation measures to ensure readiness for implementation. The preparation required considerable dialogue with other 26This included the submission of the fraud and corruption report for the first semi-annual of FY2014; the financial audit report for the year ended July 2021; and procurement audits for FY2020 and 2021, ISR#21, April 2022). Page 29 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) development partners who were providing larger amounts of money in the form of grants, with the task team skillfully persuading them that the World Bank Anti-Corruption Guidelines would take precedence and be used for all donor-supported activities through the MDGPF. The only minor shortcoming in the design was the exclusive focus on supply-side impediments, on the assumption that providers would be incentivized to address demand-side barriers and the health extension workers were already providing extensive support. Quality of Supervision 58. The World Bank’s performance in ensuring quality of supervision is rated Satisfactory. The World Bank conducted regular supervision missions and produced high quality, comprehensive, and timely Aide Memoires, and Implementation Status Reports. Government counterparts noted appreciation for the strong and frequent joint supervision missions which helped to identify strengths and weaknesses and propose remedial actions. Even though the Program was supervised by three different task teams for nearly a decade, there were seamless transitions between task team leaders (TTLs) with effective use of country based TTLs or co-TTLs. The teams established excellent working relationships with government counterparts and partners; provided sound and timely advice; and demonstrated flexibility and responsiveness when the country context necessitated changes to DLIs and implementation arrangements. The Program’s AF was prepared in a timely manner, documented well the initial achievements, and identified and addressed pending implementation challenges. The World Bank team did an excellent job in the final stages of implementation when the Program was adversely affected by multiple shocks, which required adaptations and amendments. The only slight shortcoming is that while the AF appropriately focused on scaling up targets, and setting more ambitious objectives, it also involved an expansion in the number of areas supported under the Program which may have stretched the government’s capacity in an increasingly difficult context. Justification of Overall Rating of Bank Performance 59. The overall rating of Bank Performance is Satisfactory. The World Bank team prepared an innovative PforR, successfully leveraged substantial grant financing, provided intensive technical support, and was responsive but rigorous in its approach to assessing compliance with DLIs. The PforR was well designed with most activities staying on track and putting Ethiopia on a positive trajectory to attain the maternal MDG and sustain the child health MDG. The World Bank team worked well with other development partners to align support for RMNCAH+N, a neglected and underfunded area. The PforR generated important lessons for replicability, as discussed below. During the August 2022 ICR mission, both Government counterparts and development partners expressed satisfaction with the World Bank support and collaboration. D. RISK TO DEVELOPMENT OUTCOME 60. The risk to development outcome is Substantial. On the positive side, the government has shown continuous and sustained commitment to the Program. Despite the difficult security situation and the COVID- 19 pandemic, the government maintained the Program largely on track without major reversals. Substantial progress has been made in improving maternal and child health services with virtually all PDO/DLI targets surpassed as validated by the 2019 DHS. Steady progress was also made in strengthening institutional capacity, promoting a results-focused culture, fostering a reform-oriented agenda, and creating strong platforms and Page 30 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) mechanisms for donor harmonization. The government’s strong stewardship of the health sector combined with strong country-led multi-stakeholder platforms will help mitigate risks.27 61. The main factors that run the risk of adversely affecting the impact of the Program are exogenous and stem from the continued insecurity and fragile peace agreement as well as the risk of another COVID-19 wave. The conflict in the Tigray, Amhara, and Afar regions is of particular concern as it has resulted in millions of IDPs and significant disruptions in essential health services. There are also persistent risks that are related to capacity constraints in the large and complex devolved health system. While Ethiopia has made remarkable progress, there is still a large and unfinished maternal and child health agenda which requires sustained and continued support with persistent disparities in access to quality health services across regions and in financial protection. The SDGPF has proven its utility but will require sustained donor support as financing gaps are pervasive. To this end, the recent approval by the Board of the follow up PforR will be instrumental in continuing to support government to maintain gains, accelerate progress, and lower the risk to development outcome. The inclusion of an investment project financing component in the next PforR will be key to providing targeted support for vulnerable groups and IDPs in conflict-affected regions. One of the challenges raised by partners during the August 2022 ICR mission is the importance of striking a balance between consolidating gains and addressing immediate humanitarian needs stemming from the ongoing conflict. V. LESSONS AND RECOMMENDATIONS 62. Ethiopia was a leader in using the PforR instrument with important lessons generated to inform future World Bank support to Ethiopia and to other countries. The main lessons can be summarized as follows: • The PforR instrument was fit for purpose both operationally and institutionally and proved its utility and adaptability over time in an increasingly difficult country context. Even though the World Bank financial contribution to the government’s HSDP was relatively modest, the PforR demonstrated value-added by reinforcing the sector’s results-oriented culture; aligning the PDO and DLIs to the government’s M&E plan; and harmonizing support through the SDGPF with all partners funding one program, one budget, and one common set of results.28 The PforR brought critical support for RMNCAH+N, a neglected and under-funded area of the HSDP, and empowered the MOH to manage these funds (i.e., in contrast to other resources which were donor-managed). While it is difficult to directly attribute results to the PforR, the targeted financial support through the SDGPF, combined with the pay for results approach, were critical drivers of the Program’s success. Partners came together to find solutions to common problems and to leverage each other’s comparative advantages rather than duplicate efforts. The DLIs provided an incentive to deliver key services to avoid the risk of losing funds. Building on the successful implementation of the initial Program, after nearly four years the World Bank team skillfully and rapidly processed substantial AF which proved important to provide continuity and maximize development impact. The progressive scale up of key DLIs that were attained and introduction of new DLIs that focused on emerging issues (quality, equity) was good practice, allowing Ethiopian authorities and the World Bank team to deepen the focus over time. Despite the increasingly 27 For example, several key informants in Ethiopia mentioned the Risk Assurance Plan that are jointly prepared between Government and partners to identify risks and remedial actions, ensuring continued strong implementation. 28 This was one of the key issues (i.e., fear that the World Bank contribution was too small to make a difference) raised by the chair of the QER in 2012 and it is encouraging that the experience has been overwhelmingly positive. Moreover, while the Bank funding represented roughly 21 percent of the SDGPF over the past decade, it rose from about 16 percent at Program inception to about roughly 60 percent by 2022. Page 31 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) difficult country context the MOH team persevered in addressing emerging issues (e.g., high number of IDPs which necessitated specialized support; disruptions in essential health services with a shift to outreach activities; staff turnover which required retraining) and ensuring that the Program remained broadly on track. • The PforR instrument has the potential to further incentivize results, provide more targeted support, and enhance performance of the SDGPF. First, it is important to ensure that there are clear accountability mechanisms for all key stakeholders, particularly state health authorities which were not directly incentivized through the DLIs (i.e., they received critical commodities and drugs but not financial resources). The proposed matching fund mechanism (i.e., whereby Regional Health Bureaus will match national funding from the SDGPF under the recently approved PforR) will help provide more direct incentives to regions to expand domestic mobilization and bolster accountability. Second, judicious use of DLI indicators and PAP measures is critical. While the main DLIs related to maternal and child health services were in line with international good practices, the other DLIs and PAP actions were numerous, sometimes too specific, and several ultimately proved a challenge to implement in a timely manner or became irrelevant over time. The PAP was generally useful in generating a dialogue on bottlenecks, capacity needs and institutional challenges but did not receive the same attention as the DLIs. Third, the number of areas to be addressed expanded with the approval of the AF (i.e., adding adolescent health, civil registration, health financing, gender-based violence) which may have stretched the capacity of the government in an increasingly difficult context, reinforcing the importance of selectivity. • The Program was highly successful in improving the availability of information for tracking health sector performance, adopting the principle of ‘what gets measured gets done’. As a result of harmonized donor support, Ethiopia benefitted from the production of regular DHS; annual facility readiness surveys to track information on availability of key inputs; assessment of the CBHI; several major nutrition surveys/assessments; as well as an enhanced HMIS system with costly surveys co-financed by partners. The timely availability of data enabled verification of DLIs; tracking of overall results; setting sectoral and state targets; and identification of challenges that required attention. The results were discussed during Joint Donor or Program Reviews and Annual Consultative meetings between national and regional authorities with the goal of identifying strategic priorities and remedial actions. While Ethiopia benefited from a gold mine of information and data, more could probably have been done to ensure that the results were systematically shared with all stakeholders at different levels of the health system. • The IPF component provided critical technical support and capacity building. To bolster capacity, the World Bank team also provided hands on support on a wide range of issues, together with other development partners. The technical support was well appreciated by government counterparts and was repeatedly cited during the August 2022 ICR mission by key stakeholders (e.g., SDGPF, nutrition, CRVS) as a key factor facilitating implementation. Use of country fiduciary systems has enabled continual improvements with issues progressively addressed over the course of implementation. • The challenging context in Ethiopia, including persistent gaps and large disparities, underscores the importance of sustained, long-term engagement. At Program inception Ethiopia lagged regional averages on several key indicators. While the country made steady and impressive progress on all PDO/DLIs over nearly a decade of support, much more needs to be done to have a major impact on maternal and child health. Designing and rolling out interventions that are appropriate for different regions (agrarian, nomadic, urban) will take time and require more in depth understanding of what works and how these regions can be Page 32 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) incentivized and supported to revamp their health services. Addressing the perennial problem of high out of pocket spending (which remains at about 30 percent) requires continued attention to the viability and sustainability of the CBHI scheme. Moreover, the large number of IDPs and the difficult context in conflict- affected regions, requires specialized, well-tailored support to restore maternal and child health services. 63. The key recommendations are as follows: • Ensure that there are clear accountability mechanisms and incentives for all key stakeholders . Channeling funds directly to the health sector as was done for the Ethiopia PforR held the ministry accountable and provided strong incentives. In a strong federal structure where states have an important degree of independence, it is important to ensure that there are also transparent and clear mechanisms in place for sharing incentives, so that stakeholders on the frontlines are also incentivized to mobilize additional resources and deliver results. Results-based financing approaches and program- based budgeting strategies should be piloted at the state and/or woreda levels to further bolster accountability and performance. • Continue to generate data for enhanced decision making while strengthening utilization. While it is important to continue generating and disseminating data on sector performance, it is critical to ensure that there are mechanisms for strengthening utilization of this information. Service providers need to be provided regular information on their performance and given incentives to improve. Introducing feedback mechanisms for woredas and health facilities would allow benchmarking of performance and would enhance accountability. • Deepen the analytic work on financial protection. While some modest progress was made in expanding financial protection, out-of-pocket spending remains above recommended levels. Further analytical work is needed to better understand and address the challenges facing households (e.g., affordability, quality of services) and providers (e.g., rising workloads, modest premiums) with a view to making the CBHI scheme viable and sustainable. • Stay on course with support for the health sector in Ethiopia. Building and leveraging on the successful implementation of the first PforR in the health sector in Ethiopia, the World Bank and the Ministry of Finance need to prioritize continued support to the health sector, tackling emerging issues and challenges. Despite excellent progress, disparities in the availability and quality of health services remain pervasive. To sustain the gains made and continue to make progress towards strengthening human capital there is a need to stay on course. . Page 33 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) ANNEX 1. RESULTS FRAMEWORK, DISBURSEMENT LINKED INDICATORS, AND PROGRAM ACTION PLAN Annex 1A. RESULTS FRAMEWORK (i) PDO Indicators Objective/Outcome: To Improve the delivery and use of comperhensive package of Maternal and Child Health (MCH) Services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Deliveries attended by skilled Percentage 10.00 18.00 40.00 50.00 birth providers 30-Mar-2012 30-Dec-2016 31-Dec-2020 30-Jun-2022 Comments (achievements against targets): Result surpassed the target and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Deliveries attended by skilled Percentage 19.00 28.00 33.40 birth providers for the bottom 3 performing regions 31-Oct-2016 31-Dec-2020 30-Jun-2022 (Afar, Oromia & Somali) Comments (achievements against targets): Page 34 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Result surpassed the target and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Pregnant women receiving at Percentage 32.00 38.00 43.00 least four antenatal care visits (Percentage, Custom) 31-Oct-2016 31-Dec-2020 30-Jun-2022 Comments (achievements against targets): Result surpassed the target and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Children 12-23 months Percentage 65.70 75.70 81.30 immunized with Pentavalent 3 vaccines 31-Dec-2012 30-Jun-2016 30-Jun-2022 Comments (achievements against targets): Data source/methodology was changed from cluster survey to EDHS 2019 in April 2021 as government could not conduct the Household Cluster Survey. Based on the adjusted result for Penta3 vaccine coverage (81.3%), the result surpassed the end target (75.7%) and disbursement was made fully. Page 35 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Contraceptive prevalence Percentage 27.30 35.00 35.00 rate 30-Mar-2012 30-Dec-2016 30-Jun-2022 Comments (achievements against targets): Result surpassed the target and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Contraceptive Prevalence Percentage 32.00 38.00 37.70 Rate ( for Rural women only) 31-Oct-2016 31-Dec-2020 30-Jun-2022 Comments (achievements against targets): Result surpassed the target and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of pregnant women Percentage 42.10 50.00 60.00 taking Iron Folic Fcid (IFA) 31-Oct-2016 31-Dec-2020 30-Jun-2022 Page 36 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): Result surpassed the target and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Conduct immunization Number 0.00 4.00 4.00 campaign at IDP/Refugee camp for missed vaccination 18-Oct-2021 30-Jun-2022 30-Jun-2022 Comments (achievements against targets): The original Indicator "Percent of Children 6-59 months receiving Vitamin A Supplements "was modified due to error in the 2019 Mini DHS data collection procedures (data on coverage of Vitamin A was only collected with vaccinations for children 6-35 months) which the result for this DLI could not be verified as the survey did not captured the correct age group. The result for the modified DLI achieved and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of Woredas in non Percentage 48.00 80.00 100.00 emerging regions delivering vitamin A supplements to 31-Oct-2016 31-Dec-2020 30-Jun-2022 children Page 37 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): Result surpassed the target and disbursement was made fully. (ii) Intermediate Results Indicators Results Area: Improve delivery and utilization of a comprehensive package of Maternal and Child Health services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Introduction of Procurement Yes/No No Yes Yes Key Performance Indicators developed by Federal Public 30-Dec-2016 31-Dec-2020 30-Jun-2022 Procurement Agenc Comments (achievements against targets): Result achieved and verified, and disbursement was made accordingly. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion PFSA submission of Backlog Yes/No No Yes Yes Yes audit reports and timely quality audit reports 30-Dec-2016 31-Dec-2020 18-Oct-2021 30-Jun-2022 thereafter Page 38 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): Although results were achieved, disbursement was made only for target 2018 as MOH could not submit the independent verification report timely /missed the disbursement schedule for target 2017 and 2019. Also MOH also could not meet 2020 target due to COVID-19. Hence, the allocated amount for those DLRs was reallocated to new DLR 12.1b: Conduct catch up campaigns to increase the uptake of essential health and nutrition services including nutrition screening, deworming and Vitamin A in security constrained area ( Amhara, Oromia, Somalia Benishangul and Afar). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Conduct catch-up campaigns Number 0.00 5.00 5.00 to increase the uptake of essential health and nutrition 18-Oct-2021 30-Jun-2022 30-Jun-2022 services including Vitamin A, deworming, and nutrition screening in security- constrained contexts Comments (achievements against targets): DLI12.1: Percent of children age 0-23 months participating in Growth Monitoring Promotion (GMP) result was partially achieved. The 2020 target was not verified as the Joint Review Mission could not be conducted due to the COVID-19 outbreak and political unrest in the country. Given these constraints, this DLI was restructured to a new process indicator DLIR 12.1b: (b) conduct catch up campaigns to increase the uptake of essential health and nutrition services including nutrition screening, deworming and Vitamin A in security constrained area (Amhara, Oromia, Somalia Benishangul and Afar). The result was achieved and disbursement was made fully. Page 39 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of Woredas in non- Percentage 0.00 50.00 89.00 emerging Regions transitioning from EOS to 30-Dec-2016 31-Dec-2020 30-Jun-2022 Community Health Days- CHD Comments (achievements against targets): Result surpassed the end target and disbursement was made fully Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of PHC Facilities Percentage 42.00 47.00 48.00 48.00 having all drug from the MOH list of drug available 30-Dec-2016 31-Dec-2020 18-Oct-2021 30-Jun-2022 Comments (achievements against targets): All targets for this indicator were met except the 2017 target. Disbursement for this specific target year was not made as the target was time-bound, and verification report was not submitted timely. Hence, this DLR 13.1a was modified to: Endorse the Health Center Clinical Audit Tool to expedite the implementation of health sector transformation quality standard. Result achieved and disbursement was made fully. Page 40 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Developed and Implement Yes/No No Yes Yes Yes postnatal Care Service directive to improve the 30-Dec-2016 31-Dec-2020 31-Dec-2021 30-Jun-2022 quality of Postnatal service Comments (achievements against targets): Result was fully achieved and disbursement was made. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improve quality of Yes/No No Yes Yes Yes adolescent services 30-Dec-2016 31-Dec-2020 31-Mar-2022 30-Jun-2022 Comments (achievements against targets): The verification of the 2020 target was dependent on SPA survey report, however, the survey could not be undertaken due to COVID-19 outbreak. Hence, the 2020 DLR was modified to: Conduct national assessment on the integration of youth friendly health services into PHCU at select health facilities. Result was achieved and disbursement was made fully. Page 41 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of Woreda with Percentage 23.00 53.00 67.00 Functional Community Health Insurance Schemes 19-Jun-2017 31-Dec-2020 30-Jun-2022 Comments (achievements against targets): The target surpassed and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Undertake CBHI review every Yes/No No Yes Yes two Years 30-Dec-2016 31-Dec-2019 30-Jun-2022 Comments (achievements against targets): Result was achieved and the disbursement was made accordingly. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Devise and implement a Yes/No No Yes Yes Yes Page 42 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) mechanism for documenting 27-Jun-2017 31-Dec-2020 30-Jun-2022 30-Jun-2022 consultations Comments (achievements against targets): This indicator was restructured along with the verification procedure in October 2021. Although MOH shared the progress report in February 2022, the verification report could not be sent to the Bank timely. The Ministry submitted the result verification report based on agreed procedure on June 23, 2022. Result was achieved and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Development and Yes/No No Yes Yes implementation of health sector community score card 30-Jun-2016 31-Dec-2020 30-Jun-2022 Comments (achievements against targets): Result fully achieved and disbursement was made accordingly. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Health Centers reporting Percentage 50.00 86.00 96.00 HMIS data in time 30-Mar-2012 01-Jan-2021 30-Jun-2022 Page 43 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): Result achieved and disbursement was made. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Development and Yes/No No Yes Yes Yes implementation of Annual Rapid Facility Assessment 30-Mar-2012 31-Dec-2020 31-Dec-2021 30-Jun-2022 Comments (achievements against targets): The government could not conduct the SARA and SPA surveys in 2021 due to factors associated with the COVID-19 outbreak and security problems in the country. As a result, the achieved result for this indicator could not be verified. Hence, the fund allocated to the end target was allocated to a new DLR 13.3d: Conduct national assessment on the integration of youth friendly health services into PHCU at select health facilities. Result achieved and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved transparency of Yes/No No Yes Yes Yes Pharmaceutical Fund and Supply Agency (PFSA) 29-Mar-2013 31-Dec-2020 31-Dec-2021 30-Jun-2022 procurement process Comments (achievements against targets): Page 44 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Result achieved and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion People who have received Number 0.00 88,943,602.00 103,602,023.90 essential health, nutrition, and population (HNP) 31-Dec-2012 30-Dec-2016 30-Jun-2022 services People who have received Number 0.00 45,091,862.00 61,348,802.45 essential health, nutrition, and population (HNP) services - Female (RMS requirement) Number of children Number 0.00 2,203,481.00 3,171,227.00 immunized 31-Dec-2012 30-Dec-2016 30-Jun-2022 Number of women and Number 0.00 85,500,000.00 63,763,085.90 children who have received basic nutrition services 31-Dec-2012 30-Dec-2016 30-Jun-2022 Number of deliveries Number 0.00 1,240,121.00 2,195,738.00 attended by skilled health personnel 31-Dec-2012 30-Dec-2016 30-Jun-2022 Page 45 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): Results Area: Sub-Component 1: Civil Registration and Vital Statistics Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of births occurring in Percentage 0.00 40.00 40.00 20.90 a given year registered 30-Dec-2016 31-Dec-2020 31-Dec-2021 30-Jun-2022 Comments (achievements against targets): Partially achieved. End target was not met due to disruption in the registration service that was resulted from COVID-19 outbreak and conflict in the country. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of deaths occurring Percentage 0.00 10.00 25.00 12.00 in a given year registered 30-Dec-2016 31-Dec-2020 31-Dec-2021 30-Jun-2022 Comments (achievements against targets): Partially achieved. End target was not met due to disruption in the registration service that was resulted from COVID-19 outbreak and conflict in the country. Page 46 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Strategic plan developed Yes/No No Yes Yes 30-Nov-2019 30-Dec-2022 30-Jun-2022 Comments (achievements against targets): Result fully achieved and disbursement was made. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of kebeles storing Percentage 0.00 80.00 90.00 68.00 and transferring registration forms safely 30-Nov-2019 30-Jun-2020 30-Jun-2022 30-Jun-2022 Comments (achievements against targets): Partially achieved. End target was not met due to disruption in the registration service that was resulted from COVID-19 outbreak and conflict in the country. Results Area: Sub-Component 2: Support to National Nutrition Program Page 47 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of nutritional Number 0.00 4.00 4.00 4.00 operational research studies conducted 30-Dec-2016 31-Dec-2020 30-Jun-2022 30-Jun-2022 Comments (achievements against targets): Target fully met and disbursement was made. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Completion of NNP Yes/No No Yes Yes Yes evaluation 27-Jun-2017 31-Dec-2020 30-Jun-2022 30-Jun-2022 Comments (achievements against targets): NNP evaluation completed. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of joint reviews of Number 0.00 7.00 7.00 7.00 NNP II carried out 30-Dec-2016 30-Dec-2020 30-Jun-2022 27-Jun-2022 Page 48 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): Target met and disbursement was made fully. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of woredas with Percentage 0.00 60.00 42.00 42.00 nutrition coordination platform 30-Dec-2016 31-Dec-2020 30-Jun-2022 27-Jun-2022 Comments (achievements against targets): Significant progress was made to increase proportion of woredas with nutrition coordination platforms across Ethiopia under the nutrition IPF sub- component budget; however it was not possible to meet the initial target (60% of all woredas) because of unexpected reporting disruptions and security constraints in some woredas due to the COVID-19 outbreak and ensuing conflict. Given these challenges, remaining resources for this indicator were used to pilot and strengthen the Unified Information Syst em Nutrition for Ethiopia (UNISE) using innovative satellite communication infrastructure and solar energy technology. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of woredas Percentage 0.00 60.00 42.00 42.00 reporting multisectoral nutrition information to 30-Dec-2016 31-Dec-2020 30-Jun-2022 27-Jun-2022 national level Page 49 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): During the project period, considerable success has been achieved in increasing the proportion of woredas reporting multisectoral nutrition information to national level in Ethiopia under the nutrition IPF subcomponent budget. Significant disruptions caused by widespread security challenges and the COVID-19 pandemic, however, made it very difficult to achieve the initial target (60% of all woredas). The remaining resources for this indicator due to these challenges was redirected to pilot and strengthen the Unified Information System Nutrition for Ethiopia (UNISE). The pilot is completed and lessons are documented. ANNEX 1B. DISBURSEMENT LINKED INDICATORS DLI IN01235620 TABLE DLI 1: Deliveries attended by skilled birth providers (Scaled DLI- 1a) (Percentage) Baseline Program period Total Original values 18.00 0.00 Actual values 40.00 Allocated amount ($) 45,430,000.00 45,430,000.00 Disbursed amount ($) 45,430,000.00 45,430,000.00 Comments (achievements against targets): Result surpassed the end target and disbursement was made fully. DLI IN01235621 TABLE DLI 2: Deliveries attended by skilled birth providers for the bottom 3 performing regions -Afar, Oromia & Somali- (New DLI- 1b) (Percentage) Page 50 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Baseline Program period Total Original values 19.00 0.00 Actual values 28.00 Allocated amount ($) 20,000,000.00 20,000,000.00 Disbursed amount ($) 20,000,000.00 20,000,000.00 Comments (achievements against targets): Result surpassed the end target and disbursement was made. DLI IN01235622 TABLE DLI 3: Children 12-23 months immunized with Pentavalent 3 vaccine (Scaled- DLI- 2c) (Percentage) Baseline Program period Total Original values 65.70 0.00 Actual values 75.70 Allocated amount ($) 8,130,000.00 8,130,000.00 Disbursed amount ($) 8,130,000.00 8,130,000.00 Comments (achievements against targets): Based on the agreed restructuring, modeling/adjustment of DPT-HepB-Hib3 Coverage Estimate was conducted by in dependent consultants and it was verified that the DLI was fully achieved and disbursement was made accordingly. Page 51 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) DLI IN01235623 TABLE DLI 4: Pregnant women receiving at least four antenatal care visits (New DLI -3a) (Percentage) Baseline Program period Total Original values 27.00 0.00 Actual values 38.00 Allocated amount ($) 20,000,000.00 20,000,000.00 Disbursed amount ($) 20,000,000.00 20,000,000.00 Comments (achievements against targets): Result surpassed the target and disbursement was made. DLI IN01235624 TABLE DLI 5: Contraceptive prevalence rate for rural women only (New DLI -4a) (Percentage) Baseline Program period Total Original values 32.00 0.00 Actual values 38.00 Allocated amount ($) 17,000,000.00 17,000,000.00 Disbursed amount ($) 17,000,000.00 17,000,000.00 Comments (achievements against targets): Page 52 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Result surpassed the target and disbursement was made fully. DLI IN01235625 TABLE DLI 6: Health Centers reporting HMIS data in time (restructured -DLI- 5a) (Percentage) Baseline Program period Total Original values 68.00 0.00 Actual values 86.00 Allocated amount ($) 7,770,000.00 7,770,000.00 Disbursed amount ($) 7,770,000.00 7,770,000.00 Comments (achievements against targets): The 2018 DQA and SARA surveys were released timely. The verification report and the necessary documentations were submitted, and disbursement was made fully. DLI IN01235626 TABLE DLI 7: Development and implementation of Annual Rapid Facility Assessment (restructured-DLI- 7c) (Yes/No) Baseline Program period Total Original values No Actual values Yes Allocated amount ($) 10,950,000.00 10,950,000.00 Page 53 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Disbursed amount ($) 10,950,000.00 10,950,000.00 Comments (achievements against targets): The SPA Plus survey and SARA surveys were conducted in 2014, 2016 and 2018 respectively. However, the MOH could not conduct the SARA and SPA surveys in 2021 as planned due to factors associated with the COVID-19 outbreak and security problems in the country. Due to this fact, MOH was not able to verify the achieved result for this indicator. Hence, the fund allocated to DLR 7c/end target was allocated to the modified DLR 13.3d: Conduct national assessment on the integration of youth friendly health services into PHCU at select health facilities with a new target year (December 2021). The last actual date refers to the last ISR date even if the data has not been updated for that reporting period. DLI IN01235627 TABLE DLI 8: Transparency of PFSA procurement process (scaled - DLI 8c) (Yes/No) Baseline Program period Total Original values Yes Actual values No Allocated amount ($) 10,860,000.00 10,860,000.00 Disbursed amount ($) 10,860,000.00 10,860,000.00 Comments (achievements against targets): Result was achieved and disbursement was made accordingly. DLI IN01235628 TABLE Page 54 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) DLI 9: Introduction of Procurement Key Performance Indicators developed by Federal Public Procurement Agency at PFSA (New DLI- 9(1)) (Yes/No) Baseline Program period Total Original values No Actual values Yes Allocated amount ($) 2,000,000.00 2,000,000.00 Disbursed amount ($) 2,000,000.00 2,000,000.00 Comments (achievements against targets): Result was achieved and disbursement was made accordingly. DLI IN01235629 TABLE DLI 10: Automate the PFSA Core Business Fiduciary System using selected software in PFSA HQ and Addis Ababa City (New_ DLI- 9(2) ) (Yes/No) Baseline Program period Total Original values No Actual values No Allocated amount ($) 7,000,000.00 7,000,000.00 Disbursed amount ($) 0.00 0.00 Comments (achievements against targets): MOH could not meet the set targets for this indicator due to: i) frequent management change at EPSA requiring a full set of independently conducted review assessments to provide information to the management for decision; (ii) although EPSA management agreed to purchase ERP, this required another Page 55 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) set of discussions with MoF which required IFMIS to be adopted by EPSA which meant more time to showcase how IFMIS was not catering to the needs of EPSA; iii) COVID-19 and associated lockdown measures have affected the procurement process; and (iv) request by the financier to conduct an audit before an award is offered has delayed the contract signing time. In addition, there was no sufficient time to implement this activity as the closing date of the program was in December 2021. Hence, this indictor was dropped and the fund ($7,000,000.00) was reallocated to a new DLR12.1b: conduct catch up campaigns to increase the uptake of essential health and nutrition services including nutrition screening, deworming and Vitamin A in security constrained area ( Amhara, Oromia, Somalia Benishangul and Afar) with new target year (December 2021). DLI IN01235630 TABLE DLI 11: PFSA submission of Backlog audit reports and timely quality audit reports thereafter (New - DLI 9(3) ) (Yes/No) Baseline Program period Total Original values No Actual values Yes Allocated amount ($) 6,000,000.00 6,000,000.00 Disbursed amount ($) 6,000,000.00 6,000,000.00 Comments (achievements against targets): Although result was achieved partially, disbursement was not made as MOH could not submit the independent verification report timely and missed the disbursement schedule. MOH also could not meet 2020 target due to COVID-19 and the associated lockdown measures that have affected the procurement process. Hence, this indictor was dropped and the fund was reallocated to new DLR 12.1b: Conduct catch up campaigns to increase the uptake of essential health and nutrition services including nutrition screening, deworming and Vitamin A in security constrained area ( Amhara, Oromia, Somalia Benishangul and Afar) with new target year ( December 2021) DLI IN01235631 TABLE DLI 12: Percent of children 6-59 months receiving Vitamin A supplements (New- DLI -10a) (Percentage) Page 56 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Baseline Program period Total Original values 45.00 0.00 Actual values 53.00 Allocated amount ($) 5,000,000.00 5,000,000.00 Disbursed amount ($) 5,000,000.00 5,000,000.00 Comments (achievements against targets): The VAS coverage could not be verified as planned in the current environment. Because of the continued disruption of routine health services due to the COVID pandemic and civil unrest, the lifesaving VAS for young children was maintained by restructuring the VAS DLI into a process indicator to support the integration of VAS into immunization catch-up campaigns. DLR12.1(b): conduct catch up campaigns to increase the uptake of essential health and nutrition services including nutrition screening, deworming and Vitamin A in security constrained area was proposed and the remaining amount was reallocated to this indicator. End-of-program targets for the following 4 DLIs/DLRs have been fully met and the results have been verified and disbursement will be made once the government submit the Result Notification Letter. DLI IN01235632 TABLE DLI 13: Percent of Wordas in non-emerging Regions delivering Vitamin A Supplements to children through routine system – Health Facilities (New- DLI-10b) (Percentage) Baseline Program period Total Original values 48.00 0.00 Actual values 80.00 Allocated amount ($) 5,000,000.00 5,000,000.00 Disbursed amount ($) 5,000,000.00 5,000,000.00 Page 57 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): Result surpassed the target and disbursement was made fully. DLI IN01235633 TABLE DLI 14: Percent of Pregnant women taking Iron Folic Acid (IFA) tablets (New- DLI- 11) (Percentage) Baseline Program period Total Original values 42.00 0.00 Actual values 60.00 Allocated amount ($) 5,000,000.00 5,000,000.00 Disbursed amount ($) 5,000,000.00 5,000,000.00 Comments (achievements against targets): Results surpassed the end targets and disbursement was made fully. DLI IN01235634 TABLE DLI 15: Percent of Children 0- 23 month participating in GMP (New DLI-12a) (Percentage) Baseline Program period Total Original values 38.00 0.00 Actual values 51.00 Allocated amount ($) 15,000,000.00 15,000,000.00 Page 58 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Disbursed amount ($) 10,000,000.00 10,000,000.00 Comments (achievements against targets): The result was achieved partially as the result for 2020 target was not verified and an analysis of routine data showed that GMP data was not being reported in several regions/woredas due to COVID and insecurity. Given these constraints, this DLR was restructured to a new process indicator (DLIR 12.1b) to promote child growth through the integration of nutrition screening into immunization catch-up campaigns in security-constrained areas and/or refugee camps/IDPs. End-of-program targets have been fully met and disbursement will be made accordingly. DLI IN01235635 TABLE DLI 16: Percent of Woredas in emerging Regions transitioning from EOS to Community Health Days- CHD (NewDLI- 12b) (Percentage) Baseline Program period Total Original values 0.00 0.00 Actual values 50.00 Allocated amount ($) 5,000,000.00 5,000,000.00 Disbursed amount ($) 5,000,000.00 5,000,000.00 Comments (achievements against targets): The achieved results was verified through the JRM, and documented that 89% of woredas transition to CHDs, which is a significant increase from 2018 (26%) and surpassing the end of project target. DLI IN01235636 TABLE DLI 17: Percent of PHC Facilities having all drug from the MOH list of drug available(New- DLI DLI-13(1)) (Percentage) Page 59 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Baseline Program period Total Original values 42.00 0.00 Actual values 47.00 Allocated amount ($) 7,000,000.00 7,000,000.00 Disbursed amount ($) 7,000,000.00 7,000,000.00 Comments (achievements against targets): All DLRs under this indicator were fully achieved except the 2017 target as it was time-bound. The Bank could not disburse the allocated amount for this DLR as verification report was not submitted timely. Hence, the DLR13.1a was modified to: Endorse the Health Center Clinical Audit Tool to expedite the implementation of health sector transformation quality standard. Result achieved and disbursement was made fully. DLI IN01235637 TABLE DLI 18: Developed and Implement postnatal Care Service directive to improve the quality of Postnatal services (New- DLI-13(2)) (Yes/No) Baseline Program period Total Original values No Actual values Yes Allocated amount ($) 5,000,000.00 5,000,000.00 Disbursed amount ($) 5,000,000.00 5,000,000.00 Comments (achievements against targets): Result was fully achieved and disbursement was made accordingly. Page 60 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) DLI IN01235638 TABLE DLI 19: Improve quality of adolescent services (New- DLI 13(3)) (Yes/No) Baseline Program period Total Original values Yes Actual values Yes Allocated amount ($) 6,000,000.00 6,000,000.00 Disbursed amount ($) 6,000,000.00 6,000,000.00 Comments (achievements against targets): The verification of the 2020 target was dependent on SPA survey report, however, the survey could not be undertaken due to COVID-19 pandemic. Hence, the DLR was modified to Conduct national assessment on the integration of youth friendly health services into PHCU at select health facilities. Target met and disbursement was made fully. DLI IN01235639 TABLE DLI 20: Percent of Woreda with Functional Community Health Insurance Schemes (New- DLI-14 (1)) (Percentage) Baseline Program period Total Original values 20.50 0.00 Actual values 50.50 Allocated amount ($) 19,500,000.00 19,500,000.00 Disbursed amount ($) 19,500,000.00 19,500,000.00 Page 61 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments (achievements against targets): The 2020 target surpassed and disbursement was made. DLI IN01235640 TABLE DLI 21: Undertake CBHI review every two Years (New- 14- (2)) (Yes/No) Baseline Program period Total Original values No Actual values Yes Allocated amount ($) 5,000,000.00 5,000,000.00 Disbursed amount ($) 5,000,000.00 5,000,000.00 Comments (achievements against targets): The review was conducted and the disbursement was made accordingly. DLI IN01235641 TABLE DLI 22: Devise and implement a mechanism for documenting consultations when communal Private land is used for construction of health facilities (New- DLI-15(1)) (Yes/No) Baseline Program period Total Original values No Page 62 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Actual values Yes Allocated amount ($) 0.00 0.00 Disbursed amount ($) 4,500,000.00 4,500,000.00 Comments (achievements against targets): This indicator (DLI 15.1) was restructured along with the verification procedure in October 2021 with a timeline of March 31, 2022. Although MOH shared the progress report in February 2022, the verification report could not be sent to the Bank timely. As the closing date of the Program was extended to June 2022, the Ministry has been able to submit the result verification report based on agreed procedure on June 23, 2022. Disbursement will be made once the safeguards team validates the progress report along with verification report. DLI IN01235642 TABLE DLI 23: Development and implementation of health sector community score card(New- DLI- 15(2)) (Yes/No) Baseline Program period Total Original values No Actual values Yes Allocated amount ($) 5,000.00 5,000.00 Disbursed amount ($) 5,000,000.00 5,000,000.00 Comments (achievements against targets): The end of Program targets have been surpassed and disbursement was made fully. Page 63 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) ANNEX 1C. PROGRAM ACTION PLAN PAP_TBL Achieved Action Timing Completion Measurement (Yes/No) Development and implementation of a Due Date 31-Dec-2018 Yes Developed Postnatal Care Directive. Postnatal Care Directive/policy Comments: Postnatal care directive was developed on August 5, 2017 and verified by implementing partners. The directive has been implemented in all PHCs. Commodity distribution process - Roll out Recurrent Yearly Yes Rolled out APTS of of Auditable Pharmaceutical Transactions and Services (APTS) and provide progress update to ensure existence of adequate monitoring mechanism of delivery to ultimate beneficiaries at the branch level Comments: Action completed. PFSA launches an open call for pre- Recurrent Yearly Yes Launched an open call prequalification and qualifications bidders as per the introduced framework contract methods. Recipient's law at least once and introducing Framework Contracting Methods for common and repetitive Page 64 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) procurement items. Comments: Action completed. Ensure a system of recording fraud and Due Date 31-Dec-2018 Yes Recorded complaints and deployed ethics and corruption complaints at all levels anticorruption liaison officers. including Woredas; Ensure the deployment of Ethics and Anticorruption Liaison Officer and experts at all levels (PFSA; PFSA regional Hubs; RHBs, offices). Comments: EPSA recruited 12 ethics officer for the entire branch offices. FPPA undertaking annual procurement Recurrent Yearly Yes Action completed audit and FMOH and Bank team to consult OFAG on the feasibility of undertaking financial and value for money audits for SDGPF. Review and improve the SBD and agree Recurrent Yearly Yes Improved Standard Bidding Document with FPPA; and review current Bid Evaluation reporting methods and develop a template that keeps and records all relevant evaluation information Comments: Page 65 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Action completed. EPSA has started using health sector SBD prepared by FPPA. Improve tracking system of monitoring Recurrent Yearly Yes Disclosed award decisions of UN Agencies based contracts with UN agencies and disclose on the developed tracking system award decisions to the public Comments: Action completed. Undertake assessment study and develop Recurrent Yearly Yes Developed and implemented the coding and coding and categorization system of categorization system of procurable items procurable items Comments: Action completed. Gender based violence strategy for the Recurrent Yearly Yes Developed health sector Gender Based Violence health sector is prepared and strategy and disclosed gender analysis from HMIS. implemented and analysis of gender disaggregated HMIS data is conducted Comments: Action completed. Page 66 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Conduct Training and regular Recurrent Yearly Yes Training provided to health workers on CRVS. implementation support of health workers at the facility level on cause of death as per national disease notification codes, registration of births and other VE registration requirements. Comments: Action completed. MoH Provides and Federal Ethics and Anti- Recurrent Semi-Annually Yes Biannual report on fraud and corruption complaints corruption ( FEACC) verify and submit to verified and submitted by Federal Ethics and Anti the Bank quality and timely biannual Corruption Commission. report on Fraud and Corruption complaints and priority actions related to the program Comments: Action completed. Ensure inclusiveness of Fraud and Recurrent Continuous Yes Fraud and corruption and compliant handling Corruption and complaint handling process discussed at joint FMOH- RHB meeting processes or priority actions in FMOH and Regional Health Bureaus joint forum discussions semiannually. Page 67 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Comments: FEACC submitted the second bi-annual report for the Ethiopian Fiscal Year (EFY) 2013 of the Health SDGs PforR for the period January 2021 – June 2021 to the Bank. Availing appropriate temporary storage Recurrent Continuous Yes Availed temporary storage facilities for collection of facilities for collection of hazardous hazardous medical wastes. wastes until final disposal, enforce compliance with MWM and Disposal Directive in HF constructed before issuance of the Directive Comments: Action completed. Update relevant documents to Due Date 31-Dec-2018 Yes Developed a document/checklist to incorporate incorporate environmental impact and risk environmental impact and risk criteria in the site criteria in site selection screening for all selection template for constructing health facilities. health facilities, strengthen the coordination and reporting mechanism on social and environmental safeguard in MOH. Comments: Action completed. Page 68 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Documenting consultations and Recurrent Continuous Yes Report on documenting consultations using the participatory nature of discussions where developed checklist. communal land is used for construction of health centers and where applicable compensation for land and livelihood paid. Comments: Action completed. Documenting outreach and specific Recurrent Continuous Yes Report on actions taken on vulnerable group actions focused on providing services to all vulnerable persons Comments: Action completed. Implementation of the postnatal care Due Date 31-Dec-2020 Yes Implemented Postnatal Care directive directive/policy. Comments: Action completed. Monitor the quality of the VAS transition Recurrent Yearly Yes Monitored the quality of VAS transition in emerging through existing platforms according to and non-emerging regions established transition standards in both emerging and non-emerging regions. (New Page 69 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) September 2020) Comments: Action completed. Strengthening system to monitor and Recurrent Yearly Yes i) The reporting format from the regions to be timely liquidate advances at regional level: updated to clearly show detailed closing balances; (New September 2020) ii) Address the capacity gaps to be identified in the regional FM assessment to be conducted jointly by MoH and the Bank in April 2020. Comments: Action completed. Strengthening the audit function and Due Date 30-Sep-2020 No i) Submit audit reports for EFY 2009 and 2010 of follow up of audits at EPSA and MoH: EPSA by September 30, 2020; ii) Submit the (New September 2020) financial statement of EFY 2011 prepared under IFRS to external auditors by July 31, 2020; iii) Fill the vacant internal audit positions both at MoH & EPSA Comments: Partially achieved. While the audit reports and financial statements had been submitted timely, activity related to filling out of internal audit vacant positions was not met. Page 70 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Gandham N.V. Ramana Task Team Leader Anne Margreth Bakilana Task Team Leader Berhanu Legesse Ayane Public Sector Specialist Binyam Bedelu Mekbib, Pascal Tegwa Procurement Specialist(s) Feben Demissie Hailemeskel Social Development Specialist Meron Tadesse Techane Financial Management Specialist Tamene Tiruneh Matebe Environmental Specialist Yalemzewud Simachew Tiruneh Social Development Specialist Asegid Regassa Team Member Eleni Albejo Team Member Erika Marie Lutz Team Member Karine N. MOUKETO-MIKOLO Team Member Kidist Kebebe Demissie Team Member Lisa Shireen Saldanha Team Member Maletela Tuoane Team Member Qaiser Khan Team Member Roman Tesfaye Team Member Shafali Rajora Team Member Page 71 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Tseganeh Amsalu Guracha Team Member Wubedel Dereje Alemu Team Member Yonas Regassa Guta Team Member Yoseph Abdissa Deressa Team Member Yvette M. Atkins Team Member Supervision/ICR Anne Bakilana, Paul Robyn, Roman Tesfaye, Enias Task Team Leader(s) Baganizi, Berhanu Legesse Ayane Public Sector Specialist Binyam Bedelu Mekbib, Shimelis Woldehawariat Badisso Procurement Specialist(s) Feben Demissie Hailemeskel Social Development Specialist Meron Tadesse Techane Financial Management Specialist Tamene Tiruneh Matebe Environmental Specialist Yalemzewud Simachew Tiruneh Social Development Specialist Andrea Vermehren Team Member Asegid Regassa Team Member Fowzia Yahya Musleh Al-Qobi Team Member Gandham N.V. Ramana Team Member Gertrude Mulenga Banda Team Member Karine N. MOUKETO-MIKOLO Team Member Kidist Kebebe Demissie Team Member Lisa Shireen Saldanha Team Member Maletela Tuoane Team Member Marion Jane Cros Team Member Phoebe M. Folger Team Member Shafali Rajora Team Member Tewodros Assefa Tesemma Team Member Tseganeh Amsalu Guracha Team Member Page 72 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) Wubedel Dereje Alemu Team Member Yonas Regassa Guta Team Member Yvette M. Atkins Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY11 20.480 120,581.32 FY12 27.729 271,183.82 FY13 41.965 541,003.50 FY20 0 1,170.00 Total 90.17 933,938.64 Supervision/ICR FY13 12.200 25,834.81 FY14 71.383 475,872.47 FY15 41.083 287,239.48 FY16 50.235 502,174.93 FY17 83.075 487,994.42 FY18 46.590 266,642.96 FY19 53.108 418,899.50 FY20 88.727 611,674.62 FY21 102.260 1,244,381.43 FY22 84.127 791,252.45 FY23 14.400 61,464.52 Total 647.19 5,173,431.59 Page 73 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) ANNEX 3. PROGRAM EXPENDITURE SUMMARY Actual Expenditures (Disbursement) Estimates at Source of Program Type of Co- Appraisal EFY 2009-2022 Financing Financing US$ Mill. Percentage of Percentage of Actual Appraisal Actual World Bank IDA 100 100 100 100 TF 20 20 100 100 IDA 150 150 100 100 TF 60 60 100 100 TF 20 20 100 100 Total 350 Other Partners (SDGPF) 1,340 Total 1,690 Page 74 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) ANNEX 4. BORROWER’S COMMENTS We are in overall agreement with the ICR and would like to reiterate a few aspects, as described below. Relevance of the PDO. The PDO was well aligned with the Government’s HSTP I strategic priorities areas which focused on the RMNCH-N, quality improvement, and system strengthening. The PDO design was consultative from government and World Bank side. The PDO was well-articulated reflecting the priority areas, the DLIs had clear verification protocols and implementation arrangements. Also, the close follow up of top leadership at both the MoH and MoF was exemplary during the overall process. The PDO played a crucial role in focusing attention on achieving major targets set in HSDP IV and HSTP I and strengthening the health system. Disbursement Linked Indicators. The DLIs were well aligned with the Government’s Monitoring and Evaluation plan of the HSTP, and this provided an opportunity to strengthen the M&E system of the Government. With the DHIS2 implemented during the P4R period there was progress in improving data quality and promoting evidence-based decision making. The overall process of identifying the DLIs, establishing baseline data and setting targets was informed by evidence and involved a consultative process. Key Challenges • Some of the process DLIs were time-bound which was a challenge as the situation changes due to some external factors like COVID-19 and insecurity in some regions which affected the timely implementation. Thus, some DLIs were not met on the specified timeframe with resources reallocated to other DLIs. • Insecurity and COVID-19 posed challenges to the overall implementation of the HSTP II, which necessitated restructuring of some DLIs. Main Lessons • Strong and frequent joint supervision missions helped to discuss strengths, weaknesses and to identify remedial actions included in joint plans of action. • The PDO articulation was strong and robust with DLIs remaining relevant following restructurings. • The strong alignment of the PDO with the Government’s One plan, One report and One budget principle represented good practice. Page 75 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) ANNEX 5. SUMMARY OF REVISED OUTCOMES AND DLIS With the processing of the AF (2017) the following changes were made to the PDO/IO/DLI indicators29: PDO indicators/DLIs: • PDO Indicator 1/DLI 1a (Deliveries attended by skilled birth provider) was met, and the end of project target was increased to continue to make progress on this key indicator. In addition, a new indicator (Deliveries attended by skilled provider for bottom three performing regions (1a) was added to focus attention on addressing inequities. • PDO Indicator 2/DLI 2b/c (Children 12-23 months immunized with Pentavalent 3 vaccine) was maintained while the target was increased and the methodology for verifying results was amended, as it was not feasible to conduct cluster surveys, hence modelling was done using the DHS survey to validate results. • PDO Indicator 3/DLI 3 (Pregnant women receiving at least one antenatal care visit) was met and surpassed the original target and was dropped; and a new indicator (3a) was added (Pregnant women receiving at least four antenatal care visits) to focus attention on globally recommended practices in quality care. • PDO Indicator 4/DLI 4 (Contraceptive prevalence rate) was met and surpassed and dropped; and a new indicator (4a) was added (Contraceptive prevalence rate in rural areas) to improve delivery and use of contraceptives in underserved rural areas. • PDO Indicator 5/DLI 11 (Pregnant women taking iron and folic acid tablets) was added to focus greater attention on maternal health and nutrition. • PDO Indicator 6/DLI 10a (Children 6-59 months receiving vitamin A supplements) was added to focus greater attention on child health and nutrition. • PDO Indicator 7/DLI 10b (Woredas in non-emerging regions delivering Vitamin A supplements to children) was added to focus greater attention on child health and nutrition. Intermediate Results Indicators/DLIs • IO Indicator 1/DLI 5 (Health Centers reporting HMIS data in time) was restructured to allow more time for achievement and to attain a more ambitious target. • IO Indicator 2/DLI 6 (Develop and implement a Balanced Score Card Approach) was dropped as the approach was rolled out nationwide, not allowing for measurement of impact as originally envisioned. • IO Indicator 3/DLI7 (Develop and implement Annual Rapid Facility Assessment) was complied with regularly with the DLI scaled to ensure facility surveys continued to be undertaken every two years. • IO Indicator 4/DLI 8 (Transparency of PFSA Procurement Process) was scaled to ensure comprehensive information was disclosed on website for continued improved transparency. • IO Indicator 5/DLI 9(1) (Introduction of Procurement Key Performance Indicators developed by Federal Procurement Agency at PFSA) was added to monitor and track performance. 29 For the ICR, the indicators have been numbered in chronological order, dividing up the PDO and IO indicators. Page 76 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) • IO Indicator 6/DLI 9(2) (Automate the PFSA core business fiduciary system using Selected Software in PFSA HQ and Addis Ababa City) was added to improve performance. • IO Indicator 7/DLI 9(3) (PFSA submission of backlog audit reports and timely quality audit reports thereafter) was added to improve transparency and performance. • IO Indicator 8/DLI 12a (Children 0-23 months participating in Growth Monitoring and Promotion) was added to focus greater attention to child health and nutrition. • IO Indicator 9/DLI 12b (Percent of woredas in emerging regions transitioning from Enhanced Outreach Services to Community Health Days) was added to focus greater attention to child health and nutrition. • IO Indicator 10/DLI 13(1) (PHC facilities having all drugs from the MOH list of essential drugs available) was added to better track and monitor drug availability which is critical for both maternal and child health services. • IO Indicator 11/DLI 13(2) (Develop and implement postnatal care services directive to improve the quality of postnatal services) was added to enhance management of maternal health services. • IO Indicator 12/DLI 13(3) (Improve quality of adolescent health services) was added with the focus on the ‘A’ as part of the Reproductive, Maternal, Newborn, Child, and Adolescent Health (RMNCAH) agenda. • IO Indicator 13/DLI 14(1) (Woredas with functional Community Based Health Insurance schemes) was added to focus attention on lowering out of pocket spending. • IO Indicator 14/DLI 14(2) (Undertake CBHI scheme reviews every two years) was added to track and monitor progress in expanding financial protection. • IO Indicator 15/DLI 15(1) (Devise and implement a mechanism for documenting consultations when communal/private land is used for construction of health facilities) was added to establish a process to facilitate community involvement and consultation. • IO Indicator 16/DLI 15(2) (Develop and implement a Health Sector Community Score Card) was added and targets were established for progressive roll out to woredas. • IO Indicator 17/DLI 19a (Conduct immunization campaign at IDP/Refugee camps for missed vaccinations) was added during the COVID-19 pandemic. • IO Indicator 18/DLI 12.1(b) (Conduct catch up campaigns to increase the uptake of essential health and nutrition services, including Vitamin A, deworming, and nutrition screening, in security-constrained contexts) was added to mitigate impact of the ongoing COVID-19 pandemic and deteriorating security situation. Page 77 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) ANNEX 6. SUMMARY OF ACHIEVEMENTS: DISBURSEMENT LINKED INDICATORS No Indicator Status Priority Area 1: Accelerate Progress towards the Maternal Health MDGs DLI/PDO: DLI1a Deliveries attended by skilled birth provider rose from 10% Surpassed (2011) to 50.0% (2019), surpassing end of Program target of 40.0% (Original/Scaled up) DLI1b Deliveries attended by skilled birth provider for bottom three Surpassed performing regions increased from 19% (2016) to 33.4 (2019), surpassing end of Program target of 28.0% (AF) DLI3 Pregnant women receiving at least one antenatal care visit rose Surpassed from 43% (2011) to 62.8% (2016), surpassing target of 56%, and rose further to 74.5 (2019); DLI was dropped when the AF was processed, and 3a was added to set more ambitious target (Original) DLI3a Pregnant women receiving at least four antenatal care visits Surpassed increased from 32% (2016) to 43% (2019), surpassing the end of Program target of 38% (AF) DLI4 Contraceptive Prevalence Rate rose from 27.3% (2011) to Surpassed 35.9% (2016) against the end of Program target of 35%, and was subsequently dropped with the AF to focus on contraceptive use in rural areas (Original) DLI4a Contraceptive Prevalence Rate in rural areas increased from Achieved 32% (2016) to 37.7% (2019) against a target of 38% (AF) DLI11 Pregnant women taking iron and folic acid tablets rose from Surpassed 42.1% (2016) to 60% (2019), surpassing the end of Program target of 54.1% (AF) Priority Area 2: Sustain gains made in child health MDG DLI/PDO/IO: DLI2b/c Children 12-23 months immunized with Pentavalent 3 vaccine Surpassed was confirmed to have increased from 65.7% (2011) to 81.3% (2022), surpassing the end of Program target (76.7%), using the option of verifying data from DHS surveys rather than cluster surveys which was the original source of data, which was endorsed by OPCS and DECR (Original/Restructured/Scaled up) DLI10a Children 6-59 months receiving Vitamin A supplements could Dropped not be verified as the 2019 DHS captured data for the incorrect age group (6-35 months), hence the indicator was dropped (AF) DLI10b Woredas in non-emerging regions delivering vitamin A Surpassed supplements to children rose from 48% (2016) to 100% (2020), surpassing the end of Program target of 80% (AF) Page 78 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) DLI12a Children 0-23 months participating in Growth Monitoring and Achieved Promotion (GMP); achieved results could not be verified as the JRM could not be conducted due to the COVID-19 pandemic and political unrest in the country and routine data could not be used; hence, the DLI was restructured to a new process indicator to conduct GMP during immunization catch-up campaigns; end-of-program targets have been fully met and results verified (AF) DLI12b Woredas in emerging regions transitioning from Enhanced Surpassed Outreach Services to Community Health Days increased from zero (2016) to 89% (2022), surpassing the end of Program target of 50.0% (AF) DLI19a Conduct immunization campaign at IDP/Refugee camps for Achieved missed vaccinations; targeted catch up immunization campaigns were conducted in four priority regions during the COVID-19 pandemic (Restructured/New) DLI12.1b Conduct catch-up campaigns to increase the uptake of essential Achieved health and nutrition services, including Vitamin A, deworming, and nutrition screening, in security-constrained contexts; catch- up campaigns were conducted in 5 priority regions during the COVID-19 pandemic (Restructured/New) Priority Area 3: Strengthen health systems DLI/IO: DLI5 Health centers reporting HMIS data in time rose from 50% Achieved (2011) to 84% (2022) nearly attaining the revised target of 86% (Original/Scaled up) DLI6 Develop and implement a balanced score card was dropped as Dropped the implementation modality changed when the program was rolled out country wide, making it not possible to measure impact, hence it was dropped (Original) DLI7 Develop and implement Annual Rapid Facility Assessment; this Achieved/Dropped DLI was complied with during most of the life of the project with regular assessments conducted, results disclosed and plans of action developed; however, in 2021 the SARA/SPA survey could not be conducted due to the ongoing COVID-19 pandemic and the security situation in the country, hence the DLI was dropped, and funds allocated to a new DLI 13/3d (Original) DLI8 Improve transparency of Pharmaceutical Fund and Supply Achieved Agency (PFSA) and procurement process was achieved with the establishment of the website and comprehensive disclosure of information by end of project (Original/Scaled up) DLI9 (1) Introduction of Procurement Key Performance Indicators (KPIs) Achieved developed by the Federal Public Procurement Agency (FPPA); Page 79 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) results were achieved and verified by April 2022 despite some delays in previous years (AF) DLI9 (2) Automate the PFSA core business fiduciary system using Dropped selected software in PFSA HQ and Addis Ababa City; last ISR (April 2022) noted that there is not sufficient time to implement this activity as the closing date of the project was approaching; indicator was dropped, and funds were directed to DLI12.1(b) (AF) DLI9 (3) PFSA submission of backlog audit reports and timely quality Dropped audit reports thereafter; last ISR (April 2022) noted that this DLI could not be met with the imminent closing of the project, hence funds were moved to a new DLI 19a (AF) DLI13 (1) PHC facilities having all drugs from the MOH list of essential Achieved drugs available; last ISR (April 2022) noted that the end of Program target has been fully met and results verified (AF) DLI13 (2)/2d Develop and implement postnatal care services directive to Achieved improve the quality of postnatal services; directive is implemented in all PHCs; monitoring report produced and verified by implementing agencies; assessment of the effectiveness of the directive was conducted in 2021 to comply with 13 (2d) (AF) DLI13 (3)/3d Improve quality of adolescent health services; last ISR (April Achieved 2022) noted that the SARA/SPA survey could not be conducted before November 2021, the DLI was replaced with a new one (13.3d); end-of-program targets have been fully met and results verified (AF) DLI14 (1) Woredas with functional Community Based Health Insurance Surpassed (CBHI) schemes rose from 23% (2017) to 65% (2022), surpassing the end of Program target of 53% (AF) DLI14 (2) Undertake CBHI scheme review every two years (AF) Achieved DLI15 (1) Devise and implement a mechanism for documenting Achieved consultations when communal/private land is used for construction of health facilities (AF); achieved with delays. 15 (2) Develop and implement Health Sector Community Score Card Achieved (AF) Page 80 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) ANNEX 7. SUPPORTING DOCUMENTS – BORROWER’S REPORT Implementation Completion Report Ethiopia Health SDGs PforR for Results (PforR) A. Context at Appraisal 1. At Program appraisal in early 2013, Ethiopia had made impressive progress in improving health outcomes and was among countries in Sub-Saharan Africa with a good chance of attaining the health MDGs. Based on the United Nations MDG progress assessment, Ethiopia had already achieved the majority of the key MDG targets (i.e., child health, HIV/AIDS, and malaria). Ethiopia had also made commendable progress in improving childhood stunting, which dropped from 51 percent in 2005 to 44 percent in 2011; with the prevalence of anemia among women declining from 27 percent in 2005 to 17 percent in 2011.30 Contraceptive prevalence also nearly doubled (i.e., from 15 percent to 27.3 percent) and the total fertility rate declined (from 5.4 to 4.8) during the period 2005 to 2011. 2. Even though substantial achievements were made, significant maternal and child health challenges remained. Under-five mortality was still very high standing at 88 per 1,000 live births in 2011. The maternal mortality ratio had barely improved over the previous years and was standing at 676 deaths out of 100,000 live births in 2011. Coverage of maternal health services such as antenatal care and skilled care during childbirth was low with only 44 percent of pregnant women receiving at least one antenatal care (ANC) visit and just 10 percent of women benefiting from skilled birth attendance. 3. To address key health system challenges, Ethiopia needed to adopt a greater focus on results, by shifting attention from inputs to results. The shift towards results was necessitated by a need for enhanced accountability at different levels of the health system. Ethiopia had to accelerate efforts to institutionalize monitoring and evaluation (M&E) systems that provide reliable and timely information on Program results, that in turn strengthen evidence-based planning and programming, particularly at facility and district levels. The health system was also characterized by low public financing on health; and a significant share of external assistance on health focused on a few diseases, leaving a substantial financial gap in the delivery of reproductive, maternal, neonatal and child health (RMNCH) services and in strengthening of the health system. 30 Ethiopia Demographic Health Survey 2011 Page 81 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) 4. The PforR aimed to support efforts by the Government of Ethiopia and its partners to accelerate progress in achieving the health MDGs. The Health Sector Development Program (HSDP), which started in 1996, provided the overarching strategic framework for the sector and played an important role towards the gains Ethiopia achieved. The Government and its partners were jointly working with commitment and determination to reach the MDGs. At Program appraisal in in early 2013, the HSDP IV was being implemented with the Ethiopian Government’s vision to achieve the health sector goals set under the country’s Growth and Transformation Plan that were closely aligned with the MDGs targets. The PforR, thus, aimed to respond to the Government’s commitment to achieving the MDGs and to adopting a result focused approach in the health sector. Given Ethiopia’s devolved federal structure of governance, where responsibilities are shared from top to bottom structures, the PforR aimed to motivate program managers across different levels of the health system to deliver essential health services, and to find locally relevant and sustainable solutions for addressing operational bottlenecks. B. Additional Financing 5. The PforR benefitted from substantial Additional Financing (AF). To this end, US$230 million AF was approved in 2017 and the PforR was restructured and was given a new name: Ethiopia Health Sustainable Development Goals (SDGs). While the Program Development Objective (PDO) remained the same: to improve the delivery and use comprehensive package of maternal and child health (MCH) services, changes were made to some of the PDOs/Disbursement Linked Indicators (DLIs). Few PDO/DLIs whose targets had been met under the original Program but that required stretch targets or more ambitious goals were scaled up: (i) skilled birth attendance, (ii) antenatal care coverage (i.e., from one to four visits), and (iii) contraceptive prevalence rate (i.e., initially focused on national CPR and subsequently on the CPR in rural areas). Additional DLIs were added during the restructuring that would trigger disbursements upon attainment of results on: (i) vitamin A supplementation; (ii) iron folic acid (IFA) supplementation; (iii) growth monitoring and promotion (GMP); (iv) adolescent health; (v) community health insurance; (vi) community participation in health service delivery; and (vii) use of the Grievance Redress Mechanism and Community Score Cards. The restructuring also involved the introduction of a new Investment Project Financing (IPF) component to support three key areas: (i) Civil Registration and Vital Statistics System (CRVS); (ii) capacity building for nutrition and related technical assistance; and (iii) capacity building and technical assistance for various activities and institutions (e.g., health financing). C. Key Achievements 6. Ethiopia has achieved substantial improvements in health outcomes during the last decade. Ethiopia managed to achieve six of the eight MDGs31 and made substantial progress on gender equality and empowerment, and maternal health. Maternal health has improved considerably with the Maternal Mortality Ratio declining to 412 deaths per 100,000 live births in 2016.32 ANC coverage also significantly improved with 72.1 percent of pregnant women having one or more ANC visits 31All except Goal 3 and Goal 5: MDG report 2014 Ethiopia 32Central Statistical Agency (CSA) [Ethiopia] and ICF. 2016. Ethiopia Demographic and Health Survey 2016. Addis Ababa, Ethiopia, and Rockville, Maryland, USA: CSA and ICF. Page 82 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) during the period 2016 – 2019.33 Ethiopia has also achieved a significant improvement in childhood stunting, from 44 percent in 2011 to 37 percent in 2019. The country has been implementing the Health Structural Transformation Plan (HSTP) during 2015 – 2020 with priorities closely aligned with the Sustainable Development Goals (SDGs). The PforR instrument proved to be effective in supporting the achievement of key health results that were initially aligned with the MDGs and subsequently with the SDGs; and in strengthening institutional capacity that has been critical for continual success. 7. The Program’s targets have been achieved or surpassed. Overall, all the PDO/DLI indicators have surpassed and/or fully achieved the end-of-program targets. Continued progress was also made in the implementation of the intermediate indicators and all end-of-program targets have been successfully met. These achievements were made despite the significant challenges that the ongoing insecurity in the country and the COVID19 pandemic created over the last few years. Table 1 below presents the key results from the Results Framework PDO indicators, highlighting substantial improvements in coverage of these essential maternal and child health services. Table 1: Achievement of PDO indicators PDO Indicator Baseline Actual End Target (2011) (2019) Deliveries attended by Skilled Birth Provider 10% 50% 40% Deliveries attended by Skilled Birth Provider for the bottom 19% 33.4% 28% 3 performing regions (Afar, Oromia and Somali) Children 12-23 months immunized with Pentavalent 3 65.7% 81.3% 76.7% vaccine Pregnant Women receiving at least one antenatal care visits 33.9% 73% 56% Pregnant women receiving at least four antenatal care visits 32%* 43% 38% Contraceptive prevalence rate (CPR) 27.3% 41% 35% Contraceptive prevalence rate (CPR) (for rural women only) 32%* 37.7% 38% Percent of pregnant women taking iron and folic acid (IFA) 42.1%* 60% 54.1% Percent of woredas in non-emerging regions delivering 48%* 100% 80% vitamin A supplements to children through routine systems * The baseline year for these is 2016 as these were added during the AF. 33Ethiopian Public Health Institute (EPHI) [Ethiopia] and ICF. 2019. Ethiopia Mini Demographic and Health Survey 2019: Final Report. Rockville, Maryland, USA: EPHI and ICF. Page 83 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) 8. The PforR has also achieved most of the IOs/DLIs with some targets surpassed. The end-of-Program targets for almost all of the DLIs have been met with the exception of a few DLIs: automate the PFSA Core Business Fiduciary System using selected software in PFSA HQ and Addis Ababa; few of the DLIs whose achievements were not possible to be verified due to the disruption created by the COVID19 pandemic and insecurity in the country were restructured into process indicators to address emerging needs (i.e., rise in internally displaced persons). The achievement of these process indicators was verified, and targets have been achieved. 9. Progress has been made in building capacity of institutions and improving performance. The PforR played a crucial rule in strengthening institutional capacities of the various government agencies that fall within the Program’s boundary. The Pharmaceutical Fund and Supply Agency (PFSA) improved transparency of its procurement processes; and introduced a system for tracking Key Performance indicators developed by the Federal Public Procurement Agency. Substantial results were also achieved with regard to health financing as part of the PforR. The end-of-Program target for percent of woredas with functional Community Based Health Insurance (CBHI) schemes was surpassed. In addition, the DLI on undertaking CBHI review every two years was also achieved. The DLI on development and implementation of health sector community score card was surpassed. 10. The IPF component of the PforR was also instrumental in making substantial progress on the CRVS and nutrition agenda. Though the COVID19 pandemic and the insecurity in the country made achieving end-of-program targets difficult for CRVS indicators, promising progress has been made with the percent of births occurring in a given year registered rising from 0 percent (2016) to over 21 percent (2022) in comparison to a target of 40 percent; and the percent of deaths occurring in a given year registered increasing from about 0 percent (2016) to nearly 12 percent (2022) in comparison to a target of 25 percent. The IPF sub-component on support to the National Nutrition Program (NNP) was also achieved with substantial results. The end-of-program target of conducting four nutritional operational research studies was achieved and so was the target of completing the NNP evaluation. Similarly, the target on the number of joint reviews of the NNP II carried out was achieved. Substantial progress has also been made on the proportion of woredas with a nutrition coordination platform; and the proportion of woredas reporting multisectoral nutrition information to national level. Also, the P4R played a crucial role in institutionalization of campaign-based nutrition services into routine service delivery in agrarian woredas; and in enhanced outreach-based services (every six months) to Community Health Days (every three months) service provision in pastoralist woredas. D. Key Strategies that Contributed to the Results Attained The Government adopted and implemented several key strategies and high impact interventions which contributed to the significant improvement in key results and in strengthening the health system. 11. Adopting the transformation agenda: The Government adopted four transformation agendas as focus areas during the HSTP I period which were Equity and Quality, Woreda transformation, Information revolution and Compassionate, competent, and Motivated health work force. The implementation of these agendas played a crucial role in providing strategic guidance and focusing attention and resources on attaining the aligned targets of the HSTP I and the P4R. Page 84 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) 12. Focusing on high-impact RMNCH-N interventions: The Government adopted key high impact maternal and child health interventions during the implementation of the Program. There was expansion in the Basic Emergency management of obstetric care (BEmONC) and Comprehensive emergency management of obstetric care (CEmONC) at health facilities; deployment of midwifes and health officers to health facilities; expansion of maternity waiting rooms for pregnant and laboring mothers; establishment of pregnant mothers forum at community level which helped in early identification and referral of pregnant mother to health facilities; expansion of integrated management childhood illnesses at facility and community level; expansion of neonatal ICUs at hospitals; improved availability of life saving drugs; improved availability of ambulances with at least one per woreda during the HSTP I period for referral of pregnant mothers. These strategies and innovations contributed to the remarkable achievements noted above. During the same period the Government developed complementary strategies like the child survival strategy, adolescent and youth strategy, and reproductive health which have been widely disseminated to inform the design and implementation of operational plans, and investments by both Government and non- government organizations. Access to high impact maternal and child interventions improved, the quality of these services was bolstered, and the equity gap narrowed, in line with the targets of the HSTP I and the P4R. 13. Promoting community participation and engagement: Community Participation and Engagement (CP&E) involved the transfer of knowledge and skills to communities to facilitate their involvement in planning, implementation, and monitoring of their health-related activities. To realize this the Government designed a strategy to establish a platform for community members to discuss key topics of interest (e.g., how health extension workers can better serve communities; key disease prevention and control measures). Community engagement played a significant role in creating model households, and model kebeles and in stimulating demand for key maternal and child health services. Model households practice all the 16 health extension packages and guide other community members under the health development network to do the same, which in turn, helps to increase the health literacy and health seeking behavior. Similarly, community members together with their HEW work to make their kebele to be models by making their kebele free from home delivery, avoiding open defecation, and creating model schools which have WASH service. 14. Strengthening critical infrastructure: The Government gave a special focus in the HSDP IV and HSTP I to the expansion of health facilities to make services more accessible. This was funded both through the Government budget and the SDG Performance Fund. In the period between HSDP IV and HSTP the number of health posts increased to 17,550, health centers to 3,735 and hospitals to 353 which respectively, represent a 23%, 74% and 204% rise, from the HSDP IV baseline figures. 15. Bolstering the supply chain and logistics system: The Government adopted different reform initiatives to improve the supply chain and logistics system during the HSTP I period (2015/16 to 2019/20). The reforms focused on institutionalization of the decentralized distribution of essential commodities; outsourcing of transport and distribution services; improving procurement procedures, which reduced the product list from 2500+ to 1300 items to focus on a smaller number of procurement items; expansion of the Auditable Pharmaceutical System at hospitals; introducing Page 85 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) a digital logistic system at the central hub and at all hubs and at select high load health facilities to improve the availability of essential drugs. 16. Adopting health financing reforms: The Government identified health financing reforms as one of the strategic objectives of the HSTP with a focus on expanding the community health insurance to lower out of pocket spending, and increasing public spending on health, with promising achievements during the HSTP I period. During the HSTP I period (2015/16-2019/20) the percentage of woredas implementing the CBHI scheme increased from 15% to 70% at the end of HSTP I, health- care spending by the Government as a percentage of GDP increased from 1.4% in to 2%, and OOP spending dropped from 34% to 31%, though further efforts are needed to improve performance during HSTP II period. E. Key Factors that Affected Performance 17. External factors have significantly affected the PforR performance. The COVID19 pandemic has resulted in service disruptions, particularly at the late stage of the Program. However, the Government designed different mitigation strategies like task shifting; placing different guidelines to maintain the provision of essential health services; and operationalizing the HSTP I activities which helped to minimize risks and maintain gains. 18. Political and social unrest, as well as frequent and protracted disease outbreaks and disasters were some additional challenges during the HSTP I and P4R period. Also, frequent turnover of leadership and management at the federal and woreda levels influenced consistency of actions and sustaining some of the achievements. F. Sustainability 19. The PforR has supported the Government to adopt a greater focus on results and to build substantial institutional capacity for the delivery of RMNCAH+N services. While important progress was made over the past decade, challenges persist and require sustained attention. The Government aims to leverage the achievements made and use the capacity that has been built to achieve even higher targets. The Government has recently adopted the health sector Transformation Plan II (HSTP II 2020/21 to 2024/25) that puts a strong focus on quality, equity, and financial protection, building on the achievements made so far. The Government has also introduced programs and initiatives that scale-up activities that were important components of the PforR. A notable example is the Seqota Declaration, an initiative the Government, in collaboration with its partners, is expanding into many woredas in the country to scale-up nutrition services based on multisectoral nutrition planning and that is aimed at ending stunting in children under two years by 2030.34 The recent approval of the follow-up Bank-funded PforR/hybrid operation represents another key element in sustaining the gains made and addressing emerging humanitarian issues in some regions due to conflict, drought and floods. 34 https://www.moh.gov.et/site/initiatives-4-col/Seqota_Declaration Page 86 of 87 The World Bank Health Sustainable Development Goals Program-for-Results (P123531) G. Main Lessons 20. Leveraging Government’s commitment and institutional capacity were key factors facilitating the implementation of the PforR. As the first PforR rolled out by the Ministry of Health, it relied and further strengthened existing institutional arrangements and systems to take on the challenge of piloting and implementing this financing approach. 21. The PforR instrument leveraged and strengthened country systems and demonstrated flexibility in aligning and harmonizing support with that of other development partners. The PforR allowed to harmonize partner support under a single umbrella of the MDGPF/SDGPF and to provide much needed resources for RMNCAH+N services. Also, the P4R played crucial role in health system strengthening, including reforming the supply chain system; strengthening health information system, and human resource development; expanding health infrastructure; and scaling up and institutionalization the CBHI scheme. The PforR proved flexible in its design, allowing for adjustments to respond to changing circumstances. This was indeed critical during the last few years when various natural and man-made shocks such as droughts and conflicts resulted in unprecedented challenges. The PforR allowed to adjust some DLIs to address immediate health needs of the population and to respond in timely manner, preventing substantial health crises for the most vulnerable groups – women, children, and displaced persons. 22. Efficient utilization of data is critical to identify key gaps and propose remedial actions. The PforR has helped the Government to harness donor support in the generation and exploitation of various health sector data including the Demographic Health Surveys, annual facility readiness surveys, and the Health Management Information System. The use of data has proved to be instrumental in helping the Government to identify health gaps and devise appropriate strategies. Efficient generation and exploitation of information has also substantially improved coordination with the states and woredas, supporting them to identify gaps, set targets, and track achievements. Page 87 of 87