Report No: ICR00125 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON IDA CREDIT 55720 IN THE AMOUNT OF XDR 88 MILLION (USD 118.5 MILLION EQUIVALENT) AND IDA CREDIT 59980 IN THE AMOUNT OF XDR 88.4 MILLION (USD 119 MILLION EQUIVALENT) AND IDA CREDIT 67050 IN THE AMOUNT OF USD 121 MILLION AND IDA GRANT D0210 IN THE AMOUNT OF XDR 60.9 MILLION (USD 82 MILLION EQUIVALENT) AND IDA GRANT D4390 IN THE AMOUNT OF XDR 85.7 MILLION (USD 115.4 MILLION EQUIVALENT) AND IDA GRANT D6750 IN THE AMOUNT OF XDR 57.9 MILLION (USD 77.9 MILLION EQUIVALENT) AND TRUST FUND TF-18375 IN THE AMOUNT OF USD 6.5 MILLION AND TRUST FUND TF-A4579 IN THE AMOUNT OF USD 40 MILLION AND TRUST FUND TF-A5096 IN THE AMOUNT OF USD 3.47 MILLION AND TRUST FUND TF-A6945 IN THE AMOUNT OF USD 4.53 MILLION TO THE Democratic Republic of Congo FOR THE Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) April 21, 2025 Health, Nutrition & Population Eastern And Southern Africa The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) CURRENCY EQUIVALENTS (Exchange Rate Effective {Aug 12, 2024}) Currency Unit = Congolese Franc CGF 2820 = USD 1 US$1.34603 = SDR 1 FISCAL YEAR January 1 - December 31 Regional Vice President: Victoria Kwakwa Country Director: Albert G. Zeufack Regional Director: Daniel Dulitzky Practice Manager: Francisca Ayodeji Akala Task Team Leader (s): Fatima El Kadiri El Yamani, Michel Muvudi Lushimba ICR Main Contributor: Christel M.J. Vermeersch The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ABBREVIATIONS AND ACRONYMS AF additional financing CAGF Resource Management Unit (Cellule d’Appui et de Gestion Financière) CDF Congolese Franc CDR Regional Distribution Center (Centrale de Distribution Régionale) CERC contingent emergency response component CGPMP Procurement Unit (Cellule de Gestion en Passation de Marchés Publics) CPA complementary package of activities CPF country partnership framework CRI corporate results indicator CT-FBR PBF Technical Unit (Cellule Technique –Financement Basé sur les Résultats) DALY Disability-adjusted life years DEP Planning Directorate (Département d’Etude et de Planification) DFF direct facility financing DHIS2 District Health Information System - version 2 DHS Demographic and Health Survey DPS Provincial Health Directorate (Direction Provinciale de la Santé) DRC Democratic Republic of Congo ECZS Health Zone Team (Equipe Cadre de Zone de Santé) EU European Union EUP public service institution (établissement d'utilité publique) EVD Ebola virus disease FEDECAME Federation of Essential Medicines Supply Centers (Fédération des Centrales d’Approvisionnement en Médicaments Essentiels) FY fiscal year GBV gender-based violence IGS General Health Inspectorate (Inspection Générale de la Santé) ISR implementation status and results report HIV/AIDS human immunodeficiency virus/ acquired immunodeficiency syndrome HMIS health management information system HR human resources HRITF Health Results Innovation Trust Fund HZ health zones ICR implementation completion report IDA International Development Association IPS Provincial Health Inspection (Inspection Provinciale de la Santé) IT information technology M&E monitoring and evaluation MCH maternal and child health The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) MoPHHSP Ministry of Public Health, Hygiene, and Social Protection (Ministère de la Santé Publique, de l’Hygiène et de la Prévoyance Sociale) MPA mininum package of activities NA not available / not applicable OOP out-of-pocket PAD Project appraisal document PBF performance-based financing PDO project development objective PDSS Health System Strengthening for Better Maternal and Child Health Results Project (Projet de Developpement du Secteur de la Santé) PHC primary healthcare PIU project implementation unit PMNS Multisectoral Nutrition and Health Project (Projet Multisectoriel de Nutrition et Santé) PNHF National Program for Border Hygiene (Programme National d’Hygiène aux Frontières) PP project paper PPA project preparation advance QALY Quality-adjusted life years RMNCAH-NUT reproductive, maternal, newborn, child, and adolescent health and nutrition RP restructuring paper SDR Special Drawing Rights SNAME National System for Pharmaceutical Supply (Système National d'Approvisionnement en Médicaments) SPA Service Provision Assessment SRP strategic response plan TF trust fund UHC Universal Health Care UNICEF United Nations Children's Fund USAID United States Agency for International Development USD United States Dollar WB World Bank WBG World Bank Group WDI World Development Indicators WHO World Health Organization XDR Special Drawing Rights The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) TABLE OF CONTENTS I. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES, AND COMPONENTS ......................................................... 1 II. OUTCOMES ............................................................................................................................................... 10 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ........................................................ 20 IV. WB PERFORMANCE, COMPLIANCE, AND RISK TO DEVELOPMENT OUTCOME ......................................... 25 V. LESSONS AND RECOMMENDATIONS...................................................................................................... 32 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ....................................................................................... 34 ANNEX 2. WB LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................................................... 55 ANNEX 3. PROJECT COST BY COMPONENT ....................................................................................................... 58 ANNEX 4. TRIANGULATION OF DATA AND IMPACT EVALUATION OF PBF .......................................................... 59 ANNEX 5. EFFICIENCY ANALYSIS ....................................................................................................................... 63 ANNEX 6. SUPPORTING INFORMATION AND DOCUMENTS ............................................................................... 69 ANNEX 7. LIST OF CONSULTED DOCUMENTS .................................................................................................... 71 ANNEX 8. BORROWER’S PROJECT COMPLETION REPORT - EXECUTIVE SUMMARY ............................................ 77 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) DATA SHEET @#&OPS~Doctype~OPS^dynamics@icrbasicdata#doctemplate BASIC DATA Product Information Operation ID Operation Name Health System Strengthening for Better Maternal and P147555 Child Health Results Project (PDSS) Product Operation Short Name Investment Project Financing (IPF) DRC -Health System Strengthening Project Operation Status Approval Fiscal Year Closed 2015 Original EA Category Current EA Category Partial Assessment (B) (Restructuring Data Sheet - 22 Partial Assessment (B) (Approval package - 18 Dec 2014) May 2024) CLIENTS Borrower/Recipient Implementing Agency Democratic Republic of Congo Ministry of Finance, Ministry of Health DEVELOPMENT OBJECTIVE Original Development Objective (Approved as part of Approval Package on 18-Dec-2014) The proposed project development objective is to improve utilization and quality of maternal and child health services in targeted areas within the Recipient's Territory. Current Development Objective (Approved as part of Restructuring Package Seq No 10 on 22-May-2024) To improve utilization and quality of maternal and child health services in targeted areas within the Recipient's Territory and, to provide an immediate and effective response to an eligible crisis or emergency s s s s s s s s s s s s s s s s s s s s i The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) @#&OPS~Doctype~OPS^dynamics@icrfinancing#doctemplate FINANCING Financing Source Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 660,000,000.00 618,018,723.00 648,958,809.95 IDA-59980 120,000,000.00 115,266,528.00 122,352,489.27 IDA-67050 121,000,000.00 121,000,000.00 120,999,978.00 IDA-55720 130,000,000.00 115,118,960.00 122,308,473.67 IDA-D4390 120,000,000.00 111,738,231.00 118,849,582.80 IDA-D0210 90,000,000.00 79,403,247.00 83,830,388.00 IDA-D6750 79,000,000.00 75,491,757.00 80,617,898.21 World Bank Administered 54,501,483.92 54,429,814.06 60,000,000.00 Financing TF-A5096 3,500,000.00 3,473,350.38 3,473,350.38 TF-A6945 10,000,000.00 4,528,133.54 4,528,133.54 TF-18375 6,500,000.00 6,500,000.00 6,500,000.00 TF-A4579 40,000,000.00 40,000,000.00 39,928,330.14 Total 720,000,000.00 672,520,206.92 703,388,624.01 RESTRUCTURING AND/OR ADDITIONAL FINANCING Amount Disbursed Date(s) Type Key Revisions (US$M) 01-Sep-2015 Portal 0.00 • Components • Results 31-Mar-2017 Portal 32.69 • Additional Financing • Loan Closing Date Extension • Results 02-Mar-2018 Portal 77.82 • Additional Financing 31-Oct-2018 Portal 165.28 • Reallocations 06-Dec-2018 Portal 164.36 • Reallocations • Development Objective • Results 27-Feb-2019 Portal 217.09 • Additional Financing • Procurement • Components 19-Aug-2019 Portal 276.36 • Disbursement Arrangements • Other Changes ii The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) • Components • Results 26-Feb-2020 Portal 430.58 • Reallocations • Institutional Arrangement • Additional Financing 11-Jun-2020 Portal 435.25 • Reallocations 03-Sep-2020 Portal 449.97 • Reallocations 07-Dec-2021 Portal 638.48 • Loan Closing Date Extension 24-Jun-2022 Portal 641.26 • Reallocations 30-Jun-2023 Portal 701.95 • Loan Closing Date Extension • Development Objective 22-May-2024 Portal 705.08 • Reallocations @#&OPS~Doctype~OPS^dynamics@icrkeydates#doctemplate KEY DATES Key Events Planned Date Actual Date Concept Review 09-Jun-2014 09-Jun-2014 Decision Review 10-Sep-2014 08-Sep-2014 Authorize Negotiations 13-Nov-2014 13-Nov-2014 Approval 18-Dec-2014 18-Dec-2014 Signing 20-Jan-2015 Effectiveness 27-May-2016 30-May-2016 ICR/NCO 31-Mar-2025 27-Mar-2025 Restructuring Sequence.01 Not Applicable 01-Sep-2015 Additional Financing Sequence.04 Not Applicable 31-Mar-2017 Additional Financing Sequence.05 Not Applicable 02-Mar-2018 Restructuring Sequence.02 Not Applicable 31-Oct-2018 Restructuring Sequence.03 Not Applicable 06-Dec-2018 Additional Financing Sequence.06 Not Applicable 27-Feb-2019 Restructuring Sequence.04 Not Applicable 19-Aug-2019 iii The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) Restructuring Sequence.05 Not Applicable 26-Feb-2020 Additional Financing Sequence.07 Not Applicable 11-Jun-2020 Restructuring Sequence.06 Not Applicable 03-Sep-2020 Restructuring Sequence.07 Not Applicable 07-Dec-2021 Restructuring Sequence.08 Not Applicable 24-Jun-2022 Restructuring Sequence.09 Not Applicable 30-Jun-2023 Restructuring Sequence.010 Not Applicable 22-May-2024 ICR Sequence.01 (Final) -- 26-Mar-2025 Operation Closing/Cancellation 30-Jun-2024 30-Jun-2024 @#&OPS~Doctype~OPS^dynamics@icrratings#doctemplate RATINGS SUMMARY Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest ISR RATINGS Actual Disbursements No. Date ISR Archived DO Rating IP Rating (US$M) 01 11-Feb-2015 Satisfactory Satisfactory 0.00 02 06-Aug-2015 Satisfactory Satisfactory 0.00 03 29-Feb-2016 Moderately Satisfactory Moderately Satisfactory 0.00 04 07-Sep-2016 Satisfactory Satisfactory 24.41 05 31-Mar-2017 Satisfactory Satisfactory 32.69 06 25-Oct-2017 Satisfactory Satisfactory 62.36 07 27-Jun-2018 Satisfactory Satisfactory 110.87 08 30-Jan-2019 Satisfactory Moderately Satisfactory 214.06 09 29-Oct-2019 Moderately Satisfactory Moderately Satisfactory 298.73 Moderately 10 03-Jan-2020 Moderately Satisfactory 338.30 Unsatisfactory iv The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 11 31-Mar-2020 Moderately Satisfactory Moderately Satisfactory 433.25 12 12-May-2020 Moderately Satisfactory Moderately Satisfactory 435.25 13 23-Oct-2020 Moderately Satisfactory Moderately Satisfactory 452.65 14 29-Jun-2021 Moderately Satisfactory Moderately Satisfactory 626.91 15 06-Jan-2022 Moderately Satisfactory Moderately Satisfactory 638.48 16 06-May-2022 Moderately Satisfactory Moderately Satisfactory 639.74 17 25-Jul-2022 Moderately Satisfactory Moderately Satisfactory 673.83 18 30-Jan-2023 Moderately Satisfactory Moderately Satisfactory 673.38 19 09-Aug-2023 Moderately Satisfactory Moderately Satisfactory 705.15 20 01-Apr-2024 Moderately Satisfactory Moderately Satisfactory 705.08 21 27-Jun-2024 Satisfactory Satisfactory 705.08 @#&OPS~Doctype~OPS^dynamics@icrsectortheme#doctemplate SECTORS AND THEMES Sectors Adaptation Mitigation Major Sector Sector % Co-benefits Co-benefits (%) (%) FY17 - Health FY17 - Health 100 0 0 Themes Major Theme Theme (Level 2) Theme (Level 3) % FY17 - Non- FY17 - Disease Control 10 communicable diseases FY17 - Child Health 25 FY17 - Human FY17 - Health System FY17 - Health Systems and Policies 25 Development and Strengthening Gender FY17 - Reproductive and 30 Maternal Health FY17 - Food Security 5 FY17 - Nutrition and Food Security FY17 - Nutrition 5 v The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) FY17 - Private Sector FY17 - Public Private Partnerships 10 Development vi The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ADM STAFF Role At Approval At ICR Practice Manager Trina S. Haque Francisca Ayodeji Akala Regional Director NA Daniel Dulitzky Global Director Timothy G. Evans Juan Pablo Eusebio Uribe Restrepo Practice Group Vice President NA Mamta Murthi Country Director Jan Walliser (acting) Albert G. Zeufack Regional Vice President Makhtar Diop Victoria Kwakwa ADM Responsible Team Leader Hadia Nazem Samaha Fatima El Kadiri El Yamani Co-Team Leader(s) none Michel Muvudi Lushimba ICR Main Contributor Christel M. J. Vermeersch I. PROJECT CONTEXT, DEVELOPMENT OBJECTIVES, and COMPONENTS A. CONTEXT AT APPRAISAL 1. At appraisal, the Democratic Republic of Congo was struggling with poor health outcomes, including high infant and maternal mortality rates and malnutrition, despite some progress in lowering under-five mortality rates. Gender inequality remained a significant issue, with women facing numerous health, economic, and social barriers. The fertility rate increased from 6.3 to 6.6 between 2007 and 2014, among the highest in the world, while the unmet need for family planning was estimated at 23 percent (DHS-2014). The country was not on track to meet Millennium Development Goals related to maternal and child health. Malnutrition remained a pervasive problem, contributing to high mortality rates among children and women. Additionally, the country grappled with a high burden of neglected tropical diseases, human immunodeficiency virus/ acquired immunodeficiency syndrome (HIV/AIDS), tuberculosis, and recurrent outbreaks of Ebola Virus Disease (EVD). Despite these challenges, there were some improvements, such as increased access to antiretroviral therapy, which reduced HIV-related mortality. 2. The Democratic Republic of Congo faced significant challenges in its health system. Public spending on health was only approximately US$2.6 per capita per year, one of the world's lowest levels of public expenditure (WDI). Despite a rapidly growing economy, the country struggled to mobilize domestic resources for health. With public funding for health at only 16 percent of current health expenditures, most current health expenditures were financed privately by households (46 percent) and financial and technical partners (40 percent) (WDI, 2014). Most primary healthcare (PHC) centers received no operational funding from the government and relied on user fees to cover the cost of health services. User fees in government health facilities (PHC and hospitals) were relatively high and comparable to the private sector, leading to financial barriers to care. In government health facilities, only a small share of health workers was on the official staff roster or received formal salaries (estimated at less than 10 percent of actual staff). Insufficient skills, overstaffing in urban areas, and understaffing in remote regions were additional issues. The Page 1 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) availability of medicines at an affordable cost varied across health facilities, and the average availability of essential drugs was estimated at around 30 percent. The private pharmaceutical market needed to be better regulated, and there was a high percentage of substandard or counterfeit medicines in the private market. Health service quality was generally low, structurally (poor infrastructure and equipment) and process-wise (low compliance with clinical care protocols where they existed). 3. At appraisal, the Democratic Republic of Congo belonged to the Harmonized List of Fragile Situations1 owing to its score on an institutional assessment and the presence of an international peacekeeping force. The latter originated in the 1999 Lusaka Ceasefire Agreement to monitor the peace process after the Second Congo War. To this date, the country remains on the list of Conflict-affected Situations. 4. Despite these challenges, the Democratic Republic of Congo had a rich experience piloting performance- based financing (PBF), a supply-side results-based financing (RBF) approach. These pilots were financed by multiple development partners, including the European Union (EU), the Catholic Organization for Relief and Development Aid (Cordaid), the United States Agency for International Development (USAID), Memisa, and the World Bank (WB) under the Health Sector Rehabilitation Project (P088751). In PBF, a public entity acting as the “purchaser” would sign contracts with health facilities, agreeing to pay facilities based on the quantity and quality of services delivered. Contracts would specify the type of service to be provided and unit rates (e.g., US$4 per assisted childbirth), as well as the quality indicators and their impact on the payments (e.g., prorated payments based on a facility’s score on a quality checklist). In 2014, the government deemed the pilots sufficiently successful and decided to roll out PBF as part of its health financing strategy. Project Development Objective (PDO) 5. The original PDO was to improve the utilization and quality of maternal and child health (MCH) services in targeted areas within the Recipient’s territory. 6. The originally targeted areas consisted of 140 health zones (HZ) in four provinces from the pre-2016 administrative division of the country: Equateur (58 HZ), Bandundu (52 HZ), Maniema (14 HZ), and Katanga (16 HZ). These areas included 17 million people or 23.5 percent of the country’s population. Following the administrative reforms of 2015, this corresponded to the following 11 new provinces: Equateur, Mongala, Sud-Ubangi, Tshuapa (formerly Equateur), Haut-Katanga, Haut-Lomami, Lualaba (formerly Katanga), Kwango, Kwilu, Maï-Ndombe (formerly Bandundu), and Maniema. Theory of Change (Results Chain) 7. The Project aimed to achieve its objective by scaling up the PBF approach in its target areas and investing in strengthening policy and coordination functions. The original PAD did not include an explicit theory of change diagram as it was not required at the time. Therefore, the theory of change was developed by this report from Project documentation (Annex 1, Figure 5). PBF aimed to increase the utilization and quality of MCH services by introducing performance contracts and payment mechanisms that directly rewarded facilities for the quantity and quality of services provided. Key MCH services included prenatal care, delivery and postnatal care, vaccination, child growth monitoring, contraception, HIV testing, and curative visits. The PBF payments did not replace any existing funding stream and, therefore, represented a net increase in government cash flow to facilities, often from virtually zero. Contracts allowed health facilities to use up to 50 percent of their revenues (including user fees) for staff payments. 1 https://thedocs.worldbank.org/en/doc/3d4356ac2aee9f0b2db90ae9ce49f639-0090082024/original/FCSList-FY06toFY24.pdf https://thedocs.worldbank.org/en/doc/373511582764863285-0090022020/original/FCSHistorialnote.pdf Page 2 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) Unlike in other countries where health workers receive salaries and PBF generates top-up payments to staff, in this case, most health staff in PHC facilities do not receive publicly funded wages. The remaining PBF funds were destined to improve infrastructure, buy medicines, inputs, and equipment, and cover operational costs. In exchange for the funding, health facilities agreed to charge “flat out-of-pocket tariffs” for the most common services, which were generally below the OOP fees previously charged to patients. In addition, the PBF fund committed to paying these flat tariffs (resulting in a total fee exemption) for up to 10 percent of the patients identified as the poorest within the facility’s catchment area. These two measures aimed to improve the financial accessibility to health services by reducing user fees in contracted facilities. 8. Besides improving the utilization and quality of MCH services, PBF would try to address underlying health system constraints by improving health financing, policy, and service delivery capacities at various levels of the health system pyramid. PBF was thought to help improve: a. motivation and distribution of human resources (through a reliable stream of income), b. health worker skills (through in-service training, coaching, and supervision), c. availability of quality and affordable medicines (through improved supply chains and local purchasing), d. community engagement (through involvement in health facility business plans, patient satisfaction surveys, and monitoring of PBF), e. better supervision by central, provincial, and health-zone-level authorities (through training, resources, performance contracts, and new tools). 9. The Project incorporated an impact evaluation that sought to estimate the impact of PBF by comparing it with an alternative approach labeled as direct facility financing (DFF). One hundred health zones were randomly assigned either to the PBF or DFF intervention (50 zones each). DFF health zones and facilities received equivalent financial transfers to PBF health zones and facilities; however, the transfer amount was not conditioned on their performance evaluation. The comparison between PBF and DFF was meant to isolate the incentive effect from the income effect of PBF. PDO indicators and targets TABLE 1: PDO INDICATORS AND TARGETS PDO Indicator Baseline Target ICR reference2 1. Percentage of women having at least three antenatal care visits before 29.3 39 PDO 1 delivery (%) (Percentage, Custom) 2. Percentage of children aged between 6-23 months receiving preventive 26.9 35 PDO 2 nutritional services at least four times per year (Percentage, Custom) (later dropped) 3. New curative consultations per capita per year (Text, Custom) 0.3 0.6 PDO 3 4. Percentage of children fully immunized (%) (Percentage, Custom) 37.4 55 PDO 7 5. Average score of the quality checklist at the health center level (%) 47.5 65 PDO 8 (Percentage, Custom) 2 The order and numbering of the PDOs differs across the Project documents. The ICR introduces its own numbering for easier reference throughout the document. Page 3 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) Components Component 1: Improve Utilization and Quality of Health Services at Health Facilities through PBF (Original allocation: US$120 million; revised allocation US$226.6 million; actual spending US$348.6 million) 10. Component 1 financed the purchase of two services packages for the beneficiary population: The minimum package of activities (MPA) to be provided in public and non-profit PHC facilities and the complementary package of activities (CPA) to be provided in district hospitals. The MPA and CPA responded to critical factors in the country’s burden of disease. Health facilities signed contracts with provincial purchasing agencies (see below in Component 2). These contracts provided for performance-based incentives to be paid to the facilities based on (i) increased population utilization of MPA/CPA services; (ii) quality scores that measured primarily structural improvements (such as cleanliness, availability of drugs and commodities, equipment, etc.); (iii) compliance with contractual terms (such as avoiding significant discrepancies between reported, verified, and counter-verified service data, elaborating a business plan, and respecting fund utilization restrictions). Component 2: Improve Governance, Purchasing, and Coaching and Strengthen Health Administration Directorates and Services through PBF (Original allocation: US$65.2 million; revised allocation US$102.7 million; actual spending US$127.8 million) 11. Component 2 strengthened and financed the institutions tasked with coordinating and purchasing services from health facilities. At the provincial level, the direct purchasers of services were provincial purchasing agencies set up as hybrid public-private service institutions (Etablissement d’Utilité Publique; EUP). These EUPs were responsible for negotiating and signing contracts for primary and secondary health facilities, verifying the invoices submitted by health facilities, submitting the invoices to the central level for payment, mobilizing communities, and coaching health facilities. At the central level, the Planning Directorate (Département d’Etude et de Planification; DEP) was appointed as the PBF fund-holder and tasked with coordinating activities with the technical units of the Ministry of Public Health, Hygiene and Social Protection (MoPHHSP) such as the PBF Technical Unit (Cellule Technique Financement Basé sur les Résultats; CT-FBR). The Health Management Information System (HMIS) Directorate was responsible for the PBF web- enabled database application. Component 2 introduced performance contracts at all levels of the health system, including for the EUPs, the Provincial Health Directorates (Direction Provinciale de la Santé; DPS), Health Zone Teams (Equipe Cadre de Zone de Santé; ECZS), Regional Distribution Centers of drugs and supplies (Centrale de Distribution Régionale; CDR), DEP, CT-FBR and HMIS directorate. These contracts held institutions accountable for their results and incentivized them. 12. The component design emphasized local engagement and citizen participation to improve governance and results. Local Committees were involved in the performance-based approach through their participation and oversight in (i) determining lists of indigent patients eligible for fee exemptions, (ii) planning health activities, developing the health center business plan, and monitoring fund utilization by the health facility toward the business plan targets, (iii) discussing and negotiating user fee levels with the heads of health facilities, and (iv) household visits to verify the existence of users and assess patient satisfaction. Component 3: Strengthen Health System Performance Financing, Health Policy, and Surveillance Capacities (Original allocation: US$41.3 million; revised allocation US$140.3 million; actual spending US$38.0 million) 13. Component 3 included three subcomponents of health system strengthening, project management, M&E, and communications. Page 4 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 14. Sub-component 3.1: Support to Improve Health Financing Reform and System Decentralization included: a. Strengthening capacity at the central and provincial levels: DEP, the Primary Health Services Directorate, the HMIS Directorate, the CT-FBR, the Human Resources Directorate, the Resource Management Unit (Cellule d’Appui et de Gestion Financière; CAGF), the Procurement Unit (Cellule de Gestion en Passation de Marchés Publics; CGPMP), the M&E unit, and provincial entities. b. Supporting the ongoing decentralization reforms by introducing a single contract (contract unique) and performance framework between each of the 26 DPSs and health sector financiers. These contracts aimed to improve coordination, transparency, and accountability, and reduce funding fragmentation. c. Supporting the policy dialogue on human resources management reform. d. Improving the quality of health care (QoC) by (i) developing a National QoC Improvement Strategy and operational plan, (ii) disseminating clinical practice guidelines and protocols, (iii) developing and scaling up quality assurance and continuous quality improvement modalities, and (iv) strengthening the capacity of professional associations (doctors, nurses, lab technicians) to self-regulate and improve QoC among its members. 15. Sub-component 3.2: Health Sector M&E Strengthening and Project Management. This sub-component supported the HMIS Directorate and CT-FBR in the rollout of the DHIS2 and PBF information technology (IT) platforms. The web-based DHIS2 platform started replacing the former MS-Access-based HMIS in 2013, and the sub-component aimed to complete the transition. Meanwhile, the web-based PBF-IT platform was needed to manage PBF in the Project’s target provinces. In addition, the sub-component aimed to support the interoperability of web-based IT systems and the development of an integrated e-health architecture. The combination of DHIS2, the PBF-IT platform, and an open data layer was part of an effort to set up a data-driven health system, including a website that published the results verified and purchased with PBF, both on the service delivery and administrative side. Benchmarking the quantity and quality of services provided by health facilities and the public health administration was seen as a tool to encourage better governance. 16. Sub-component 3.3: Ebola Preparedness Plan aimed to ensure the country’s preparedness for a possible EVD (or other infectious disease) outbreak by developing a nationwide protocol, providing essential equipment, supplies, drugs, and vehicles, and carrying out additional community-level activities. Close collaboration with UN agencies and international NGOs was envisaged. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 17. While the Project started out with an objective to improve the utilization and quality of maternal and child health services for approximately 25 percent of the country’s population over 5 years, at closure, the Project had considerably expanded: it had an implementation period of over 10 years, an expanded geographic scope that covered 33 percent of the population, and it had functioned as the main conduit for WB support to multiple Ebola Virus Disease (EVD) outbreaks. With four rounds of additional financing (AF), the financial envelope of the Project almost tripled from US$226 million to US$725 million and the scope of activities changed considerably, as detailed below. The WB’s health portfolio in the DRC also changed significantly during the Project’s life. Initially, the PDSS and the regional Great Lakes Emergency Sexual and Gender-Based Violence (SGBV) and Women’s Health Project (P147489, approved FY14) were the only active health projects.3 The PDSS’s successful implementation and disbursement meant it was repeatedly tapped to address urgent needs such as community-based nutrition and EVD response. However, the WB also financed more focused projects such as the Gender-based Violence (GBV) Prevention and Response Project (P166763, approved August 2018), the REDISSE IV Project (P167817, approved October 2019) to help strengthen surveillance and 3 A Human Development Project, the DRC Human Development Systems Strengthening (P145965, approved March 2016), also included investments in the health sector. Page 5 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) emergency response, the DRC COVID-19 Strategic Preparedness and Response Project (SPRP) (P173825, approved April 2020), and the Multisectoral Health and Nutrition Project (PMNS) (P168756, approved March 2021). Revised PDO 18. The PDO was revised with the second AF and restructuring, approved on February 19, 2019, to reflect the activation of the contingent emergency response component (CERC) introduced in 2017 with zero allocation. The revised PDO was “To improve utilization and quality of maternal and child health services in targeted areas within the Recipient's Territory and, to provide an immediate and effective response to an eligible crisis or emergency.” Change in targeted areas 19. The targeted areas of the Project were changed three times. In March 2017, the Project added 16 new HZs in the former Katanga province, reaching 156 health zones. In 2018, the Project added health zones in Nord-Kivu to the PBF scheme to support the EVD response. Financing came partly from PDSS and partly from the Great Lakes Emergency SGBV Project, though achievements and population in Nord Kivu were incorporated into the PDSS results framework. In 2020, the Project dropped the HZs in the (new) province of Equateur and added the province of Kinshasa. In 2022, the Project dropped the province of Kwilu as it became part of the newly launched PMNS. Revised components 20. In March 2017, Component 3 gained a new sub-component, and a new Component 4 was added. Component 3: Strengthen Health System Performance - Financing and Health Policy (March 2017 allocation: US$ 110.3 million; actual spending US$ 38.0 million) a. Sub-component 3.3: Retirement Reform Program: The sub-component included (i) supporting the retirement program for the MoPHHSP; (ii) supporting the organizational reform of the MoPHHSP by implementing a newly revised organizational chart; and (iii) supporting the management of the retirement program. Component 4: Disease surveillance Strengthening and Response (March 2017 allocation: US$244.9 million; actual spending US$181.4 million) a. Sub-component 4.1 Disease Surveillance Strengthening: This sub-component included all activities on surveillance that were already implemented under the former sub-component 3.3. Ebola Preparedness Plan, while remaining activities were moved to the new Regional Disease Surveillance Systems Strengthening Phase IV (REDISSE IV) Project (P504532). b. Sub-component 4.2 CERC: This new sub-component included all EVD 9 response activities. Revisions in the scope and financing of the Project, including rationale. 21. Summary. As mentioned above, the Project’s scope was modified several times, and its closing date was extended by 4.5 years. (Figure 1) Between Project effectiveness in May 2016 and early 2018, two AFs expanded the Project’s targets and its technical scope, introducing new areas of focus for the PBF, such as community-based PBF, nutrition, adolescent health, and GBV. However, a turning point came in May 2018, when the Democratic Republic of Congo declared the ninth EVD outbreak (EVD 9), and the Project’s CERC was activated to help the country respond to the epidemic. In the following two years, the Project was tapped several times to respond immediately to EVD 9, EVD 10, and EVD 11. Funds from the PBF components were reallocated to three CERCs and later replenished through AF 3 Page 6 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) and 4. While waiting for AF 3 and 4, the reassignment of funds to the Ebola response generated a funding crunch for Components 1 and 2. The envisaged expansions into community-based PBF, nutrition, adolescent health, and GBV were put on hold while efforts moved to control EVD. By 2020, it became clear that the Project's expanded scope envisaged before the EVD outbreaks was no longer feasible. The Project was restructured in February 2020 and its scope was reduced. The strategy for nutrition interventions changed, and the PMNS became its main source of financing. Finally, cost overruns and the impact of the COVID-19 pandemic necessitated a scale-back of the Project scope and additional funds from the fourth and final AF. The details under each restructuring and AF are provided below. FIGURE 1: TIMELINE OF MAIN PROJECT MODIFICATIONS 22. In March 2017, AF1 (US$163.5 million equivalent) expanded the Project’s scope as follows: (i) Scale-up of the package of essential HNP services contracted through PBF at the health facilities, such as adolescent health; (ii) community-based contracting for primary health service delivery, focusing on nutrition, (iii) activities to prevent and address GBV, (iv) retirement benefits for health workers, (iv) additional disease surveillance and response activities, (v) expansion of the geographical scope to 16 new health zones in the former Katanga province. The accompanying restructuring (i) added a new Sub-component 3.3 on the Retirement Reform Program, (ii) moved the disease surveillance activities under Component 3 of the original project to a new Component 4 (Disease Surveillance Strengthening and Response) with two sub-components: 4.1 Disease Surveillance Strengthening, and 4.2 CERC, and (iii) extended the project closing date by 24 months to December 31, 2021. The results framework was significantly expanded. One PDO indicator was changed to reflect a WB corporate results indicator (CRI), “People who have received essential health, nutrition, and population services (number).” Eight new intermediate indicators were added to reflect the AF1 activities. The targets of the original indicators were revised to reflect the increase in the number of beneficiaries and the new Project closing date. 23. In March 2018, AF2 (US$10 million) increased the envelope for PBF subsidies under Component 1. Results indicator targets were increased to reflect this increase in subsidies: (i) People who have received essential Health, Nutrition, and Population services; (ii) Average score of the quality checklist at the health centers; (iii) Health facilities receiving Client Tracer and Satisfaction Survey feedback; and (iii) Percentage of pregnant women counseled and tested for HIV. Page 7 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 24. In May 2018, the WB activated the CERC to respond to EVD 9. The Government subsequently declared EVD 10 in August 2018, and a formal restructuring in December 2018 formally reallocated US$80 million from components 1, 2, and 3 to sub‐component 4.2 (CERC) to respond to EVD 10. 25. In February 2019, AF3 provided US$120 million to fill financing gaps in the project. AF3 was used to replenish US$80 million that was previously reallocated from components 1, 2, and 3 to sub‐component 4.2 (CERC). This replenishment aimed to allow the Project to achieve its PDO and targets set in the results framework. AF3 also added US$40 million in financing to Sub‐component 4.2 (CERC) for emergency response activities to stop EVD 10. In the August 2019 restructuring, US$80 million was reallocated to CERC 2 and 3 disbursement categories while awaiting the effectiveness of AF3. 26. The February 2020 restructuring responded to several of the findings from the July 2019 Mid-term review by adjusting the Project’s technical design:4 (i) revised the Project’s results framework to ensure consistency between data sources used for target setting and quarterly monitoring; (ii) stopped financing PBF in Equateur province and reallocated this financing to the remaining provinces and Kinshasa; (iii) reduced the technical scope of the Project, by removing community-based nutrition, GBV, and disease surveillance activities, as these activities had not started under PDSS and were covered by new HNP projects in the country (PMNS, Gender-based violence Prevention and Response Project, and REDISSE IV); (iv) revised the Project’s institutional arrangements by moving fiduciary and procurement functions to a PIU; and (v) reallocated funds back to Components 1 and 2 following the effectiveness of AF3. The strategy for nutrition interventions changed: instead of a broad approach in many provinces as envisaged under PDSS, the Government decided to focus on nutrition-related efforts in the most highly affected areas using a more multisectoral strategy under the PMNS. In addition, a new GBV Prevention and Response Project (approved August 2018) rolled out GBV-focused activities in Nord and Sud Kivu, Maniema, and Tanganyika, where this issue was particularly elevated. 27. In May 2020, AF4 (US$200 million equivalent) covered financing gaps in the PBF activities and in the EVD 10 response. It allocated US$150 million to PBF activities (US$108 million financing gap and US$42 million cost overrun) and US$50 million to the CERC for the EVD 10 response (cost overrun). The Project paper argued that a cost overrun for PBF activities resulted from the socio-economic impact of COVID-19 on households, which financed a large share of PHC costs out-of-pocket. The Project paper rationale for the US$42 million was to (i) increase subsidies to the 2,800 health centers contracted by the PDSS in the Provinces already covered by the Project so that financial access would not deter care-seeking5; (ii) purchase essential drugs and vaccines to prevent stockouts; (iii) improve the infection prevention control and water and sanitation aspects of the health facilities to reinforce quality of care; and (iv) finance GBV interventions as per the GBV action plan developed under the safeguard documents. A second cost-overrun for EVD 10 response resulted from the unexpected length (over 21 months) and geographic spread of EVD 10 across three provinces in a fragile and conflict-affected setting. The response costs were initially estimated in early 2019 to inform AF3 but underestimated the need to expand treatment centers, reinforce points-of-entry, and strengthen 4 During the mid-term review in 2019, the team determined that the key performance indicators were moving in a positive direction toward the PDO and that the project had surpassed four of six PDO indicators. However, the rating was downgraded during the mid-term review from Satisfactory to Moderately Satisfactory to reflect methodological issues identified in quarterly results reporting. It appeared that results were reported quarterly using data from the National HMIS. The team noted large increases or decreases in the project indicators during the first quarter of the project, which was most likely due to the way in which the indicators were calculated, and not associated with project activities and impact. The exact calculations made to set the baselines had not been recorded, and therefore the results indicators (including the baseline and targets) had to be re-calculated using the same HMIS data that were reported quarterly. 5 Subsidies did not appear to have “increased” –rather, PBF payments to health facilities were continued for an additional two years, thereby continuing to suppress OOP fees as evidenced in the impact evaluation. Page 8 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) community surveillance. AF4 allowed the Project to support the Government’s Strategic Response Plan (SRP) 4.1 through September 2020. 28. Four further rounds of restructuring were carried out to reallocate funds and extend the Project closing date. The August 2020 restructuring reallocated funds from the disbursement category on retirement benefits to core activities. In December 2021, the Project closing date was extended by 18 months to June 30, 2023. Purchasing of services continued until June 2023 and was then discontinued as Project funds were exhausted. In June 2023, the Project closing date was extended by 12 months to June 30, 2024, to allow the completion of all construction work as well as contracts for the delivery of supplies and equipment. Revised PDO Indicators and outcome targets 29. To the original five PDO indicators, another five were added at different points in the Project, and one of the new indicators was eventually dropped. Eight out of the ten PDO indicators saw changes in their baselines and targets (Table 2). Details of the changes can be found in Annex 1 and in the monitoring and evaluation section. TABLE 2: REVISED PDO INDICATORS ICR # Indicator Origin Baseline Target modified? Substantial modification in modified? the definition? PDO 1 Women having at least 4 antenatal care Parent Twice: Twice: Yes: visits before delivery (%) 29.3 to 57 to 36 39 to 65 to 55 4 visits instead of 3 (Percentage, Custom) PDO 2 Percentage of children aged between Parent No: Yes: Yes: DROPPED 6-23 months receiving preventive 26.9 35 to 45 % of children under 24 nutritional services at least four times months participating in the per year. Replaced with Replaced with Growth Monitoring and (Number, Custom) IRI, then IRI, then Promotion activities at dropped dropped community level PDO 3 New curative consultations per capita Parent Twice: Once: No per year 0.3 to 0.25 to 0.6 to 0.5 (Number, Custom) 0.38 PDO 4 People who have received essential AF1 Twice: Thrice: Yes: health, nutrition, and population (HNP) 0 to 5,208,170 14,350,000 to Cumulative to annual services 25,000,000 to (Number, Corporate) 7,150,000 to 7,819,376 PDO 5 Number of children immunized AF3 Once: Once: Yes: (Number, Corporate Breakdown) 0 to 434,750 1,250,000 to Cumulative to annual 1,063,706 PDO 6 Number of women and children who AF3 Once: Once: No, have received basic nutrition services 0 to 4,352,206 4,200,000 to but the chosen target and (Number, Corporate Breakdown) 5,505,670 baseline were not in line with the indicator definition or the CRI guidelines. PDO 7 Number of deliveries attended by AF3 Once: Once: No skilled health personnel 0 to 421,214 1,7000,000 to (Number, Corporate Breakdown) 1,250,000 PDO 8 Children Fully Immunized (%) Parent Twice: Twice: Yes: Set of vaccines and (Percentage, Custom) 37.4 to 54 to 62 55 to 65 to 70 calendar Page 9 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR # Indicator Origin Baseline Target modified? Substantial modification in modified? the definition? PDO 9 Average score of the quality checklist at Parent Thrice: Twice: No the health centers (%) 47.5 to nil to 20 65 to 55 to 60 (Percentage, Custom) to 21 PDO 10 Eligible individuals vaccinated during AF3 No: No: No EVD (# - cumulative) 0 120,000 (Number, Custom) Implications of the revisions on the original Theory of Change 30. AF2 represented a significant change in the theory of change of the PBF model by extending the model's reach to community-based interventions and attempting to engage community health workers. This was thought to be particularly helpful in addressing malnutrition challenges, which are difficult to address with a health-facility-based approach. However, this complementary approach was not actually deployed as Project funding was reallocated to EVD response, and the Government eventually decided on a different approach to tackle malnutrition under PMNS. The other revisions were the result of the CERC activations to respond to EVD 9 and 10. The response had its own logic and theory of change, as illustrated in Annex 1, Figure 6. II. OUTCOMES 31. Justification for using a split rating methodology in assessing project outcomes. Up to 2020, the Project’s technical scope was expanded three times with AF1, AF2, and AF3, as discussed above. AF1 (2017) and AF2 (2018) introduced retirement benefits for civil servants, nutrition, gender-based violence, disease surveillance, new approaches such as community-based PBF, and higher targets on existing indicators. However, the retirement of civil servants did not start due to changes in policy priorities, and the activities on nutrition, community-based PBF, and GBV did not have funding due to the reassignment of Project funds to the EVD response after the CERC activations in 2018 and 2019. AF3 (2019) formally expanded the Project to include EVD response. During the February 2020 restructuring, activities on the retirement of civil servants, nutrition, community-based PBF, GBV, and disease surveillance were formally dropped from the Project’s technical scope. Disease surveillance activities (originally under component 4.1) were reassigned to the REDISSE IV project. The reduction in scope was evident in the results framework, as most nutrition-related indicators were dropped. The efficacy of the Project is therefore assessed in two Phases: Phase I from the effectiveness of the Project until February 2020, and Phase II from February 2020 to closing in June 2024. A. RELEVANCE OF PDO 32. The Bank’s current Country Partnership Framework (CPF) was approved on January 24, 2022, for the period FY22-26 (Report No. 168084-ZR). Its second focus area (out of 3) is “Strengthen systems for improved service delivery and human capital development.” The CPF proposed a shift toward the human development sectors compared to the previous heavier emphasis on rebuilding public infrastructure, with additional focus on the education and social protection sectors, which were seen as lagging behind the health sector. The Project is aligned with the current CPF as follows: a. The CPF's objective is to strengthen systems for improved service delivery and human capital development, which includes improved health service delivery and health outcomes. Such improvements in health service delivery encompass both the quantity and quality of services and their impact on health outcomes. Therefore, the CPF Page 10 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) objective encompasses the Project’s objective to improve the utilization and quality of maternal and child health services. b. The CPF focuses on addressing drivers of fragility and conflict, which include recurrent outbreaks of communicable diseases. The Project’s second objective (to provide an immediate and effective response to an eligible crisis or emergency) is therefore in line with the CPF objectives. The Project supported the response to the EVD 10 outbreak, whose hotspots were Nord-Kivu and Ituri, which are historically two of the most fragile and conflict-affected areas in the country (Annex 5, Figure 9). c. In health, the CPF targeted a five percent expansion in routine vaccination, strengthened epidemiological surveillance, and access to sexual and reproductive health services. These objectives are matched by PDO and intermediate targets in the Project: the number of children vaccinated, the number of assisted deliveries, access to contraceptive services, HIV counseling of pregnant women, EVD case follow-up and vaccination, etc. d. The CPF generally outlined a readjustment of the World Bank Group (WBG) portfolio for investments to be geographically concentrated in areas that were “affected or have been affected by violence and/or conflict; areas that may potentially see increased levels of conflict or violence; and areas where refugee inflows lead to significant pressures on host populations.” However, the CPF did not identify a geographic focus for HD engagements, rather noting that the Project would address the Democratic Republic of Congo’s drivers of fragility by “extending basic health services to all people in [the Democratic Republic of Congo], thereby extending the reach of social services and building greater trust in the country’s health system.” The CPF proposed to extend the Project by 2 years. e. In terms of technical approaches, the CPF proposed to use (i) results-based modalities and (ii) strengthening decentralized administrative structures in the social sectors where possible, which is in line with the approach chosen by the Project several years prior. 33. Considering the factors above, the relevance rating of the PDO is High. B. ACHIEVEMENT OF PDOS (EFFICACY) 34. The PDO statement contains three objectives, which the ICR rates separately, across two Project phases: (1) to improve the utilization of maternal and child health services; (2) to improve the quality of maternal and child health services; and (3) to provide an immediate and effective response to an eligible crisis or emergency. 35. Overall, the ICR finds significant improvements in the utilization and quality of services and an effective response to the EVD emergency. The ICR bases this conclusion on indicators from the results framework (which are based on administrative data sources) complemented and triangulated with survey-based indicators. This triangulation counterbalances a possible overstatement of Project achievements due to the reliance of the results framework on HMIS (administrative) data. The ICR also adds new indicators to complement the results framework indicators in areas such as financial protection (which drives utilization) and process aspects of quality of care. 36. Before assessing actual outcomes, it is important to explain about the changes made in the results-framework with regards to indicators, baseline and target values. Some changes were due to the modifications in Project target areas (dropping Equateur and adding Nord Kivu and Kinshasa). Other changes were due to modifications in the data sources. Many original baseline values were extracted from the incipient HMIS system and updated in the February 2020 restructuring when the system became more complete and reliable. When changes were due to improvements in measurement, correction of errors, or changes in the source of data, the ICR retains the most consistent set of values over the entire Project. The following indicators received differential achievement ratings for Phase I and Phase II of the Project: Page 11 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) • PDO 2 and IRIs 17, 19, and 20 (nutrition) were dropped in the February 2020 restructuring. The ICR evaluated them against their final achievement and took them into account for the evaluation of Phase I. • PDO 8 (curative consultations) and IRI 3 (HIV testing of pregnant women) had target changes that were adjusted downward in the February 2020 restructuring. The ICR uses the higher target value for the Phase I rating and the lower target value for the Phase II rating. • IRI 8 (retirement of civil servants) remained in the results framework till closing, although the corresponding financing was reassigned in the February 2020 restructuring. The ICR only considers IRI 8 in Phase I. • IRI 11 (performance of Provincial Health Directorates) was introduced in the February 2020 restructuring. The ICR only considers this indicator for Phase II. 37. The ICR introduces changes and corrections to the results framework to align it with CRI definitions and ensure the internal consistency of indicators. A complex change relates to PDO 6, which (as a CRI) was meant to measure the cumulative number of children aged 6-23 months who received growth monitoring visits, but instead, it measured the annual number of growth monitoring visits for children aged 0-23 months. Although the number of growth monitoring visits does not qualify as a CRI, it is a relevant measure of the Project’s achievement, and, therefore, the ICR creates a new PDO 6a indicator to capture the baseline, target, and reported achievements for these visits. The ICR recomputes PDO 6 following the CRI breakdown guidelines by estimating and summing (i) the number of women receiving iron supplementation during pregnancy and (ii) the number of children under five who received preventive care visits. While the Project’s achieved value (numerator) can be calculated in this manner, there is no basis for retrofitting a target for PDO 6. Without a denominator, the percentage achievement for PDOs 6 and 4 cannot be computed. 38. The ICR makes the following additional changes: (i) recomputing PDO 4, 5, and 6 to provide cumulative numbers as specified in the CRI guidelines; (ii) resolving contradictions in the baselines and targets for PDO 7 and IRI 5 (deliveries) and PDO 5 and PDO 8 (child vaccination); (iii) removing boys from the denominator of IRI 4 (adolescent girl contraception); (iv) fixing the inconsistency in the targets for IRIs 1, 2, and 3 (contraceptive use); (v) dropping IRI 9 (poor people benefitting from fee exemption mechanisms (%)) which did not relate to poor people and replacing it with two survey-based indicators; (vi) reinstating IRI 18 (HAT patients) (dropped in the February 2020 restructuring) since PBF continued to pay health facilities for the diagnosis and treatment of HAT patients in Phase II and the target was surpassed in 2023. Details of the corrections can be found in Annex 1. 39. The ICR enriches the results framework with additional indicators that measure financial access to care and quality of care (Table 3). These additions are meant to capture the impact of the Project on aspects of financial access to care and quality of care that were not amenable to measuring with administrative data. Financial access is measured through reductions in out-of-pocket (OOP) fees for child curative care and ANC, extracted from the IE surveys (IRIs 22 and 23). The quality-of-care measurement is enriched through two indicators (IRI 24 and 25) on the availability of essential drugs in PHC centers and hospitals. These two additional measures also counter some weaknesses in the measurement of IRI 12 (outlined in Annex 1). The ICR also introduces a new IRI 26 to measure the physical infrastructure updates carried out during Phase II of the Project. Finally, the ICR introduces three new measures of the content of antenatal and reproductive health care (IRI 27, 28, 29). 6 In the Donabedian framework for measuring quality of care along the structure/process/outcome dimensions, these three measures correspond to the process dimension, which was not previously reflected in the results framework. 6 It is important to note that these indicators could not have been part of the original results framework since they were only measured twice as part of the IE. Their absence from the original design should therefore not be construed as a shortcoming. Attribution questions and the interpretation of IE data are discussed in Annex 4. Page 12 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) TABLE 3: NEW IRIS INTRODUCED IN THE ICR # and name of the indicator Source of data Justification Target set by ICR IRI 22 NEW – Reduction in OOP payment IE household Replaces IRI 13. 50 percent for child curative consultation in PHC surveys Measure increased financial access to reduction from centers care not measured through any baseline in real existing indicator. terms IRI 23 NEW – Reduction in OOP payment IE household Replaces IRI 13. 50 percent for ANC in PHC centers surveys Measure increased financial access to reduction from care not measured through any baseline in real existing indicator. terms IRI 24 NEW - Availability of essential drugs IE facility Survey-based data to complement IRI Same as IRI 12, i.e. 6 in PHC centers (22 items) surveys 9 and triangulate PDO 9. pp. improvement. IRI 25 NEW – Availability of essential drugs IE facility Survey-based data to complement IRI Same as IRI 12, i.e. 6 in hospitals (22 items) surveys 9 and triangulate PDO 9. pp. improvement. IRI 26 NEW – Number of health facilities Project Measures improvements in physical 175 rehabilitated through a decentralized documentati infrastructure of health facilities. contracting modality at the EUP level on IRI 27 NEW – Percentage of women who IE household Measures the timeliness of ANC (an 50 percent received skilled ANC during the first surveys aspect of quality) using survey data. improvement over trimester baseline (15.3) IRI 28 NEW – Percentage of women who IE household Measures the content/quality of ANC 50 percent received antimalarial medicines during surveys using survey data. improvement over pregnancy baseline (44.5) IRI 29 NEW – Percentage of women (who IE household Measures the content/quality of 50 percent gave birth within the last two years) who surveys reproductive care using survey data. improvement over discussed family planning with a health baseline (44.8) care provider TABLE 4: NUMBER OF INDICATORS USED IN THE ICR, BY PDO PART PDO part 1 PDO part 2 PDO part 3 Number of PDO indicators 8 1 1 Number of IRIs 9 14 3 Total 16 15 4 Note: The full list of indicators and their level of achievement can be found in Annex 1, Table 8. PDO Part 1 “to improve the utilization of maternal and child health services”: Phase I: Modest, Phase II: Substantial 40. PDO Part 1 is assessed using eight PDO indicators. PDO 2 only applies to Phase I while achievement of PDO 3 is rated separately for phases I and II. The other six PDO indicators do not have a differential achievement rating between phases I and II. Three indicators (PDO 1 - four prenatal care visits, PDO 6a – nr. of growth monitoring visits, PDO 7 – skilled delivery) are surpassed, one indicator (PDO 5/8 –children immunized) is substantially achieved, one indicator (PDO 3 – curative consultations) is partially achieved in Phase I and fully achieved in Phase II, one indicator (PDO 2 – nutritional services) is not achieved in Phase I and dropped in Phase II, and two indicators (PDO 4 - essential Page 13 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) HNP services, and PDO 6 - basic nutrition services) cannot be evaluated, as explained above. Of the nine IRIs considered in the ICR, four are surpassed (IRI 5 – skilled birth attendance, IRI 7 – single contracts, IRI 22 and 23 – OOP payments), two are substantially achieved (IRI 4 – adolescent contraception, IRI 6 – health services) are fully achieved, and two are not achieved in Phase I and dropped in Phase II (IRI 8 - retirement of civil servants, IRI 17 – family participation in nutrition sessions). 41. Triangulation. The result framework indicators that capture the number of services provided were extracted from the HMIS, which can change due to the level of implementation of the system. The fact that PBF tied payments to the reported number of services (albeit after verification) also provided incentives for overreporting. Finally, population-based denominators in the results framework use official population estimates derived from the last census in 1984. Therefore, they could be significantly over or underestimated and lead to apparent high coverage rates that may not be confirmed in survey data. Therefore, the ICR triangulates the results with survey-based data, including the Demographic and Health Surveys (DHS) (2014 and 20237), and the IE surveys (2026 and 2021). PDO 1 (4 ANC visits) was triangulated using the DHSs, which shows a slight increase in 4 ANC visits from 49% to 52% in 2023 in PDSS provinces. Meanwhile, the IE surveys in 6 PDSS provinces do not show an increase in 4 ANC utilization although there is an increase in skilled ANC utilization from 81 to 93 percent. PDO 3 (curative consultations) was triangulated with an indicator for the likelihood of consultation when a child under 5 who presented with fever. The DHS shows an increase in the likelihood of consultation from 39.9 percent in 2014 to 50.7 percent in 2023 at the national level.8 PDO 5/8 (children fully immunized) was triangulated using the IE surveys, which showed an increase in the likelihood of a child having all basic vaccinations from 27 to 49 percent (albeit for a slightly different age group and set of vaccines). 9 PDO 7 (skilled birth attendance) was triangulated using the 2014 and 2013 DHS which show that skilled delivery care increased from 79 percent to 86 percent in PDSS provinces. In the six IE provinces, the IE surveys show an increase in institutional delivery from 81 percent to 92 percent. Triangulation of the IRIs, where applicable, is outlined in Annex 1. 42. Overall, the HMIS and the survey-based indicators concur to show a substantial increase in the utilization of services in the targeted geographic areas. Many results framework indicators are substantially exceeded. The main weakness under PDO Part 1 lies in not achieving the nutrition-related indicators, which were dropped in Phase II of the Project, and the lack of fitting targets for PDOs 4 and 6. Overall, the achievement of PDO Part 1 is Modest in Phase I and Substantial in Phase II. PDO Part 2 “to improve the quality of maternal and child health services”: Phase I and II: Substantial 43. PDO Part 2 is assessed using one PDO indicator that applies to both phases. This indicator (PDO 9 – quality checklist scores) is surpassed. Of the fourteen IRIs, seven are surpassed, including two survey-based measures of availability of drugs (IRI 24 and 25), which represent structural quality of care, and three indicators that represent process quality of maternal health care (IRI 27 – first trimester ANC, IRI 28 – antimalarials at ANC, and IRI 29 – family planning discussions). Three IRIs were not achieved in Phase I and dropped in Phase II (IRI 19- breastfeeding, IRI 20 – quality of nutritional services, IRI 21 - client tracer surveys). One IRI (IRI 10 – HIV testing of pregnant women) was not achieved in Phase I and surpassed in Phase II. One IRI (IRI 11 – health personnel training) was not achieved in either 7 At the time of the ICR, the DHS 2023 results are outlined in a draft report that is pending official endorsement. This report disaggregates some but not all indicators at the province level. The source data have not yet been released and, therefore, calculations are approximative. 8 Disaggregated data for PDSS provinces are not yet available for the 2023 DHS. 9 The draft 2023 DHS report shows a significant drop in vaccination, as do the Government’s official statistics starting in 202 1. It is possible that this drop happened between 2021 and 2023 and is related to post-COVID-19 vaccine reluctance. Page 14 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) phase. Two IRIs were introduced in Phase II, of which one (IRI 26 – rehabilitated facilities) was achieved, and one (IRI 13 - performance of provincial health directorates) was not achieved. 44. Triangulation. The ICR triangulates PDO 9 with a set of IE survey-based measures that are closely aligned with the components of the quality checklist and confirm the improvements evidenced in the quality checklist. (Figure 2) Between the IE baseline and end-line, the availability of basic equipment, vaccines, family planning products, clinical protocols, handwashing stations, incinerators, and protocols and procedures improved markedly in both PBF and DFF health zones. The only areas that did not see an improvement were improved sources of water and the availability of containers for sharps and needles. The lack of improvement in sources of water can be traced to the timing of infrastructure works under the Project, which mostly started after 2020 while the IE endline survey took place in 2021. PBF zones improved slightly more than DFF zones in most of these indicators. Similar progress in structural quality is evident in hospitals, as shown in Annex 2. Triangulation of the IRIs, where applicable, is outlined in Annex 1. FIGURE 2: SURVEY-BASED STRUCTURAL QUALITY INDICATORS (FOR HEALTH CENTERS) Source: Author elaboration based on impact evaluation report (Annex 7, ref. 13). 45. Overall, the administrative data and IE surveys coincide on significant progress in both structural and process quality of care. Some IRIs regarding nutrition and performance of administrative units lagged in Phase I before they were removed in the 2020 restructuring. Overall, the PDO part 2 achievement is Substantial in both Phase I and II. PDO Part 3 “to provide an immediate and effective response to an eligible crisis or emergency”: Phase I and II: Substantial 46. PDO Part 3 is being assessed using one PDO indicator and three IRIs that apply to both phases. The PDO target (PDO 10 – EVD vaccination) was surpassed. Two IRIs (IRI 14 - loss of follow-up, IRI 16 – burials) were substantially achieved, while one IRI (IRI 15 – laboratory testing of received samples) was partially achieved. Page 15 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 47. Analysis and triangulation. The ICR counted three different sources for PDO 10 - the number of people vaccinated against EVD, ranging from 168,200 to 303,262. To be conservative, the ICR uses the lowest number, which still indicates the target was surpassed. The targets on the intermediate indicators for Part 3 were set at “perfect compliance” – i.e., 100% testing of samples, 100% safe burials, and 0% loss of contacts – in line with WHO recommendations for epidemic control. However, in the context of the ICR, it is important to remember that “full achievement” is impossible under such high targets, especially since the response occurred in a fragile environment. For all four indicators, there is a divergence in the final numbers between the various sources of information, as indicated in Annex 1. The EVD response results were compared to various results from the published literature. A comparison with EVD outbreaks in Liberia and Sierra Leone suggests that the Democratic Republic of Congo response was more successful at identifying contacts for patients (over 85 percent of patients had at least one contact identified, compared to 27 percent in the Liberia outbreak and 44 percent in the Sierra Leone outbreak) (Polonsky et al. 2021) (Annex 4). 48. Overall, since the PDO was surpassed and the shortcomings in the achievement of the intermediate indicators are partly due to setting perfect compliance targets, the achievement on Part 3 of the PDO is Substantial. TABLE 5: EFFICACY RATING SUMMARY PHASE I PHASE II PDO part 1 PDO part 2 PDO part 3 PDO part 1 PDO part 2 PDO part 3 PDOI IRI PDOI IRI PDOI IRI PDOI IRI PDOI IRI PDOI IRI Fully achieved/ 3 4 1 7 1 0 4 4 1 9 1 0 surpassed (100%+) Substantially 1 3 0 0 0 2 1 3 0 0 0 2 achieved (80%- 99%) Partially 1 0 0 0 0 1 0 0 0 0 0 1 Achieved (65%- 79%) Not Achieved 1 2 0 5 0 0 0 0 0 2 0 0 (less than 64%) Cannot be 2 0 0 0 0 0 2 0 0 0 0 0 computed Total number 8 9 1 12 1 3 7 7 1 11 1 3 of indicators % achieved* 58% 87% 120% 70% 120% 67% 83% 111% 120% 98% 120% 67% Efficacy rating Modest Subst. High Modest High Modest Subst. High High Subst. High Modest Modest Substantial Substantial Substantial Substantial Substantial Substantial Substantial *The percentage is weighted with a 1.2 weight for fully achieved / surpassed indicators. 49. Considering the overall ratings above for the achievement of the indicators across three PDO parts and two phases, the efficacy rating for both phases is Substantial albeit the overall performance was better in Phase II. (Table 5) The performance of the Project is particularly noteworthy given the overwhelming achievement of several key PDO indicators. Although the PDO indicator targets and measurement were based on administrative data, household and facility survey data (from the DHS and IE surveys) confirm that the improvements that were measured in the Page 16 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) administrative data were not only due to increased reporting but translated into actual increases in service utilization and quality. EFFICIENCY 50. The Project spent US$485.3 million (68.8 percent) on the PBF intervention, US$181.4 million (25.7 percent) on the EVD response, and US$38.0 million (18.1 percent) on system strengthening (Annex 3). The total disbursement amounted to approximately 98 percent of the total committed funds. Despite the challenges of the FCV context, several EVD outbreaks, and the COVID-19 pandemic, the Project was able to successfully absorb the original financing and four Afs while adjusting flexibly to changing circumstances. The actual implementation period of the Project was 9 years and 6 months, although PBF activities ceased in June 2023 when funding for PBF was exhausted. The Project’s final year consisted mostly of completing pending infrastructure works and procurement processes. 51. The PBF intervention focused on increasing the quantity and quality of a package of highly cost-effective reproductive, maternal, newborn, child, and adolescent health and nutrition (RMNCAH-NUT) services that have been vetted and recommended under subsequent editions of the Global Disease Priorities work. Although the Project indicators did not monitor the burden of disease, it is reasonable to assume that the improved services contributed to reducing the burden of disease from RMNCAH-NUT conditions. The impact evaluation results also demonstrated a significant reduction in the financial burden of care for households, as evidenced by a decrease in real OOP expenses. Under a very conservative assumption of 3 percent of disease burden reduction in RMNCH in the three years of full implementation of the Project and even lesser impact in the other years, the benefit-cost ratio of the intervention is 1.29, and the internal rate of return of the interventions is 17.28 percent (Annex 3). In medium and high scenarios of 4 and 5 percent disease burden reduction, the benefit-cost ratio of the intervention is 1.72 and 2.15, respectively. 52. The operational efficiency of the Project’s financing for the PBF and DFF interventions and system strengthening was substantial. Most funds reached the provincial and operational level through PBF or DFF. The yearly number of contracts signed with health facilities and zones was between 2439 and 3128, most of them with PHC health centers. (Table 6) Compared to the “ex-ante” situation where most PHC health centers received no public funding as salaries, investment, or inputs, the Project ensured a steady stream of funding to frontline providers, coupled with systems to verify service delivery (in PBF) and enhanced support and supervision from health zones and provinces. This was a game changer for local health workers and communities, as evidenced by the improvements in service delivery under otherwise challenging circumstances. PBF/DFF administrative costs included training, management of contracts, verification, counter-verification, and strengthening of the EUP agencies – these costs amounted to 26 percent of the amount spent on PBF/DFF. This is in line with estimates from other countries such as Nigeria (24 percent), Cameroon (30 percent), and Benin (50 percent). It is important to note that, to some extent, the high administrative cost stems from low established capacity and lack of public funding for the different levels of the health pyramid. Rather than building on an existing, funded health administration pyramid, PBF/DFF essentially had to fund the entire pyramid. TABLE 6: NUMBER OF CONTRACTS SIGNED Type of contract 2016-2017 2018 2019 2020 2021 2022 PMA contracts with health centers 2215 2663 2599 2777 2261 2126 (PBF + DFF) PCA contracts with hospitals and 115 177 175 191 193 175 reference centers (PBF + DFF) Health zone contracts (PBF + DFF) 140 156 156 160 146 138 Total 2470 2996 2930 3128 2600 2439 Page 17 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 53. Weaknesses in efficiency include deficiencies in fiduciary and procurement management as documented in the Project’s FM and procurement assessments. These deficiencies were especially acute in the first half of implementation when line units in the MoPHHSP carried these responsibilities. The Project course was corrected in 2020 by changing the implementation arrangements and creating a high-capacity PIU. The PIU compares favorably in performance to other PIUs across the WB’s country portfolio and was put in charge of other WB projects, including the PMNS. Important efficiency-enhancing innovations in that time included additional private pharmaceutical distributors and the implementation of cost-effective, faster modalities for infrastructure works. 54. The response to EVD was costly due to complicated logistics, the need for security to protect responders in an active conflict zone, the involvement of international and non-local staff, and inefficient human resources (HR) management stemming from the general weakness of systems in the health sector. The WB took a leading role in improving the transparency of spending on the response (such as building an HR database and tracking funds across donors) and in pushing for a more effective response model that built community trust, including a cash-for-work program, subsidies for “regular care” in health facilities, and investments in water and sanitation. Some of these interventions were (co-) financed by other WB Projects, such as the DRC Social Fund (DRC Eastern Recovery Project - P145196) and the Gender-based Violence Prevention and Response Project (P166763). Moreover, the Project’s spending on EVD response generated immense positive spillovers by preventing the spread of the disease to other provinces and countries. Extrapolating from the costs that were estimated for the 2014 EVD epidemic in Sierra Leone, Guinea, and Liberia, the social costs of a full-blown epidemic would have been over US$100 billion. (Annex 5) 55. Considering the factors above, the efficacy rating of the Project is Substantial. C. JUSTIFICATION OF OVERALL OUTCOME RATING 56. The relevance rating is High, the efficacy rating is Substantial for both phases, and the efficiency rating is Substantial. The outcome ratings are Moderately Satisfactory for Phase I and Satisfactory for Phase II. The MS rating for Phase I is due to the Modest rating for Part 1 in that phase, which can be traced down to not achieving the nutrition- related objectives in Phase I of the Project. The overall outcome rating is Moderately Satisfactory. TABLE 7: OUTCOME RATINGS PHASE I PHASE II Relevance of the PDO High Efficacy Substantial Substantial Part 1 Modest Substantial Part 2 Substantial Substantial Part 3 Substantial Substantial Efficiency Substantial 1 Outcome ratings Moderately Satisfactory* Satisfactory 2 Numerical value of the outcome ratings 4 5 3 Disbursement (US$ million) 430.58 283.95 4 Share of disbursement 60% 40% 5 Weighted value of the outcome rating 2.41 1.99 6 Overall outcome rating 4.4 Moderately Satisfactory * The rating for Phase I is Moderately Satisfactory due to the modest rating on Part 1 of the PDO in Phase I. (Table 5) Page 18 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) D. OTHER OUTCOMES AND IMPACTS 57. Poverty reduction and shared prosperity. The baseline and endline surveys provide valuable evidence on the evolution of out-of-pocket spending (OOP), which is a major driver of impoverishing health expenditures. Between the baseline and endline survey, average patient-reported OOP costs for child curative care visits at PHC facilities decreased from CF 5595 to CF 4,508, in a time where cumulative inflation was 69 percent (World Development Indicators - WDI). In real terms, this represents a drop of 76 percent in OOP costs for households. Similarly, OOP costs for antenatal care in PHC facilities dropped by 81 percent in real terms. Given that an estimated 70-75 percent of the population outside of the capital, Kinshasa, is considered extremely poor, such reductions in burden on households plausibly removed barriers to care and decreased impoverishing health expenditures. 58. Gender. The Project had a built-in gender focus by focusing on the provision of maternal and reproductive health services and active participation of women in health center committees and community client satisfaction surveys. Beyond the indicators of health service delivery to women, the Project did not collect data to measure women’s participation or voice in the Project activities. AF1 aimed to extend the Project's reach by supporting gender and sexual violence interventions such as training and information sharing for health staff, but these interventions were ultimately moved to a different project. 59. Institutional strengthening. The PDSS formed an integral part of the MoPHHSP’s agenda for universal health coverage. As such, the Project supported the development of the MoPHHSP’s health stewardship capacity. The CT-FBR was crucial in rolling out PBF tools, supporting training in 11 provinces, aiding provincial EUP and health departments, and revising tools and manuals based on emerging field lessons. The project improved institutional capacity through multiple avenues such as (i) requiring facilities to develop a business plan outlining how funds will be used; (ii) ensuring that all facilities opened and maintained commercial bank accounts; (iii) taking into account community satisfaction through community audits and involving communities in the development of business plans; (iv) using performance contracts to formalize expectations and performance metrics at all levels of the system; (v) developing data systems to track and verify health service delivery. Regarding Project management, the initial setup attempted to strengthen these functions within the line units of the MOPHHSP. Accordingly, between 2016 and 2020, the PIU was located under DEP, while CAGF and CGPMP were responsible for procurement and financial management. However, in February 2020, continued poor performance and low capacity eventually led to the relocation of the PIU under the Secretary- General and reassigning all procurement and FM responsibilities to the PIU. Overall, the new working model established under the Project allowed the Government to expand the HNP portfolio to four projects to deliver investments in nutrition, COVID-19, and pandemic preparedness and response. 60. Mobilizing private sector financing. Although the Project did not mobilize private sector financing per se, the Project utilized non-state actors to deliver project activities and services. Health facilities' contracts included Government facilities and private non-profit (denominational) providers. The latter are important providers of health services in rural communities. The Project also strengthened the pharmaceutical distribution market by incorporating additional private firms through certification and contracting. 61. The Project built on existing community structures to strengthen the accountability framework of the PBF intervention. Community involvement included: the involvement of each health area’s health committee in the elaboration of the annual work plan which included PBF financial flows, putting the health committee in charge of identifying indigent patients at the local level, recruiting community members to carry out satisfaction surveys, and utilizing the existing community health liaison (Relais Communautaire – RECO) to help health workers reach their populations. Page 19 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. PROJECT DESIGN AND IMPLEMENTATION FACTORS RELATED TO THE PBF INTERVENTION 62. The Project enjoyed strong support from the Government as its main intervention (PBF) formed a part of the national health strategy. The Project’s start suffered an initial delay of 16 months between signing (January 2015) and effectiveness (May 2016). However, after effectiveness, activities started quickly. A highly effective training cascade was rolled out from the central level to the province level and then to the health facility level between September 2016 and March 2017. The Project was exceptional in its ability to roll out the PBF system to such a geographically dispersed territory within 2 years of effectiveness. In 2017, the Project signed 2,366 contracts with health facilities (1530 for PBF, 836 for DFF) and 140 contracts with health zones. The DFF modality was meant to provide the control group for the impact evaluation of the PBF intervention (Annex 4). 63. Efforts to align health sector financing faced bottlenecks. Under Component 3, the Project implemented an innovative initiative to rationalize financing from multiple donors to the province level by supporting Single contracts in 11 provinces. These contracts formed part of a national initiative, and therefore, contracts were also signed in other provinces that were not supported by the Project. Theoretically, such a contract could regroup all essential health administrative functions at the province level and ensure streamlined, coordinated financing from one or multiple sources. contracts successfully demonstrated a model of how financing could be more streamlined. Implementation improved over time with close supervision but faced difficulties, with varying levels of engagement in different provinces. The necessary coordination and support to address these issues were partly challenged by the COVID-19 pandemic. Leadership from the Government’s central level to prioritize and align funding between provinces and communicate priorities to the provinces continued to be weak. Development partners did not all embrace the initiative and were not always able to align their financing procedures and timelines with the contracts, especially when there were breakdowns between the donors’ central level and implementing agencies at the provincial level. For example, of the 2018 financial commitments by technical and financial partners made under single contracts, only 40 percent materialized. Some partners did not sign on, while others withdrew from the contracts. In some provinces, the single contract’s focus on transparency and funding accountability faced resistance from the provinces, which saw the contract as a Project initiative rather than an integral Government strategy. 64. The Democratic Republic of Congo is a large, extremely diverse country with distinct regional circumstances and province-level implementation capacity. The PBF program tasked the provincial level with purchasing functions such as negotiating and signing contracts with PHC facilities and district hospitals, verifying the quantity and quality of services, and providing the necessary support and coaching to health facilities. Given the low capacity of provincial health departments, essential PBF functions (signing contracts, verifying the quantity of services, and contracting local associations to carry out community client satisfaction surveys) were delegated to the EUPs, independent not-for- profit entities. Meanwhile, the quality of services was verified by health departments. In the initial stages of the PBF up to 2020, counter-verification was entrusted to a contracted external counter-verification agency. After 2020, the responsibility for counter-verification of services transitioned to the provincial health inspection bodies (Inspection Provinciale de la Santé; IPS). Overall, the performance of the provincial EUPs and health departments was highly variable. The checks and balances built into the PBF system were no match for the governance challenges in Equator province, which was eventually excluded from Project funding due to persistent fraud and corruption issues. Other provinces like Maniema and Katanga also faced cases of fraud, which resulted in EUP staffing changes. The counter- verification also highlighted significant variation in the accuracy of service quality assessments done by provincial and district health departments. Nonetheless, assessments carried out by government bodies rather than outsourced contributed to durable capacity. Page 20 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 65. Despite the regional variation, the Project adopted a uniform model of PBF that was not modulated according to the density of the service delivery network. In provinces such as Maï-Ndombe, the PBF model faced significant difficulties related to a lack of geographical access to facilities and high dispersion of services among facilities due to low population density. Such remoteness made verification and counter-verification very expensive for a limited volume of services. The MTR identified the need to consider a different approach than the “classical PBF,” possibly through DFF based on a quality index or other criteria, such as implementing a flat tariff system. However, this idea was not pursued, partly because the initiative coincided with the start of the COVID-19 pandemic, which commanded the health sector’s attention and left little space for new initiatives. 66. Broken supply chains and missing markets for pharmaceuticals and inputs limited the impact of PBF’s cash infusion to health facilities, but significant changes were made mid-project to address this challenge. On the supply side, the Project’s original design aimed to strengthen the national supply chain for pharmaceuticals, inputs, and equipment following the 2017 strategy for the National System for Pharmaceutical Supply (Système National d'Approvisionnement en Médicaments; SNAME) that was centered around the Federation of Essential Medicines Supply Centers (Fédération des Centrales d’Approvisionnemets en Médicaments Essentiels; FEDECAME). FEDECAME is a private, non-profit umbrella organization that regroups regional CDRs and signed a public service devolution agreement with the MoPHHSP. This agreement entrusted the FEDECAME network with the purchase of medicines and medical consumables for the national essential medicines supply system and public-sector health facilities. The Project’s initial allocation of medicines to health facilities ranged from US$800 to 2500 depending on the type of facility. The purchase and distribution were done through the FEDECAME network, and CDRs were meant to strengthen their operating capital through mark-ups on Project-financed medicines. Unfortunately, the Project encountered serious challenges with FEDECAME’s management of contracts. In one contract for the supply of medicines and inputs, FEDECAME failed to deliver on basic contractual arrangements, which resulted in over US$0.9 million in ineligible expenditures. The performance of the regional CDR was highly variable. Two provinces (Maï-Ndombe and Kwango) did not have a CDR, while CDRs in three provinces (Tschuapa, Maniema, and Haut-Lomami) never had a rolling stock of medicines. FEDECAME’s and CDR’s financial situation remained weak, and their costs remained high, resulting in widespread stockouts of essential medicines. 67. On the demand side, the Project’s design relied on the cash injection to front-line PHC facilities and hospitals from PBF payments, which came with wide autonomy on how to use the payments. Health facilities were, therefore, in theory, able to purchase needed medicines, supplies, etc., from CDRs that formed part of the FEDECAME network and from local certified suppliers. From the start of the Project, the contracts of contracted facilities specified rules for how they could spend their revenues, including both PBF payments and user fees. Initially, the main restriction was that at most 50 percent of each quarter’s revenue could be used for staff bonuses, and the rest had to cover operational costs, inputs and medicines, equipment, and infrastructure. In practice, however, facilities did not systematically prioritize the purchase of medicines and consumables. Remote facilities, especially in provinces with poorly functioning CDRs and limited private supply, were locked out of the supply chain. In addition, the new system for buying medicines circumvented DPS’s institutional responsibility to supervise the supply of medicines to health facilities, leading to a vacuum in supervision and coordination. Limited data suggest that the availability of medicines in PHC facilities declined from 69 percent in 2017 to 63 percent in 2019 because of the combination of increased utilization of services (including medicines) and lack of supply (Figure 3). Page 21 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) FIGURE 3: AVAILABILITY OF TRACER MEDICINES FOR THE MPA 90% 85% Source: WB elaboration based on 80% 76% 74% Government reporting. Caveats: Data 75% 69% 69% 67% on the availability of medicines in 70% 66% 67% 65% 63% health facilities was not readily 60% available until 2017, and there are 55% issues of comparability of data at 50% 45% different times and between sources. 40% Baseline 2016 2017 2018 2019 2020 2021 2022 2023 (2015) FIGURE 4: PERCENTAGE AVAILABILITY OF BASIC INPUTS AND TRAINING FOR PRENATAL CARE (2017-2018) Prenatal care directives 98 Staff trained on Tape measure 56 prenatal care 66 Source: WB elaboration based on Service Provision Assessment (SPA) 93 83 Blood pressure 2017-2018. Fetal stetoscope meter 43 89 Height chart Stetoscope 77 Adult scale 12 PDSS provinces 68. Starting from the MTR, the Project changed its paradigm for access to pharmaceuticals and inputs with new approaches both on the supply side and on the demand side. On the supply side, FEDECAME’s quasi-monopoly was broken up as the Project supported the certification of additional private distributors in seven provinces. The private distributors started competing with the CDRs for health facility purchases. By 2021, 1,788 out of 2,851 health facilities were buying their medicines from private distributors, including all facilities in Tshuapa, Maï-Ndombe, Haut-Lomami, Lualaba, and Kinshasa, and most facilities in Sud-Ubangi and Mongala. Private distributors accounted for 49.9 percent of health facility purchases in 2021, and 66 percent in 2022. The Project also carried out a second round of allocations for items that were in particularly short supply, including essential medicines and family planning supplies. These Page 22 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) medicines and inputs were delivered directly to health facilities. In parallel, the Project continued to contract CDRs for logistics services such as storage and secondary transportation of centrally purchased equipment. On the demand side, the PBF rules were changed to require that health facilities spend a minimum spending of 20 percent of their PBF payments on medicines and other consumables. Funds were taken “off the top” from facilities’ PBF earnings and managed by the PIU. Health facilities could order from suppliers (CDR or private distributors) using their reserved balance, while the PIU paid suppliers based on goods delivered to each health facility and kept track of each facility’s spending and remaining balance. The DPSs were brought back in to enhance the supervision of medicine and supply streams. 69. In its initial phase, the Project provided startup grants to health facilities, but those were not sufficient to address the many gaps in infrastructure and equipment. The initial startup grants amounted to approximately US$2,000 per PHC center and US$4,000 per district hospital and were meant to finance an initial stock of medicines, supplies, basic equipment, management tools, repairs and construction of infrastructure, and training. Facilities also had the autonomy to use their funds to improve infrastructure, but there was no explicit guidance or standards for such investments, and the available funding level was low. In total, 592 facilities reported using funds for infrastructure improvement. Given the small amount of funds, general state of disrepair, and lack of equipment in health facilities at the start of the Project, these two mechanisms did not result in significant upgrading of physical infrastructure. In fact, at the mid-term review, it became clear that many facilities were unable to complete new infrastructure that they had started building or had built infrastructure that did not conform to building norms. Despite the cash influx in PDSS provinces, the 2017-2018 SPA survey found striking shortages in basic prenatal care equipment such as height charts and scales, stethoscopes, and blood pressure meters. 34 percent of health facilities did not have any staff that received training in prenatal care in the last 24 months. This information is confirmed by the Project’s 2020 quality scorecards, which showed significant shortcomings in quality: the provinces of Equateur, Nord-Kivu, Maniema, Haut-Lomami, and Lualaba had average quality scores for MPA of less than 50 percent, while the rest of the provinces scored between 50 and 79 percent. The main challenges were infrastructure, equipment, medical inputs, and medicines. 70. Again, significant changes were made after the MTR: the Project introduced a new round of investment grants and two models for their execution: locally managed works (travaux en régie) in Sud-Ubangi, Mongala, Tshuapa, Kwango, and Maï-Ndombe, and selection of small construction firms at the province level in Kinshasa, Nord- Kivu, Haut-Katanga, Haut-Lomami, and Lualaba. This effort resulted in the rehabilitation of 176 health facilities, including the new construction of health centers and hospital pavilions and the completion of partially built infrastructure. New PHC facilities were built using standard plans significantly faster and cheaper than previous infrastructure initiatives. The standard plans focused on functional, simple, but durable construction with essential amenities like rainwater collection, solar panels, and battery kits. Given that many rural health facilities are located far away from an electrical grid, solar kits ensure that light is available for nighttime services such as deliveries. During the last two years of the Project (2023-2024), the remaining Project funds were used to procure primary equipment kits for 1288 PHC facilities (hospital beds, delivery room beds, microscopes, water, sanitation, and hygiene (WASH) equipment, etc.), secondary equipment kits 111 district hospitals (hospital beds, operating tables, neonatal incubators, etc.), solar power kits (613 health facilities), laboratory and blood bank equipment (90 health facilities). At the time of the closing of the Project, all the equipment had arrived in the country. However, some of this equipment arrived in the provinces after the end of the Project’s support for PBF in June 2023. In some provinces, such as Sud-Ubangi and Mongala, it was still awaiting distribution to health facilities at the time of the ICR. 71. The Project innovated to improve the quality of care, though the needs were immense, and much remains to be done. The Project’s design introduced a quality scoring tool to measure structural quality, and substantial improvements were achieved in the initial phases of the Project. Between 2016 and 2019, the measured quality score increased from 62 to 83 percent on average. However, gaps in provider knowledge and competency or clinical practice Page 23 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) could not be measured with the tool. Attempts to tackle HR aspects, such as rejuvenating the health staff through retirement, proved unsuccessful. The Project introduced clinical practice vignettes as a pilot in 23 health zones in 3 provinces in 2018. Vignettes are training and evaluation tools that typical patient cases for health workers to analyze and respond to. The initial rounds showed encouraging increases in provider scores on the vignettes, and the MTR intended to review the quality scoring tools so these vignettes would form up to 50 percent of their weight. However, this was not implemented. The vignette pilot was discontinued at the end of 2021 when the tablets and software became obsolete. A second quality pilot under the Project was the introduction of training, peer coaching, medicines, and direct observation of delivery and newborn care in 53 health facilities. The pilot evidenced higher-than-expected improvements in quality (e.g., 60 pp. improvements in quality scores for assisted delivery and newborn reanimation) but was not further rolled out. The Project did not include updates to clinical practice guidelines or more comprehensive monitoring of clinical practice that were initially envisaged under Component 3. However, the Project did fully rebuild or renovate 175 health facilities, approximately seven percent of the facilities participating in the PBF scheme. 72. The Project invested in upgrading and deploying information systems, including DHIS-2 and the PBF-IT platform, resulting in increased data availability on the quantity of health services delivered. Because PBF payment was based on the number of services provided in each category, verification and counter-verification relied on the registers that tallied the number of services per type, such as prenatal visits, delivery care, outpatient visits, vaccination, and family planning. The information systems took the register approach to recording information instead of a person-based approach favored by the quality-of-care movement. Clinicians facing a patient benefit from having access to information on all services previously provided to that patient to enable them to make informed decisions and identify gaps in care instead of having this information scattered across registers. B. IMPACT OF AND RESPONSE TO THE EVD OUTBREAKS 73. The EVD 9 and EVD 10 outbreaks from May 2018 to November 2020 had significant impacts on the overall allocation of funds under the Project and on the implementation of core activities under components 1, 2, and 3. (See Annex 5 for more details on the outbreaks.) Largely through the PDSS, the WB became a major financier of the EVD response through three activations of the Project’s CERC component. In the initial CERC activation in May 2018 (formalized in November 2018), most of the available funding for components 1, 2, and 3 in PDSS were reallocated to the CERC component. As such, several planned activities related to nutrition, family planning, and pension reforms could not be implemented due to a lack of funding. In January 2019, the rating for components 1 and 2 was downgraded in ISRs from Satisfactory to Moderately Satisfactory. The US$80 million replenishment approved during AF3 for Components 1 through 3 provided the resources required to fully resume the implementation of PDSS activities that focus on improving the utilization and quality of health services through PBF. However, AF3 took almost six months to become effective, and more funds had to be reallocated in the interim. 74. The EVD 9 outbreak took place in the province of Equateur, which had seen previous outbreaks. The response was highly effective as it managed to end the outbreak after 2.5 months, with only 54 cases in total. In contrast, the EVD 10 outbreak started in Nord Kivu, a province that had not previously experienced EVD. The response was protracted and complex because it was hampered by general insecurity in the main outbreak zone (Nord-Kivu, Sud-Kivu, Ituri) and violence against responders. The first three EVD10 strategic response plans (SRP 1-3 - up to June 2019) focused on the public health response and saw WHO leading the response on behalf of the Government, with support from UNICEF, WFP, and NGOs. Active armed groups and population movement across provinces and across porous borders with South Sudan, Uganda, and Rwanda made case detection and contact tracing challenging. Many of the staff involved in the response were not local, as local staff had limited experience with EVD. Despite significant financial means, coordination of the response was an issue. Infection prevention and control and Page 24 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) safe and dignified burial activities met with community resistance amid a widespread lack of trust, fed by the involvement of police and military forces, the funds and means available for the response (international and non-local staff, helicopters, vehicles, etc.) in an environment where the local population lacked basic means of subsistence, poor communication and coordination with communities, and allegations of corrupt and questionable practices in the response itself. The violence against the EVD 10 response began almost immediately and included 483 recorded attacks on health facilities and response personnel. Extreme violence resulted in the deaths of at least 25 health workers and responders and arson attacks on 49 health facilities, including Ebola treatment centers. The violence led to several resurgences in the epidemic and was shown to be associated with higher transmission of the disease. (Annex 7, Refs. 26-33, 41, 43, 44) 75. Amid these challenges, the response model was eventually changed under SRP 4 starting in June 2019. SRP4 put in place multidisciplinary rapid response teams that included epidemiologists, safe burial teams, vaccinators, and contact tracers that responded to new cases in an integrated manner. Security arrangements were revamped: to avoid the need for response teams to have military escorts, MONUSCO was put in charge of securing areas for response teams. Efforts were ramped up to communicate with local communities and gain trust. Complementary interventions were implemented to generate benefits and humanitarian relief for local communities, such as a cash-for-work public works program and investments in water and sanitation. 76. A major operational bottleneck for both the EVD 9 and 10 responses was the lack of a unified personnel database that could manage the compensation scheme. With multiple responding agencies in place, each with its own system, it quickly became difficult to ensure transparent, fair, and efficient compensation. Attempts by the MoPHHSP to consolidate personnel lists were protracted and ineffective, and eventually, the Project hired an external consulting firm to consolidate the database. IV. WB PERFORMANCE, COMPLIANCE, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF M&E M&E Design 77. Data sources. The Project’s indicators were based on administrative HMIS and PBF-IT systems which were nascent at the time of Project design. As can be expected, these information sources for administrative data are influenced by other factors such as incentives to over- or underreport (for DFF and PBF facilities, respectively), delays in reporting, and other factors such as lack of paper registers, computers, or electricity. To some extent, the increase in the indicators from improving data systems seems to have been considered in the original targets, although this is not documented in the Project records. The indicators that are expressed in percentages were based on extrapolations from the last available census of 1984, making them approximative. Given that there was no better option, the drawbacks should have been documented, and the approximations should have been used consistently, which was not the case, as evidenced in the targets of PDO 3, 4, 5, and 7 (see above). 78. Impact evaluation. The Project design included a robust IE concept that covered all 11 provinces of the Project and included both household and facility surveys. The treatment (PBF) and control (DFF) health zones were randomly selected during public randomization ceremonies, which supported transparency and clarity on the evaluation design. The IE did not include a “pure control” group, as many non-Project provinces planned to introduce PBF financed by other donors. The impact evaluation team obtained the necessary clearances and recorded the research design in a trial registry. The baseline was completed in time before the launch of Project activities. Page 25 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) M&E Implementation 79. Changes in results framework indicators. The results framework was modified multiple times, as discussed above. For some indicators, the sources of data for the original baseline and target remain unclear, as is the source of the revised baselines and targets introduced in 2017 and 2018. The indicators framework was overhauled in 2020, and this represented a notable improvement, though some issues remained: a. The Core Results Indicators (CRI) were integrated in February 2019 (AF3) and later modified but were not in line with the CRI guidelines (data should be cumulative starting from a baseline of 0). b. IRI 1, 4, and 5 (contraceptive prevalence) had contradictory targets. c. IRI 14, 15, and 16 (EVD interventions) had aspirational 100 percent coverage targets, and IRI 15 did not correspond to a performance indicator of the main intervening/contracted organization (WHO). d. Indicator definitions, calculation formulas, data sources, baselines, and targets were minimally documented in the restructuring and Project papers. e. As outlined in Annex 1, multiple contradictions exist between the variables' definitions and the baselines and targets. 80. Insufficient attention was paid to the results framework during the supervision of the Project, which generated issues at the mid-term and ICR stages. Although the M&E ratings in the Project's first years were satisfactory, it later became clear that the reported numbers came from different data sources that were not comparable to the baselines. After overhauling the results framework in 2020, the PIU used this new framework to provide a comprehensive report on the Project indicators, utilizing available information almost exclusively from the DHIS2 platform, which was strengthened under the Project. However, the lack of clear formulas and attention to detail continued to give rise to contradictions in the data and indicators. For example, the Government reported on IRI 14, 15, and 16 for the EVD interventions. However, the information was not processed and reflected accordingly in the Project documentation. The achievements reported in the Project’s final ISR (80 and 85 percent) do not correspond with numbers reported in the aide-memoires of 2020 and 2022, the Government’s reports, contractual and other reports from the WHO, or journal publications. Other examples are outlined in Annex 1, including PDO 3, 4, and 5. 81. Data sources. Except for one indicator on the number of services provided under PBF, utilization data reported in the results framework originates in the DHIS2/HMIS system. Several aide-memoires analyze the difference between the data reported in DHIS2/HMIS and the verified data reported in the PBF system and show significant differences between the two. Since the verified data from the PBF system were not used to report on the results framework, the results framework overestimates the provided services. Two mitigations are in place for this overestimation. First, the overestimation seems to have been considered ex-ante when the targets were set, and second, the ICR triangulates the achievements with household-survey-based coverage indicators. The ICR does not use data from the PBF system, where only data from the last 2 years were accessible. 82. Impact evaluation (Annex 4). The Project worked effectively with the WB’s Research Department to design and implement a high-quality impact evaluation that provides a rich and valuable source of information to triangulate the data produced by the administrative HMIS and PBF-IT systems. Strikingly, the Project managed to guard the integrity of the evaluation design, including the allocation of treatment and control groups. Baseline and end-line surveys (household and facility) were carried out in due time, although due to cost restrictions and the impact of COVID-19, the end-line surveys had to be scaled back and only included 6 provinces. The end-line surveys were fielded under difficult circumstances during the COVID-19 pandemic. Commendably, the evaluation results were published within 1.5 years of the completion of the end-line survey, which is unusually fast. The IE data were published in the WB’s repository, though a reproducibility package was unavailable during the ICR preparation. The Project team also Page 26 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) contributed to several publications documenting lessons learned, including a published analysis of spending on EVD response. M&E Utilization 83. Administrative data. The Project designed a PBF dashboard based on the HMIS and PBF-IT systems, which was frequently used to review the performance of the PBF intervention. The HMIS platform continues to be used even after the Project's completion, for example, to monitor the impact of mpox on RMNCH service delivery. 84. The IE results were published and disseminated. However, the available survey data (IE surveys, 2014 DHS, 2017 SPA survey) were not fully used to triangulate the administrative data or document the evolution of indicators under the Project. The IE focused on comparing the PBF and DFF interventions and, therefore, focused on the additional value of the PBF mechanism above and beyond the DFF mechanism. This was appropriate from the research angle, but for the evaluation of the Project, it was important to understand the combined trends for both PBF and DFF areas, as both interventions were financed by the Project. This combined baseline-to-end-line comparison for both interventions was done as part of the ICR. Justification of Overall Rating of Quality of M&E 85. The Project made commendable efforts to strengthen data collection systems. First, an impact evaluation identified the impacts of the PBF intervention (compared to DFF) in a subset of the Project provinces. Second, the Project contributed to strengthening the HMIS system. Third, the Project set up a robust administrative M&E system (PBF-IT), widely used for PBF monitoring although not incorporated in the results framework. However, much more attention was needed on the results framework and on proper recording and reporting. The multiple changes in indicators and calculation methods, lack of definition and documentation of sources of data and calculation methods, and contradictions between Project indicators required substantial additional research and clean-up at the time of the ICR to ensure the reliability and consistency of outcome measurements used in the ICR. Therefore, the overall rating for M&E quality is Modest. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social Compliance 86. Design. The Project was classified as Category B under the Environmental and Social Safeguards policy and triggered two safeguard policies: Environmental Assessment (OP/BP 4.01) and Indigenous Peoples (OP/BP4.10). Project safeguard documents consisting of an Environmental and Social Management Framework (ESMF), Biomedical Waste Management Plan (BWM-P), and Indigenous Peoples Planning Framework (IPPF) were developed, approved, and disclosed. Following the conclusions of these documents, a Biomedical Waste Management Plan (BWM-P), and 11 Indigenous Peoples Plans (IPP) were prepared. In 2019, the Project also developed a GBV action plan, although the Project’s institutional records do not contain a copy of it. 87. Implementation. Implementation of the plans and recruitment of specialized staff was delayed. In the 21 ISRs prepared under the Project, the overall safeguards performance rating was S in 7 ISRs, MS in 12 ISRs, and MU in 2 ISRs. MU ratings were related to delays in operationalizing the Biomedical Waste Management Plan, GRM, GBV Action Plan, and IPPs. The MTR noted the poor hygiene in many health centers supported by the Project. Environmental and Social specialists for the PIU were only hired in 2020. The IPPs were implemented by hiring three specialized NGOs, and although this work was much delayed, it was satisfactorily completed in 2022. The MTR report did not contain Page 27 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) information on the state of implementation of the Project's environmental and social safeguards. In 2020, allegations of sexual exploitation and abuse surfaced in the context of the EVD 10 response.10 Several measures were taken in the wake of this development: the WB temporarily stopped providing no objections for contracts with the implicated UN agencies and NGOs. The Bank health team worked with GBV specialists and OPCS to ensure that GBV was retrofitted in the design for all health service delivery projects across the portfolio to include GBV prevention, holistic care for GBV survivors, signing of a code of conduct, and a Sexual Exploitation and Abuse/Sexual Harassment sensitive GRM mechanism. 88. Grievance (Redress) Mechanism (GRM/GM). The initial GM consisted of the community verification system and community associations' involvement in health facility business plan management. However, neither of these two mechanisms was a proper GM mechanism that allowed for a structured collection of and response to complaints, as they had a different objective (verifying services billed by health facilities). After the issues mentioned in the previous paragraph, the Project put in place a more robust GM and rolled out training and codes of conduct both at the central and provincial levels. The GM piggy-backed the IPPs by installing complaint management committees in the provinces where NGOs worked on the implementation of the IPP (Nord-Kivu, Haut-Lomami, Maniema, Maï-Ndombe, Tshuapa, Mongala, Sud-Ubangi). Finally, in 2022 the Project signed an agreement with the United Nations Population Fund (UNFPA) to support the MoPHHSP in implementing the GBV risk mitigation measures and GRM in the provinces not covered by these NGOs. A national call center was set up in November 2023 to receive complaints. This was much too late, considering that Project financing for the PBF/DFF mechanisms was discontinued in June 2023. Fiduciary Compliance 89. Ratings. The procurement performance rating in the 21 ISRs prepared under the Project was S in 4 ISRs, MS in 12 ISRs, and MU in 5 ISRs. The FM performance rating was S in the first two ISRs, MS in 11 ISRs, and MU in eight ISRs.11 All MU ratings were up to 2020, while the CGPMP and CAGF were institutionally in charge of procurement and FM for the Project. 90. Procurement. Before 2020, issues included delays in hiring or replacing procurement specialists and a lack of engagement of CGPMP. Once consultants were hired, the CGPMP was not involved in procurement processes, despite the institutional arrangements. In the 2020 restructuring, the institutional arrangements were modified so that the CGPMP was no longer in charge of procurement for the Project, and the UGP-PDSS took over the responsibility. Procurement performance after 2020 was at least MS. The key issues and risks were: (i) limited knowledge and experience in procurement matters; (ii) delays in the procurement process, particularly in the elaboration of bidding documents, evaluation of bids, and implementation of contracts. The Systematic Tracking of Exchanges in Procurement (STEP) system was not used efficiently, with many entries being recorded ex-post to regularize a process that had already taken place. Towards the end of the Project, activities mainly consisted of purchases of medical inputs and equipment, and many supply contracts were plagued by delays in the delivery times due to import and customs formalities, as well as delays in the distribution of materials to their final destinations. At the closing of the Project, some equipment still had not reached its final destination. 91. Financial management. Before 2020, FM was characterized by weak monitoring and projection of commitments of budget for the various activities financed by the project, delays in submitting annual work plans and budgets, inconsistent internal audit, weak staffing, and lack of involvement of the CAGF. Exchange rate losses and their impact on the budget were not adequately addressed. Internal control deficiencies included abnormally long lead time 10 WHO, Final Report of the Independent Commission. Annex 5, D 30. 11 There were 18 FM assessments at implementation, of which 7 were MU and 11 were MS. Page 28 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) for processing invoices, short-term loans being granted to Project Implementation Unit staff on IDA funds, long outstanding sundry debtors accounts of undocumented advances with no indication of attempts to document or collect, and recurrent delays in archiving supporting documentation. The Project had about US$906,345 in ineligible expenses from a contract with FEDECAME. (see paragraph 65) The project had eight Designated Accounts, which created confusion and errors during the EVD response when the CERC was triggered three times and funding was shifting between categories and accounts. Starting in 2020, FM performance improved, including monitoring disbursements, commitments, and available funds. Further improvements were made to correct a mismatch between the Designated Account balances in the commercial banks and the balances reported in Client Connection due to errors in processing. A more acceptable internal control system was implemented, FM staff had adequate skills, and IFRs were deemed moderately satisfactory. Continued shortcomings were: (i) sizeable outstanding advance amounts (about US$1.1 million as of April 30, 2024) with some amounts pending documentation or refund for over 2 years; (ii) bank reconciliation statements with outstanding checks for more than 6 months; (iii) internal audit reports are prepared but not transmitted within the stipulated timeline to review by the WB FM team. At various stages of implementation, the PBF scheme was marred by delays in payments to health facilities and hospitals (for provided services) and payment to suppliers (for medicines and inputs). C. WB PERFORMANCE Quality at entry is rated Moderately Satisfactory. 92. The Project’s initial design focused on introducing PBF financing in 11 provinces and incorporated lessons from previous pilots in the Haut-Katanga and other provinces. Preparation for implementation was carried out under the Health Sector Rehabilitation Project (P088751) which closed in December 2014. The PBF concept assumed that health facilities would use the obtained funding to pay staff, purchase necessary medicines, inputs, and equipment, and upgrade their physical infrastructure. This design underestimated the importance of several structural issues in the health system: (i) broken pharmaceutical supply chains that prevented facilities from buying inputs; (ii) the poor physical condition of health facilities and lack of equipment; (iii) HR issues and disparities, including poor skill sets, excessive staffing in urban areas, and lack of staffing in rural areas; (iv) lack of hygiene and proper medical waste management. 93. The team worked effectively with the WB’s Research Department to design a high-quality impact evaluation with facility and household surveys, and randomized allocation of treatment (PBF) and control (DFF) HZs. Quality of Supervision is rated Moderately Satisfactory. 94. The WB carried out regular supervision missions, including field visits. The WB’s HNP team was highly successful at coordinating with partners in a fragmented environment and in mobilizing additional funds to support technical assistance, such as experts in pharmaceutical policy, PBF, reproductive health, and many others. Deploying this assistance required extensive coordination, supervision, and follow-up. The fiduciary team provided extensive training and capacity building to the implementing entities, as well as detailed supervision reports. During the EVD 10 response, the FM team carried out specific missions to respond to the bottleneck of HR compensation management and other issues. 95. The PDSS successfully mobilized funds from IDA and donors, expanding the financial envelope for front-line services. Still, a challenge linked to this success was the fragmentation of sources of financing. Disbursement arrangements at approval were complex and inconsistent between different sources of financing. Excessive fragmentation in the sources of financing led to a high administrative burden of tracking expenses, reallocating funding, Page 29 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) and restructuring. At the time of the mid-term review, the counterpart fiduciary team was managing eight Designated Accounts in three different commercial banks. Coupled with the capacity constraints of the fiduciary team, this situation led to accounting and disbursement issues requiring accounting adjustments in the various financing sources. 96. Some aspects of the expansion of the PBF program through AF1 and AF2 were premature and overly ambitious. In March 2018, AF2 was approved when disbursements reached US$72 million (about 32 percent of the original financing). AF2 included a significant expansion of the scope of the PBF model, with new areas of focus (nutrition, adolescent health, GBV) and new approaches (community PBF). However, this expansion was not implemented, partly due to the impact of the EVD outbreak but also because they were overambitious. Following the stabilization of the health situation, establishment of a strong PIU, and capacity building within the MHSSP, the HNP portfolio expanded, which allowed some activities (nutrition and adolescent health, GBV, preparedness) to be covered under other operations. 97. The Project’s MTR was remarkably candid and practical, resulting in multiple adjustments and improvements to PBF’s technical design. The adjustments that were made included: (i) integrating two highly effective models for upgrading and rebuilding health facilities, using decentralized contracting models with technical supervision by EUP-appointed engineers; (ii) enhancing competition in the pharmaceutical and supplies market by pushing for certification of additional supplies and reserving 20 percent of PBF payments for medicine purchases; (iii) complementing the PBF incentive approach with centralized procurement arrangements for standard equipment that individual health facilities could not effectively purchase, such as solar kits and hospital beds. The MTR was delayed due to the impact of the EVD outbreaks which commanded the Bank team’s and the Government’s attention. The improvements it generated came towards the end of the Project’s support for the PBF/DFF interventions in June 2023. Some of the investments financed by the Project, such as equipment and supplies, have not yet reached frontline facilities and hospitals even though the Project is closed. Although these investments may not have optimally impacted outcomes during the PBF intervention, they will support general service delivery capacity beyond the Project timeline. 98. The WB’s response to the EVD 9 and 10 outbreaks was swift and decisive, and the WB team played a critical role in helping to rethink the response strategy when the initial public health response was not fully working. On the financing side, the CERC was activated at the onset of the EVD 9 outbreak, and funding was reallocated to the response and continued after EVD 10 broke out. The third round of AF aimed to fill the thereby created gap in financing for the PBF interventions. The WB dedicated two full-time staff to a field office in Goma to help reorganize the response strategy in the face of increasing community resistance and violence. The WB financed research to understand the reasons for the resistance and played a critical coordination role among donors to change the approach. The WB team also spearheaded the effort to bring more clarity to the financial management of the response. The WB’s team carried out three specific missions to support the EVD 10 response, focusing on critical bottlenecks such as the management and payment of human resources involved in the response. Other sectors at the WB collaborated to bring much investment needed to respond to other needs of the affected population, such as water and sanitation and cash-for- work interventions through other projects in the portfolio, showcasing a comprehensive and multisectoral effort. The changed approach led to a decrease in transmission which eventually ended the epidemic. Throughout this time, the health team kept WB senior management updated through numerous briefs and meetings. The WB team was also clearly aware of the social challenges of the EVD response and worked to find solutions. From the very start of the EVD 10 response in August 2018, a Project aide-memoires documented that community resistance to the response was an issue, and several subsequent aide-memoires provided a detailed discussion of the concerns related to the response itself. However, these concerns were not sufficiently reflected in the Project’s implementation supervision reports Page 30 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) (ISR), restructuring papers, or AF3/AF4 Project papers.12 The social and environmental risk rating remained Moderate, and the Project’s social safeguards staff were not listed as participants in the supervision of the EVD response activities. 99. Supervision challenges include delays in the start of fiduciary assessments despite High risk (September 2017 for procurement, December 2016 for FM) and occasional long intervals between the assessments. Many aide- memoires do not reflect the involvement of E&S staff, and E&S safeguards are not mentioned in aide-memoires before June 2017. AF papers lacked documentation on the choice of indicators, social risks of the EVD 10 response, and the expansion of the PBF scheme to Nord Kivu. Multiple rounds of change in the format of the operations portal, Project papers, and supervision documents resulted in fragmented, inconsistent documentation over the Project’s 10-year timespan. Mission aide-memoires and management letters were not systematically uploaded in the Operations Portal. One of the AF papers was never disclosed (PAD2651). The WB- records management systems were not used effectively to document the initial Project design, its modifications, and key aspects of implementation. Examples of missing documentation are the modifications in the results framework, the Project’s GBV Action Plan, the management of the EVD 10 response, and the internal handling of UN agency contracts following the GBV allegations under the same. Attention to the results framework was inconsistent, as outlined above. As a result, many of the indicators had to be retrofitted at the ICR stage to understand the achievements of the Project. Finally, the Project’s challenges in the areas of safeguards, GBV, GRM, and social risks were only scantily reflected in the Implementation Supervision Reports prior to 2020. 100. On balance, the WB demonstrated commitment to the Project, putting significant resources and staffing to ensure its success and achieving significant results despite the challenging country context. Overall spending on the supervision of the Project was over US$ 9 million, with peaks during the EVD response years and FY23. The cost of supervision reflects the high cost of doing business in the Democratic Republic of Congo and the need for extensive support from a large group of staff. Over 50 WB staff and consultants were involved in supervision. 101. The Project flexibly responded to evolving needs and circumstances during its 10 years of implementation. There were moderate shortcomings in supervision, which bring the supervision rating to Moderately Satisfactory. The WB’s overall performance is rated as Moderately Satisfactory, based on the above ratings for the quality of entry and supervision and to align it with the Project’s overall outcome rating. D. RISK TO DEVELOPMENT OUTCOME 102. The PBF mechanism is highly dependent on external financing since domestic funding did not fill in the financing when Project financing for PBF ended in June 2023. The PMNS project has now signed contracts with health facilities to continue PBF for an additional year starting in July 2024 in six of the PDSS provinces. Still, the approach fell into a vacuum in the in-between and the remaining provinces. The PMNS has a more limited scope, providing only about a quarter of the financing available under the PDSS and paying for eight indicators instead of 22. Government commitment to increased domestic funding has seen only limited realization. 12 The December 2018 restructuring paper mentions the issue of community resistance against the response activities, and the need to understand the causes. In the AF3 project paper of February 2019, the only references to the widespread violence around the Ebola response were to ‘’community resistance to Ebola responders” and “respondent teams have been directly attacked by members of the community”. The AF4 project paper of March 2020 discusses the violence in more detail but fails to identify the link between the response characteristics and violence, attributing it to political and governance aspects and not identifying the social aspects. Annex 1 of PAD3263 mentions the need to prepare an emergency activities section of the environmental and social management framework, but there is no evidence that an updated document was published. No further analysis or mitigation was proposed in either AF paper. Page 31 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 103. The Project did create important positive externalities. Various other donors, such as USAID, utilized variations of the PBF and DFF approaches in their own development projects. The Government built on the PBF approach to design its new “universal free maternity care” program, utilizing a pay-for-service approach in combination with quality checklists. In the second half of the Project, the PIU developed expertise in executing small construction contracts at much lower prices than previous initiatives. This expertise continues to be put to use under other donor-financed projects. Single contracts remain in place in at least half of the provinces. Although some EUPs have seen a decrease in activities following the scale-down or suspension of PBF, other projects and initiatives are using the installed capacity. V. LESSONS AND RECOMMENDATIONS 104. In the Democratic Republic of Congo’s underfunded health system, the Project provided evidence that a reliable amount of funding to facilities to cover essential services can tremendously impact on quality of health service delivery and utilization. At the same time, the question of the best way to implement the cash infusion is not yet fully resolved. The attractiveness of PBF lies in the fact that it provides structure and an accountability mechanism for the delivery of services, yielding tangible improvements in the quality of care. It provides benchmarks, clear expectations, a mechanism for verification, and guidance in a context where health workers and health facilities were previously left to their own devices. On the downside, the cost of implementing such a system is non-negligible financially and in terms of needed capacity. This does not mean there is no need for the PBF mechanism, but one could be more selective in the locations where it is applied or more flexible regarding what is included in the mechanism, especially in remote areas with difficult access. 105. Increasing the stream of funding to health facilities does not, by itself, resolve other systemic bottlenecks in the health system, most notably (i) broken supply systems and markets for pharmaceuticals, inputs, and equipment; (ii) dysfunctional systems for managing the public health workforce; and (iii) the need to upgrade and equip health facilities to a basic level. The Project utilized effective methods to increase competition in the pharmaceutical markets and rebuild PHC facilities. The latter, however, came late and only reached under 10 percent of facilities. On the flip side, the rebuilding effort utilized the significant capacity built at the EUPs through the PBF activities in the project's early years. Going forward, PBF projects in the Democratic Republic of Congo should balance the need for structural upgrades with the need for operating funds and ensure that structural upgrades are frontloaded where possible. 106. The Project made significant progress with the quality-of-care agenda by introducing a quality scoring tool and various quality improvement pilots, but given the low starting level, much more remains to be done. The importance of the quality agenda cannot be underestimated. In the last year of the PBF intervention, the Project supported the Government’s initiative to provide delivery services free of charge by waiving user fees. Coverage of institutional delivery, which had increased from about 82 percent at the IE baseline survey to 92 percent at the end- line survey, shot even higher. However, quality can decrease if there are no accompanying measures to upgrade staff skills, ensure adequate facilities, equipment, and inputs, and improve clinical protocols and processes. Therefore, the Government’s effort to expand access to institutional delivery should be accompanied by sufficient operating funds to compensate health facilities for the service, but also investments and staff upgrading. 107. Given the low level of Government spending on health, ensuring an efficient utilization of the funding is paramount. Although 78 percent of the Government health budget is spent on HR, the country needs to do urgent work addressing the inefficiencies in their management, including the lack of a reliable system to manage and fill positions and monitor personnel assignments. While the Project tried to contribute to the HR agenda by including financing for the retirement of civil servants, this approach was neither successful nor sufficient. A much more Page 32 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) comprehensive approach is needed in the future to address the HR challenges, including foundational interventions such as improving the governance and transparency of the public payroll. 108. In terms of the Ebola response, the EVD 10 response showed the limits of a mostly external, public health approach to disease containment in a fragile environment. Distrust in the response teams, misinformation, rumors, and failure to integrate local practices and consider local needs, structures, and knowledge fueled additional violence that slowed down the containment of the epidemic. Important lessons include the need to train and utilize local staff for tracing, treatment, and vaccination, adopt a “low profile”, ensure response teams are holistic and respond rapidly, integrate the response within the health system, include traditional practitioners, avoid turning the response into a business or military operation, provide economic opportunities to youth in affected areas in a transparent manner, and consider the high risks of GBV within the response itself. 109. Designing and implementing an effective M&E system in a fragile context with poor capacity is a major challenge. Teams can expect to have to update indicators, definitions, and sources of data frequently as circumstances evolve and capacity is built. To maintain credibility and trust in the data, it is critical to ensure clear definitions, proper documentation, and accurate indicator monitoring. Supervision missions should include discussing indicators with project implementation units and reviewing data sources, data collection methods, and calculations. Field visits should triangulate assumptions regarding data collection procedures in health facilities and intermediate data consolidation points. An M&E specialist should be brought on board in large or complex projects. The WB’s main tool for tracking results, the Operations Workspace, must continue to evolve to provide teams with relevant, accurate information in a user-friendly interface. 110. Discrimination against women and GBV continues to be a pervasive issue in the Democratic Republic of Congo. Although this information may not have been available at Project approval, we now know that gender distribution of human resources in health is far from equitable or optimal. Women represent only 22 percent of doctors and 47 percent of nurses and are particularly underrepresented in categories requiring longer studies and positions of responsibility. Discrimination, sexual exploitation, and abuse of female health workers are pervasive, which is particularly harrowing given that health facilities are often seen as a “first point of care” for GBV victims. Efforts to improve financing and payment of health facilities and health workers should monitor the involvement of women to ensure that mothers, children, and adolescents have a choice of access to both female and male service providers. In PBF, ensuring women’s substantial participation in verification and community involvement activities is essential. 111. AF does not necessarily justify an immediate expansion of the scope of a Project, and a Project’s ambition should be carefully tracked. AF 1 and 2 of the Project were accompanied by expansions in its scope and targets, which were overambitious and unrealized. This reflects a tendency to put pressure on projects that are seen as performing well, leading teams to overpromise. Even in AF4, additional financing for the PDSS was well justified, considering that the PBF scheme ran for much longer than originally envisaged. Yet somehow, the AF4 paper still had promised to “increase” financial transfers to health facilities, something that was not feasible. 112. The Democratic Republic of Congo requires health financing to improve the “regular” functioning of the health system and prevent and tackle frequent epidemics. Although the CERC provided a means to respond rapidly to the EVD emergency, the reassignment of funds somewhat strangled the other Project activities. To provide reasonable predictability of funds for strengthening health systems, a dedicated epidemic prevention and control project is a must alongside other operations focused on strengthening health systems. Finally, the DRC's specific challenges and needs require constant adaptation and flexibility of Projects, which benefit from close collaboration across sectors and specialties within the WB and outside. Page 33 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS @#&OPS~Doctype~OPS^dynamics@icrresultframework#doctemplate A. RESULTS FRAMEWORK PDO Indicators by Outcomes Improve the utilization of maternal and child health services in targeted areas. Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year (PDO 1) Women having at least 4 36.00 Jun/2016 55.00 Jun/2023 65.00 Jun/2023 antenatal care visits before delivery (%) (Percentage) Comments on achieving targets The original design included a similar PDO indic. w/ 3 ANC visits. Since the HMIS was still in an incipient stage, a provisional baseline & target were set with the goal of updating them once better data were available. Baseline was 29.3, and target was 39 (delta 9.7pp). In Mar. 2017 (AF1), HMIS was used to update baseline & target: baseline became 57 and target became 65 (delta 8 pp). The Feb. 2020 restructuring changed the definition of the indicator in line with WHO guidelines of > 4 ANC visits, recomputed the baseline accordingly (36) w/ HMIS, and set a new end target of 55 (delta 19 pp). Revised definition, baseline, and target provide consistency and are used in the ICR for both phases. The % achievement is 100%*(66-36)/(55-36)=153%. Triang.: DHS show the likelihood of 4 ANC visits increased in PDSS provinces from 49 to 52% between 2014 and 2023. IE surveys do not show an increase in the likelihood of 4 visits but do increase in skilled ANC from 81 to 93% from 2016 to 2021. (PDO 2 DROPPED) Percentage of 26.90 Jun/2016 45 Jun/2023 0 Jun/2023 children aged between 6-23 months receiving preventive nutritional services at least four times per year. (Percentage) Comments on achieving targets PDO 2 is an original PDO indicator w/ baseline of 26.9 and target of 35. In AF1, it was replaced with an IRI on "Percentage of children under 24 months participating in the Growth Monitoring and Promotion activities at community level" with a baseline of 26.9 and target of 45. The IRI was dropped in the Feb. 2020 restructuring. The ICR uses the PDO indicator in the evaluation of Phase I. The % achievement on this indicator is 0% as the activities were not implemented. Page 34 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT (PDO 3) New curative consultations 0.38 Jun/2016 0.50 Jun/2023 0.55 Jun/2023 per capita per year (Text) Comments on achieving targets PDO 3 is an original indicator w/ baseline of 0.3 (based on early-stage HMIS) target of 0.6 (delta 0.3). In Mar. 2017, baseline was changed to 0.25, target was set at 0.5 (delta 0.25). In Feb. 2020, based on a review of the data sources, the baseline was recomputed (0.38), target was set at 0.5 (0.12 delta). The significant reduction in the delta is a reduction in the scope of the Project, so achievement is computed separately for Phase I and Phase II. Since the original baseline was an estimate and the recomputed baseline (0.38) was based on actual data, the ICR uses the latter for both phases. The Phase I target for the ICR is the recomputed baseline (0.38) plus the Mar 2017 delta (0.25). The % achievement is (0.55-0.38)/(0.63-0.38)=68% for Phase I and (0.55- 0.38)/(0.50-0.38)=142% for Phase II. Triang.: the DHS shows an increase in the likelihood of consultation for child <5 with fever, from 39.9% in 2014 to 50.7% in 2023 (national level). (PDO 4) People who have received 0 Jun/2016 NA Jun/2023 23,379,582 Jun/2023 essential health, nutrition, and population (HNP) services (Number) Comments on achieving targets PDO 4 was introduced in Mar. 2017 (AF1) as a cumulative number with a baseline of 0 and a target of 14.35 M and modified in March 2018 (AF2) to a target of 25 M. In Feb. 2019 (AF3), the indicator was modified to become an annual number with an incorrect baseline of 0 and a target of 7.15 M. In the Feb. 2020 restructuring, the baseline for the annual number was corrected (5,208,170) and the annual target was increased from 7.15 M to 7.8 M to account for the recomputed objectives under the three PDO breakdowns (PDOs 5, 6, and 7). As explained under PDO 6, the baseline, target and achievements for breakdown PDO 6 were flawed. The achievements for PDO 6 were re-computed and PDO 4 achievements were reconstructed based on them. However, there is no valid target for PDO 6 and, therefore, no valid target for PDO 4, and PDO 4's % achievement cannot be computed. (PDO 5) Number of children 0 Jun/2016 6,542,659 Jun/2023 6,481,985 Jun/2023 immunized (Number) Comments on achieving targets PDO 5 is a PDO 4 breakdown indicator introduced in Feb. 2019 (AF3) w/ annual target of 1.25 M and an (incorrect) baseline of 0. In Feb. 2020, the baseline was corrected (434,750) and the annual target was decreased from 1.25 M to 1,063,706. The 2020 RP does not present a reason for the change but the ICR notes that the 2019 target was higher than the est. nr. of children under 1. The lower target still corresponds with a target of 80.4% of the est. population for 2021, contradicting the PDO 8 target of 70%. The ICR goes by a target of 70% to ensure consistency between PDO 5 and 8. Also, given that the CRI is supposed to be a cumulative number, such cumulative target and achievement were reconstituted for the ICR. With an annual target of 70% of the est. nr. of children under 1 year in the target areas for 2016-2023Q2, the cumulative end target is 6,542,659. The % achievement is 6,481,985/6,542,659=99%. Triangulation: See under PDO 8. (PDO 6) Number of women 0 Jun/2016 NA Jun/2023 8,557,951 Jun/2023 and children who have Page 35 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT received basic nutrition services (Number) Comments on achieving targets PDO 6 is a PDO 4 breakdown indicator. According to the RP, PDO 6 represented the nr. of first growth monitoring visits for children age 6 to 23 months. The ICR found that the est. nr. of children in that age range (6% of the pop.) is only around 1.5 M. and the baseline/target/achievement referred to the total nr of growth monitoring visits for children 0-23 months. Given that the CRI is supposed to count the nr. of beneficiaries (and not visits), the ICR moves these baseline/target/achievement to a new indicator PDO 6b NEW. The ICR retrofits PDO 6 by using the IE surverys to estimate (i) nr of women receiving iron suppl. during pregnancy; (ii) nr of children under 5 who received preventive care visits. The cumulated nr of women and children thus served is 8,557,951. There is no basis for fixing an alternate target for PDO 6 ex post and the % achievement for this PDO cannot be computed. (PDO 7) Number of deliveries 0 Jun/2016 7,864,179 Jun/2023 8,339,646 Jun/2023 attended by skilled health personnel (Number) Comments on achieving targets PDO 7 is a PDO 4 breakdown indicator introduced in Feb. 2019 (AF3) w/ annual target 1.7 M and (incorrect) baseline 0. The target is 143% of the est. nr. of pregnant women in 2021 in the original Project areas. In Feb. 2020, a baseline for the annual nr. was filled in (421,214), and target was lowered from 1.7 M to 1.25 M, equivalent to a target of 94.4% of the est. nr. of pregnant women in 2021 in the expanded Project areas. This target was still higher than the 85% target under IRI 5. ICR (i) recomputes a cumulative target in line with CRI guidelines and (ii) sets a target of 85% in line with IRI 5. ICR cumulative end target is 7,864,179. % achievement is 8,339,646/7,864,179=106%. Triangulation: DHS show that skilled delivery care increased from 79 to 86% in PDSS provinces between 2014 - 2023. This is just above the target set under IRI6, so HMIS and DHS yield consistent results. The IE surveys also show that institutional delivery increased from 81 to 92% between 2016 and 2021. (PDO 6a NEW) Number of growth 4,352,206 Jun/2016 5,505,670 Jun/2023 7,851,267 Dec/2024 monitoring visits for children under the age of 24 months (Number) Comments on achieving targets PDO 6a is a new indicator at the time of the ICR that recovers the baseline, target, and achievements erroneously reported under PDO 6 as a breakdown of the CRI. That indicator was introduced in AF3 w/ annual target of 4.2 M and an (incorrect) baseline of 0. In Feb. 2020, the baseline was updated (4,352,206). The indicator represents the annual number of growth monitoring visits for children age 0 to 23 months. The baseline was 4,352,206, target was 5,506,670, and achieved value in 2023 was 7,851,267. The % achievement is therefore (7,851,267-5,506,670)/5,506,670=303%. (PDO 8) Children Fully Immunized (%) 62.00 Jun/2016 70.00 Jun/2023 71.00 Jun/2023 (Percentage) Page 36 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT Comments on achieving targets PDO 8 is an original indicator w/ baseline of 37.4 (based on early-stage HMIS data) and target 55 (delta 17.6 pp). In May 2017, the baseline was recalculated to 54 and the target was set at 65 (delta 11 pp). In Feb. 2020, the team recomputed baseline and target based on all the available data. Also, the definition of the indicator was changed (due to a change in the national vaccine guidelines). The baseline became 62, and the target was set at 70 (delta 8 pp) Given the adjustment to a higher baseline, the change in delta from 11 pp to 8 pp does not constitute a decrease in the indicator's ambition. Hence, the ICR uses the 2020 revised baseline and target for both phases. The % achievement is (71-62)/(70-62)=113%. Triangulation: These HMIS results were triangulated with the IE surveys, which showed increase in the likelihood of a child having all basic vaccinations from 27 to 49 percent (albeit for a slightly different age group and set of vaccines). Improve the quality of maternal and child health services in targeted areas Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year (PDO 9) Average score of the quality 21.00 Jun/2016 60.00 Jun/2023 64.00 Jun/2023 checklist at the health centers (%) (Percentage) Comments on achieving targets PDO 9 is an original indicator whose baseline was estimated at 47.5 (based on a sample) and target was set at 65 (delta 17.5 pp), with a view of updating it once the data were collected from all PDSS-contracted health facilities. In Mar. 2017 (AF1), the baseline was deleted and the target was changed to 55. In March 2018 (AF2), the baseline was updated to 20 and the target was changed to 60. In Feb. 2020, the baseline was updated to 21 based on the complete data, and the target was maintained at 60 (delta 39 pp). Since the original baseline was only estimated and the Feb. 2020 baseline is the actual computed value, the ICR uses the Feb. 2020 baseline for both phases. Given the much lower computed baseline than original estimated baseline, the decrease in final target from 65 to 60 does not constitute a decrease in the level of ambition of the indicator. Hence, the ICR uses the final target (60) for both phases. The % achievement is (64-21)/(60-21)=110.3%. Provide an immediate and effective response to an eligible crisis or emergency Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year (PDO 10) Eligible individuals 0.00 Jan/2019 120,000.00 Jul/2019 217,157.00 Jun/2023 vaccinated during EVD (# - cumulative) (Number) Comments on achieving targets PDO 10 was introduced in February 2019 (AF3). The baseline and target were not modified during the Project. The EVD10 outbreak ended in 2020. WHO reports on SRP 1-3 and SRP 4 report 141,633 for SRP 1-2 and 26,567 for SRP 4, i.e. total is 168,200. According to the Government results framework, the total number of people vaccinated was 217,157. The Goverment's final report on EVD response, meanwhile, has a total of 303,262 people vaccinated. The % achievement is 168,200/120,000 = 140% using the WHO reports. Page 37 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT Intermediate Indicators by Components Improve the utilization of health services through PBF Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year (IRI 1) New acceptors of modern 5.00 Jun/2016 15.00 Jun/2023 17.00 Jun/2023 contraceptives (%) (Percentage) Comments on achieving targets IRI 1 was introduced in the Feb. 2020 restructuring. This is a redundant indicator whose target (15) is contradictory with the set targets for IRI 2 (15) and IRI 3 (also 15, should have been 30). For the ICR, this indicator was not taken into account. Instead, the ICR uses IRI 3 with a corrected target of 30, which represents the sum of the targets of IRI 1 and IRI 2. (IRI 2) Existing users of modern 2.00 Jun/2016 15.00 Jun/2023 10.00 Jun/2023 contraceptives (%) (Percentage) Comments on achieving targets IRI 2 was introduced in the Feb. 2020 restructuring. This is a redundant indicator whose target (15) is contradictory with the targets for IRI 1 (15) and IRI 3 (also 15, should be 30). For the ICR, this indicator was not taken into account. Instead, the ICR uses IRI 3 with a corrected target of 30, which represents the sum of the targets of IRI 1 and IRI 2. (IRI 3) Contraceptive prevalence (new 6.00 Jun/2016 30 Jun/2023 27.00 Jun/2023 and existing female users %) (Percentage) Comments on achieving targets IRI 3 was introduced in the Feb. 2020 restructuring as replacement for an original indicator. For the purposes of the ICR, this indicator is used instead of IRI 1 and 2. The target for this IRI should have been the sum of the targets for IRI 1 and 2. Therefore, the ICR uses a modified target so as to be consistent with the IRI 1 and 2 targets (30 as the sum of 15 and 15). The % achievement for both phases is (27-6)/(30-6)=87.5%. Triangulation: These DHIS-2-based results were triangulated using the 2014 and 2023 DHS surveys. The surveys show that modern contraceptive prevalence rates increased from 9 to 13% in PDSS provinces. The positive trend in the DHS confirms that the increase in the administrative data is not entirely due to an increase in reporting but also to an actual increase in utilization of contraceptives. (IRI 4) First time adolescent girls 0.55 Jun/2016 6 Jun/2023 5.72 Jun/2023 acceptant of modern contraceptives (%) (Percentage) Page 38 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT Comments on achieving targets IRI 4 was introduced in the Feb. 2020 restructuring. The indicator target was not modified. Since adolescent health was included in Phase I, this indicator did not represent a change in scope of the Project but rather a better measurement - it therefore applies to both phases. The baseline and target were estimated using the total number of adolescents (32% of the population) rather than the estimated number of adolescent girls (16% of the population). For the ICR, both the baseline and the target were rescaled to include only girls (baseline is 0.55 instead of 2.27; target is 6 instead of 3). The percentage achievement for both phases is (5.72-0.55)/(6- 0.55)=95.3%. Triangulation: The DHS shows that the adolescent (15-19 years) birth rate decreased from 138.1 to 91 between 2014 and 2023 (nationally). This decrease is in line with increased modern contraceptive use. 2023 DHS data are not available at the province level. (IRI 5) Births attended by skilled 52.00 Jun/2016 85.00 Jun/2023 98.00 Jun/2023 professional (%) (Percentage) Comments on achieving targets IRI 5 was introduced in the Feb. 2020 restructuring. The indicator is a percentage version of PDO 7 and did not represent an expansion of the scope of the Project but a different way of measuring Project achievements. Therefore, it applies to both phases. Since PDO 7 did not have targets that matched the estimated population, the ICR resets PDO 7 target to be in line with IRI 5 (see under PDO 5 comments). The % achievement is (98- 52)/(85-52)=139%. Triangulation: the DHS surveys show that skilled delivery care increased from 79% to 88% in PDSS provinces between 2014 and 2023. (IRI 6) Health service delivered to 0.00 Jun/2016 20,000,000.00 Jun/2023 19,979,241.00 May/2023 target population through PDSS support (# – annual) (Number) Comments on achieving targets IRI 6 was introduced in the Feb. 2020 restructuring to reflect annual number of verified services (which is a stricter criterion than reported services). This is the only indicator that was sourced from the PBF-IT system. At the time of its introduction, the indicator did not represent an expansion of the scope of the Project but a better means of measuring its results. Therefore, it applies to both phases of the Project. The percentage achievement for both phases is 19,979,241/20,000,000=99.9%. (IRI 7) Single contract signed at 2.00 Jun/2016 11.00 Jun/2023 11.00 May/2023 province level (# – annual) (Number) Comments on achieving targets IRI 7 is an original indicator whose formulation was updated in the Feb. 2020 restructuring. The target and baseline were not changed. The % achievement is (11-2)/(11-2)=100%. (IRI 8) Civil servants eligible for 0.00 Jun/2016 4,000.00 Jun/2023 0.00 May/2023 retirement in the Ministry of Health that have received their retirement indemnities/packages (# - cumulative) (Number) Page 39 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT Comments on achieving targets IRI 8 was introduced in March 2017 (AF1). However, the activities did not proceed and funding was reallocated. The % achievement is 0% for Phase I of the Project. The indicator should logically have been dropped in the Feb. 2020 restructuring (although it wasn't), and therefore the ICR does not take the indicator into account for the evaluation of Phase II. (IRI 9) Poor people benefiting from 0.00 Jun/2016 20.00 Jun/2023 29.00 Jun/2023 fee exemption mechanisms (%) (Percentage) Comments on achieving targets IRI 9 was part of the original design as an absolute number, and was modified in the Feb. 2020 restructuring to be a percentage. The original indicator had an extremely low target of 6,500 poor people, compared to the estimated number of poor people in project areas, which is above 20 million. The Feb. 2020 percentage indicator used a denominator equal to 5 percent of the population. However, the estimated % of poor people outside of Kinshasa is 75%, and therefore the chosen calculation does not appropriately reflect the indicator name. Due to the flawed design, this indicator is not considered reliable for the purposes of the ICR. Instead, the ICR uses two more fitting indicators to measure financial protection: IRIs 22 and 23. (IRI 17 DROPPED) Percentage of 0 Jun/2016 50 Jun/2023 0 Jun/2023 families participating in parental education sessions at community level (Percentage) Comments on achieving targets IRI 17 was introduced in May 2017 (AF1) to reflect the expansion of the scope of the Project to community- based nutrition. The indicator was dropped in the Feb. 2020 restructuring as no activities were carried out. The % achievement on this indicator is 0%. (IRI 22 NEW) Reduction in OOP 0 Jun/2016 50 Jun/2021 76 Jun/2021 payment for child curative consultation in PHC centers (from IE) (Percentage) Comments on achieving targets IRI 22 is a new indicator for the ICR that aims to measure the financial protection of households against impoverishing and catastrophic health expenditures, replacing IRI 9. Since OOPs expenditures are a main driver of such expenditures, they are an appropriate indicator to measure financial protection. The ICR sets an ad hoc (but ambitious) target to reduce outpatient fee for child curative care OOP expenditures by 50 percent in real terms. This indicator is measured from the baseline and follow-up surveys of the IE in both PBF and DFF areas. The average baseline fee was CDR 5595 (nominal), the average endline fee was CDR 4508 (nominal), and cumulative inflation between the baseline and endline was 69 percent. Therefore the real reduction in the average fee was (5595-4508*(1-0.69))/5595=76%, and the percentage achievement is 76/50=152%. Page 40 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT (IRI 23 NEW) Reduction in OOP 0 Jun/2016 50 Jun/2021 81 Jun/2021 payment for ANC in PHC centers (from IE) (Percentage) Comments on achieving targets IRI 23 is a new indicator for the ICR that aims to measure the financial protection of households against impoverishing and catastrophic health expenditures, replacing IRI 9. Since OOPs expenditures are a main driver of such expenditures, they are an appropriate indicator to measure financial protection. The ICR sets an ad hoc (but ambitious) target to reduce outpatient fee for antenatal care OOP expenditures by 50 percent in real terms. This indicator is measured from the baseline and follow-up surveys of the IE in both PBF and DFF areas. The average baseline fee was CDR 3121 (nominal), the average endline fee was CDR 1943 (nominal), and cumulative inflation between the baseline and endline was 69 percent. Therefore the real reduction in the average fee was (3121-1943*(1-0.69))/3121=81%, and the percentage achievement is 81/50=162%. Improve the quality of health services through PBF Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year (IRI 10) Pregnant women counseled 15.00 Jun/2016 25.00 Jun/2023 27.00 Jun/2023 and tested for HIV (%) (Percentage) Comments on achieving targets IRI 10 was an original indicator w/ baseline est. at 17.5 (based on incipient HMIS) and target of 30 (delta 12.5 pp). In Mar. 2017 (AF1) the target was increased to 38 (delta 19.5 pp). In Mar. 2018 (AF2), the target was increased to 40 (delta 22.5 pp). In Feb. 2020, the baseline was recalculated using consistent HMIS data (15) and the target was set at 25 (delta of 10 pp). The lower delta on the lower baseline represents a reduction in the scope of the Project. Hence, the rating is calculated separately for Phase I and II. The ICR uses the recalculated baseline of 15 for both phases. For Phase I, the ICR sets the target at this baseline (15) plus a delta of 22.5 - i.e. Phase I target is 37.5. The % achievement for Phase I is (27-17.5)/(37.5-17.5)=48%. The % achievement for Phase II is (27-15)/(25-15)=120%. Triangulation: An identical indicator is not available in DHS, but DHS shows an increase in prevalence of (ever) HIV testing of women 15-49 years old from 19.5% (2014) to 25% (2023). (IRI 11) Health personnel receiving 0.00 Jul/2014 10,000.00 Jun/2023 5,296.00 May/2023 training (# - cumulative) (Number) Comments on achieving targets IRI 11 is an original indicator whose target was updated in March 2017 (AF1). The original target was 2,000, while the March 2017 target was 10,000. The indicator applies to both phases of the Project. The percentage achievement is 5,296/10,000=53.0%. (IRI 12) Average availability of tracer 64 Dec/2018 70.00 Jun/2023 74.00 May/2023 medicines at hospitals/health centers (%) (Percentage) Page 41 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT Comments on achieving targets IRI 12 was introduced in the Feb. 2020 restructuring, but it does not represent a raising or lowering of the Project's ambition but rather a better way of measuring the Project's achievements. Therefore the ICR uses this indicator for both phases. The target was set at 70%. The restructuring paper mentions that the baseline (which is presumably the 2015 or 2016 baseline) was overestimated at 69 %, while the Q4 2018 value was 64%. However, the baseline of the indicator was never updated, resulting in a minimal delta for the indicator (1pp improvement to 70%). Since the Q4 2018 value is the earliest available value that is comparable to the end target and end achievement measurement, the ICR uses 64% instead of 69% as the baseline.The percentage achievement is (74-64)/(70-64)=166.7%. Triangulation: IRI 12 was triangulated using data from the impact evaluation. See IRI 24 and 25. (IRI 13) Average performance score 72.00 Sep/2017 77.00 Jun/2023 75.00 May/2023 of Provincial Health Directorates under Single Contract (%) (Percentage) Comments on achieving targets IRI 11 was introduced in the February 2020 restructuring. The baseline and target were not modified during the Project. The achievement percentage is (75-72)/(77-72)=60%. (IRI 18 DROPPED) Number of newly 3,205 Dec/2016 500 Dec/2023 394 Dec/2023 diagnosed HAT patients (Number) Comments on achieving targets IRI 18 is an original indicator that was dropped in the Feb. 2020 restructuring. The Project included the purchase of (a) newly diagnosed HAT cases and (b) treated and cured HAT cases. Although the Project aimed to continue to purchase such cases at a rate of 500 per year, the number of cases dropped significantly during the life of the Project, reaching an all-time low of 394 reported cases for the whole country in 2023. The succesful control of HAT can be considered an achievement to which the Project contributed. Since HAT diagnoses continued to be included in the package of services purchased by PBF, the ICR sees no reason to drop this indicator in Phase II. (IRI 19 DROPPED) Exclusive 41 Jun/2016 53 Jun/2023 46 Dec/2023 breastfeeding for children under 6 months (Percentage) Comments on achieving targets IRI 19 was introduced in May 2017 (AF1) and dropped in Feb. 2020. The baseline was initially set as N/A, current value was 54, and target was 66. It is not clear where these values originated. The ICR re-estimated the baseline using the 2014 DHS for PDSS provinces (41%). The draft 2023 DHS report does not break down this indicator by province, but does show a national-level improvement from 48 to 53 % between the 2014 and 2023 rounds of the DHS. The ICR extrapolates the national improvement (5 pp between 2014 and 2023) on top of the re- estimated baseline for the PDSS provinces (41%). Therefore the estimated final achievement is 46%. The indicator's 2017 current value was out of line with the 2014 DHS estimate and resulted in an inflated target of 66. Therefore the ICR sets an alternative target by adding the original delta (12 pp) to the recomputed baseline (41%). The ICR target is therefore 53%. The % achievement for phase I is (46-41)/(53-41)=43%. Page 42 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT (IRI 20 DROPPED) Average quality of 20 Jun/2023 55 Jun/2023 20 Jun/2023 nutritional services (Percentage) Comments on achieving targets IRI 20 is an original indicator that was dropped in the Feb. 2020 restructuring. The activities related to this IRI were not implemented and no data were available at the time at the ICR, and therefore the ICR considers that no progress was made beyond the baseline and % achievement is 0. (IRI 21 DROPPED) Health facilities 0 Jun/2016 80 Jun/2023 0 Jun/2023 receiving Client Tracer and Satisfaction Survey feedback (Percentage) Comments on achieving targets IRI 21 is an original indicator that was dropped in the Feb. 2020 restructuring. No data were available at the time at the ICR, and therefore the ICR considers that no progress was made beyond the baseline and % achievement is 0. (IRI 24 NEW) Availability of essential 45 Jun/2016 51 Jun/2021 69 Jun/2021 drugs in PHC centers (22 items) (from IE) (Percentage) (Percentage) Comments on achieving targets IRI 24 is a new indicator for the ICR. It was extracted from the PHC facility surveys that were carried out as part of the impact evaluation, and is used to triangulate the results under IRI 12. In health zones that were covered by the baseline and endline surveys, the availability of 22 essential drugs in health centers increased from 45% to 69%. For the purposes of the ICR, the desired delta is chosen in line with the delta in IRI 12, which is 6 pp. increase. Therefore the ICR target is 51%. The percentage achievement is (69-45)/(51-45)=400%. (IRI 25 NEW) Availability of essential 70 Jun/2016 76 Jun/2021 84 Jun/2021 drugs in hospitals (22 items) (from IE) (Percentage) Comments on achieving targets IRI 25 is a new indicator for the ICR. It was extracted from hospital surveys that were carried out as part of the impact evaluation, and is used to triangulate the results under IRI 12. In health zones that were covered by the baseline and endline surveys, the availability of 22 essential drugs in hospitals increased from 70% to 84%. For the purposes of the ICR, the desired delta is chosen in line with the delta in IRI 12, which is 6 pp. increase. Therefore the ICR target is 76%. The percentage achievement is (84-70)/(76-70)=233%. (IRI 26 NEW) Number of health 0 Jun/2016 175 Jun/2024 175 Jun/2024 facilities rehabilitated through a decentralized contracting modality at the EUP level (Number) Comments on achieving targets IRI 26 is a new indicator for the ICR. It applies to Phase II of the Project and reflects the significant efforts made to improve the physical infrastructure of health facilities. The target is reconstruction of approximately 7 Page 43 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT percent of contracted health facilitie. The average number of contracted facilities is 2500. The project reconstructed 175 facilities. (IRI 27 NEW) Percentage of women 15.30 Jun/2016 22.95 Jun/2021 23.10 Jun/2021 who received skilled antenatal care during the first trimester (from IE) (Percentage) Comments on achieving targets IRI 27 is a new indicator for the ICR and is meant to reflect the quality of ANC, complementing the administrative data on utilization of ANC reflected under PDO 1. The ICR sets an ambitious target of 50 percent increase over baseline of the percentage of women who received skilled ANC in the first trimester. The measurement were taken from the baseline and endline household surveys of the impact evaluation, which were implemented in six PDSS provinces. The % achievement is (23.1-15.3)/(22.95-15.3)=102%. (IRI 28 NEW) Percentage of women 44.50 Jun/2016 66.75 Jun/2021 71.90 Jun/2021 who received antimalarial medicines during pregnancy (from IE) (Percentage) Comments on achieving targets IRI 28 is a new indicator for the ICR and is meant to reflect the quality of ANC, complementing the administrative data on utilization of ANC reflected under PDO 1. The ICR sets an ambitious target of 50 percent increase over baseline of the percentage of women who received antimalarial prophylaxis during prenatal care. The measurement were taken from the baseline and endline household surveys of the impact evaluation, which were implemented in 6 PDSS provinces. The % achievement is (71.9-44.5)/(66.75-44.5)=123%. (IRI 29 NEW) Percentage of women 44.80 Jun/2016 67.20 Jun/2021 77 Jun/2021 (who gave birth within the last two years) who discussed family planning with a health care provider (from IE) (Percentage) Comments on achieving targets IRI 29 is a new indicator for the ICR and is meant to reflect the quality of maternal care, complementing the administrative data on utilization of institutional delivery reflected under PDO 7 and the measure of structural quality of care under PDO 9. The ICR sets an ambitious target of 50 percent increase over baseline of the percentage of postpartum women (birth within the last 2 years) who discussed family planning with a health care provider. The measurements was taken from the baseline and endline household surveys of the impact evaluation, which were implemented in 6 PDSS provinces. The % achievement is (77-44.8)/(67.2-44.8)=144%. Provide an immediate and effective response to an eligible crisis or emergency Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year Page 44 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT (IRI 14) Contacts of suspected or 0.00 Jan/2019 0.00 Jul/2019 5.90 Jun/2020 confirmed EVD cases lost to follow up (%) (Percentage) Comments on achieving targets IRI 14 was introduced in Feb. 2019 (AF3). The baseline and target were not modified during the Project. The indicator was meant to be monthly. The Goverment reported in the results framework on a trimester basis. WHO reported on the monthly percentage from July 2019 and June 2020 only: it varies between 0 and 23%, with the final two months at 23% (April 2020) and 3% (May 2023). The WHO report does not include a numerator or denominator; therefore, it is not possible to reconcile the WHO-reported % with the Government- reported numbers. The ICR calculates the average % of lost contacts throughout the duration of the Project using the Government's results framework. The total number of lost contacts was 62,071 out of 1,048,387 = 5.9%. The achievement percentage is therefore 100%-5.9%=94.1%. (IRI 15) Suspected EVD samples 0.00 Jan/2019 100.00 Jul/2019 70.30 Jun/2020 received that have been tested (%) (Percentage) Comments on achieving targets IRI 15 was introduced in Feb. 2019 (AF3). The baseline and target were not modified during the Project. The indicator was meant to be monthly. The government results framework reports that the number of samples tested was 80,368, representing 70.3% of the 114,316 suspected EVD samples. The % achievement is, therefore, 70.3%. WHO reports do not contain distinct different number of samples tested (220,000). This indicator was not part of the key indicators under SRP4. (IRI 16) Suspected and confirmed EVD 0.00 Jan/2019 100.00 Jul/2019 88.28 Jun/2020 cases for whom safe and dignified burials have been carried out (%) (Percentage) Comments on achieving targets IRI 16 was introduced in Feb. 2019 (AF3). The baseline and target were not modified during the Project. The indicator was meant to be monthly. The government reported in the results framework on a trimester basis, with a total number of safe and dignified burials (SDB) of 20,505 out of 28,040 suspected and confirmed deaths (73%). WHO reported on the monthly % from July 2019 and June 2020 only: it varies between 43 and 95%, with the final two months at 95% (April 2020) and 88% (May 2023). The WHO also reports that the total number of SDB was 26,526, representing 88% of the 30,048 suspected and confirmed EVD deaths during that period. The ICR uses the average % over the project's lifetime using WHO as the source. The % achievement is, therefore, 88%. Page 45 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT TABLE 8: ICR RATING OF THE PDO AND IR INDICATORS Differential ICR # ICR use Indicator rating Phase Achievement I / Phase II Part 1: Improve the utilization of maternal and child health services in targeted areas PDO1 Yes Women having at least 4 antenatal care visits before delivery (%) No 153% Fully achieved/surpassed (100%+) PDO 2 Yes Percentage of children aged between 6-23 months receiving preventive Yes, 0% Not Achieved (less than 64%) DROPPED nutritional services at least four times per year Phase I only No Percentage of children under 24 months participating in the Growth Monitoring and Promotion activities at community level PDO 3 Yes New curative consultations per capita per year Yes, 68% Partially Achieved (65%-79%) two phases 142% Fully achieved/surpassed (100%+) PDO 4 Yes People who have received essential health, nutrition, and population No NA Cannot be calculated due to lack of (HNP) services (Corporate) target. PDO 5 Yes - Number of children immunized (Corporate Breakdown) No 99% Substantially achieved (80%+) PDO 6 Yes - Number of women and children who have received basic nutrition No NA Cannot be calculated due to lack of services (Corporate Breakdown) target. PDO 6a Yes Number of growth monitoring visits for children under the age of 24 No 135% Fully achieved/surpassed (100%+) NEW months PDO 7 Yes - Number of deliveries attended by skilled health personnel (Corporate No 106% Fully achieved/surpassed (100%+) Breakdown) PDO 8 No, Children Fully Immunized (% No 113% Fully achieved/surpassed (100%+) use PDO 5 IRI 1 No, New acceptors of modern contraceptives (%) NA NA Cannot be calculated due to use IRI 3 incorrect target. IRI 2 No, Existing users of modern contraceptives (%)) NA NA Cannot be calculated due to use IRI 3 incorrect target IRI 3 Yes Contraceptive prevalence (new and existing female users %) No 88% Substantially achieved (80%+) IRI 4 Yes First time adolescent girls acceptant of modern contraceptives (%) No 95% Substantially achieved (80%+) IRI 5 Yes Births attended by skilled professional (%) No 139% Fully achieved/surpassed (100%+) IRI 6 Yes Health service delivered to target population through PDSS support (# – No 100% Substantially achieved (80%+) annual) IRI 7 Yes Single contract signed at province level (# – annual) No 100% Fully achieved/surpassed (100%+) Page 46 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT Differential ICR # ICR use Indicator rating Phase Achievement I / Phase II IRI 8 Yes Civil servants eligible for retirement in the Ministry of Health that have No 0% Not Achieved (less than 64%) received their retirement indemnities/packages (# - cumulative) IRI 9 No Poor people benefiting from fee exemption mechanisms (%) No NA Cannot be calculated due to incorrect definition. IRI 17 Yes Percentage of families participating in parental education sessions at Yes, 0% Not Achieved (less than 64%) DROPPED community level Phase I only IRI 22 Yes Reduction in OOP payment for child curative consultation in PHC centers No 152% Fully achieved/surpassed (100%+) NEW (from IE) (%) IRI 23 Yes Reduction in OOP payment for ANC in PHC centers (from IE) (%) No 162% Fully achieved/surpassed (100%+) NEW Part 2: Improve the quality of health services through PBF PDO 9 Yes Average score of the quality checklist at the health centers (%) No 110% Fully achieved/surpassed (100%+) IRI 10 Yes Pregnant women counseled and tested for HIV (%) Yes, two 53% Not Achieved (less than 64%) phases 120% Fully achieved/surpassed (100%+) IRI 11 Yes Health personnel receiving training (# - cumulative) No 53% Not Achieved (less than 64%) IRI 12 Yes Average availability of tracer medicines at hospitals/health centers (%) No 167% Fully achieved/surpassed (100%+) IRI 13 Yes Average performance score of Provincial Health Directorates under Yes, 60% Not Achieved (less than 64%) Single Contract (%) Phase II only IRI 18 Yes Number of newly diagnosed HAT patients (# - annual) No 104% Fully achieved/surpassed (100%+) DROPPED IRI 19 Yes Exclusive breastfeeding for children under 6 months (%) Yes, 42% Not Achieved (less than 64%) DROPPED Phase I only IRI 20 Yes Average quality of nutritional services Yes, 0% Not Achieved (less than 64%) DROPPED Phase I only IRI 21 Yes Health facilities receiving Client Tracer and Satisfaction Survey Feedback Yes, 0% Not Achieved (less than 64%) DROPPED Phase I only IRI 24 Yes Availability of essential drugs in PHC centers (22 items) (from IE) (%) No 400% Fully achieved/surpassed (100%+) NEW IRI 25 Yes Availability of essential drugs in hospitals (22 items) (from IE) (%) No 233% Fully achieved/surpassed (100%+) NEW Page 47 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT Differential ICR # ICR use Indicator rating Phase Achievement I / Phase II IRI 26 Yes Number of health facilities rehabilitated through a decentralized Yes, 100% Fully achieved/surpassed (100%+) NEW contracting modality at the EUP level (# - cumulative) Phase II only IRI 27 Yes Percentage of women who received skilled antenatal care during the first No 102% Fully achieved/surpassed (100%+) NEW trimester (from IE) (%) IRI 28 Yes Percentage of women who received antimalarial medicines during No 123% Fully achieved/surpassed (100%+) NEW pregnancy (from IE) (%) IRI 29 Yes Percentage of women (who gave birth within the last two years) who No 144% Fully achieved/surpassed (100%+) NEW discussed family planning with a health care provider (from IE) (%) Part 3: Provide an immediate and effective response to an eligible crisis or emergency PDO 10 Yes Eligible individuals vaccinated during EVD (# - cumulative) No 140% Fully achieved/surpassed (100%+) IRI 14 Yes Contacts of suspected or confirmed EVD cases lost to follow up (%) No 94% Substantially achieved (80%+) IRI 15 Yes Suspected EVD samples received that have been tested (%) No 70% Partially Achieved (65%-79%) IRI 16 Yes Suspected and confirmed EVD cases for whom safe and dignified burials No 88% Substantially achieved (80%+) have been carried out (%) Page 48 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT TABLE 9: DROPPED INDICATORS THAT WERE REPLACED OR SUBSTITUTED DURING THE PROJECT AND NOT CONSIDERED IN THE ICR RATINGS PDO Dropped indicator name Baselin Target Indicator background Replacement indicator(s) part e 1 Number of new and existing 194,48 500,000 Original indicator, dropped in IRI 1 - New acceptors of modern contraceptives (%) acceptors of modern contraceptive 0 Feb. 2020 and replaced with IRI 2 - Existing users of modern contraceptives (%) use IRI 1/4/5. IRI 3 - Contraceptive prevalence (new and existing female users %) 1 First time adolescent girls 0 50,000 New indicator with AF1, IRI 4 - First time adolescent girls acceptant of modern acceptant of modern dropped in Feb. 2020, contraceptives (%) contraceptives (Number) replaced with IRI 2. 1 Number of Direct Beneficiaries, of 0 20,090,000 Original indicator, dropped in PDO 4 - People who have received essential health, which female (cumulative) Feb. 2020 - supplanted by the nutrition, and population (HNP) services (annual) CRI indicators PDOI 2/3/4/5 PDO 5 - Number of children immunized (annual) plus the curative consultations PDO 6 - Number of women and children who have indicator PDO 7 (annual). received basic nutrition services (annual) PDO 7 - New curative consultations per capita per year (annual) 1 Single contract signed and 2 11 New indicator with AF1, IRI 7 - Single contract signed at province level (# – annual) implemented at province level (# – dropped in Feb. 2020. annual) (Number, Custom) Replaced with IRI 10. 1 Number of people benefiting from 5,248 6,500 Original indicator, dropped in IRI 9 - Poor people benefiting from fee exemption fee exemption mechanisms Feb. 2020. Replaced with IRI mechanisms (%) (Number) 13. 2 Average number of days with stock >30 >15 days New indicator with AF1, IRI 12 - Average availability of tracer medicines at out of tracer drugs in targeted days dropped in Feb. 2020. hospitals/health centers (%) health facilities on the day of the Replaced with IRI 8. visit Page 49 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT A. KEY OUTPUTS TABLE 10: KEY OUTPUTS Improve the utilization and quality of maternal and child health services in targeted areas. • Women having at least 4 antenatal care visits before delivery (%) • People who have received essential health, nutrition, and population (HNP) services PDO Indicators • Average score of the quality checklist at the health centers (%) • Children Fully Immunized (%) • New curative consultations per capita per year • Provincial EUPs set up and provincial teams trained in 11 Provinces: Kwilu, Kwango, Maï-Ndombe, Equateur, Sud-Ubangi, Monga, Tschuapa, Haut-Katanga, Haut-Lomami, Lualaba, and Maniema. In 2020, Equateur was dropped, Kinshasa and Nord- Kivu were added. In 2022, financing for Kwilu migrated to a different WB project (PMNS). • Health facilities with negotiated and signed contracts for MPA: 2205 (2016-2017), 2663 (2018), 2599 (2019), 2777 (2020), 2261 (2021), 2126 (2022); of which approximaly 2/3 with PBF facilities and 1/3 with control facilities • Hospitals with negotiated and signed contracts for CPA: 151 (2016-2017), 177 (2018), 175 (2019), 191 (2020), 193 (2021), 175 (2022) • ECZS with signed performance contracts: 140 (2016-2017), 156 (2018), 156 (2019), 160 (2020), 146 (2021), 138 (2022) Key Outputs • All contracted EUPs validated the quantity of care using PBF tools in all EUP (linked to the achievement of • ECZS and DPS evaluated quality of care using PBF tools in all provinces and health facilities. the PDO Outcome) • Performance-based grants calculated and transferred to health facility bank accounts on a quarterly basis • The external verification agency counter-verified the quantity and quality of care in 100 health facilities per trimester, with sampling based on risk. • DPS evaluated the performance of ECZ on a yearly basis in all years where they were included in the Project. Average performance varied from 67 percent to 78 percent. • Health facilities formed Indigent committees to identify indigent community members eligible for the fee exemptions: 57 percent by 2021. • PBF dashboard with up-to-date information on quantity and quality of services in contracted facilities. Page 50 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT Provide an immediate and effective response to an eligible crisis or emergency PDO Indicators • Eligible individuals vaccinated during EVD (# - cumulative) • Communications: partnerships established with local radios (140), international NGOs (12), national NGOs (17) and government institutions (7); training of 3,334 local leaders, religious leaders, teachers, nurses, traditional healers, and 75 journalists • Communications materials: 293,000 folders and 236,000 posters distributed • Surveillance: 1,289,724 suspected cases reported, of which 96.6% were investigated and 1.6% were confirmed positive. • Vaccination: 303,262 individuals, including 8,031 front line health workers • PoE screening: 178,372,341 individuals, of which 99.1% were screened • Clinical care: 1,162 individuals recovered (31.9%) • Infection prevention and control: 1,055 health centers equipped with infection prevention and control kits Key Outputs • WASH facilities installed in 8,246 communal sites (ports, markets, restaurants, churches, etc) and 621 schools (linked to the achievement of the PDO Outcome) • Timely decontamination of 426 households, 234 health facilities, 20 public places, and 7 schools (86% wihtin 72 hours of the initial alert). • Distribution of hygiene and EVD prevention kits to 11,143 housholds of confirmed cases • 104 teams carried out safe and dignified burials: 93% completion rate for 4,897 EVD death alerts. • Universal subsided for select services instituted in affected health zones and health facilities in Nord-Kivu. Approximately 1.5M services were paid for by the Project from September 2018-January 2019. Covered services included deliveries, minor and major surgery, curative consultations, and referred consultations. In 2029 and 2020 the list of covered services was modified to focus on external consultations and hospital admissions (excluding deliveries and surgery) – an additional 1.6M services was covered further. • Medicines were delivered to affected health facilities for a value of US$5.7 million. Disease Surveillance Strengthening and Response • Contacts of suspected or confirmed EVD cases lost to follow-up (%) Intermediate Results • Suspected EVD samples received that have been tested (%) Indicators • Suspected and confirmed EVD cases for whom safe and dignified burials have been carried out (%) Key Outputs See above (linked to the achievement of the Component) Strengthen Health System Performance - Financing and Health Policy • Civil servants eligible for retirement in the MoPHHSP who have received their retirement indemnities/packages (# - Intermediate Results cumulative) Indicators • Poor people benefiting from fee exemption mechanisms (%) Page 51 The World Bank Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ICR DOCUMENT • Health facilities formed Indigent committees to identify indigent community members eligible for the fee exemptions: 57 percent by 2021. Key Outputs • All contracts with health facilities specified up to 5% fee exemption to reimbursed by the Project (linked to the achievement of • Health infrastructure the Component) • WASH upgrades • Solar kits • Equipment kits for health facilities and hospitals Improve Governance, Purchasing, and Coaching and Strengthen Health Administration Directorates and Services through PBF • Health personnel receiving training (# - cumulative) Intermediate Results • Average availability of tracer medicines at hospitals/health centers (%) Indicators • Single contract signed at province level (# – annual) • Average performance score of Provincial Health Directorates under Single Contract (%) Key Outputs • Initial package of medicines distributed to health facilities (linked to the achievement of • Contraceptives and medicines distributed to facilities the Component) Improve Utilization and Quality of Health Services at Health Facilities through PBF • New acceptors of modern contraceptives (%) • First-time adolescent girls acceptant of modern contraceptives (%) • Pregnant women counseled and tested for HIV (%) Intermediate Results • Existing users of modern contraceptives (%) Indicators • Contraceptive prevalence (new and existing female users %) • Births attended by skilled professional (%) • Health service delivered to target population through PDSS support (# – annual) Key Outputs • Funds reserved for the purchase of medicines and inputs (20% of grants starting in 2020) (linked to the achievement of • Initial package of medicines distributed to health facilities the Component) • Contraceptives and medicines distributed to facilities Page 52 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) B. THEORY OF CHANGE FIGURE 5: THEORY OF CHANGE FOR PDO PARTS 1 AND 2 “IMPROVE THE UTILIZATION AND QUALITY OF MATERNAL AND CHILD HEALTH SERVICES IN TARGETED AREAS WITHIN THE RECIPIENT’S TERRITORY” Critical assumptions A = Health facilities have access to local markets where they can obtain the necessary supplies, equipment, and medicines. B = Health workers have the necessary basic skills upon which in-service training can build 53 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) FIGURE 6: THEORY OF CHANGE FOR PDO PART 3 “PROVIDE AN IMMEDIATE AND EFFECTIVE RESPONSE TO AN ELIGIBLE CRISIS OR EMERGENCY” 54 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ANNEX 2. WB LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Fatima El Kadiri El Yamani Team Leader Avril Dawn Kaplan Team Leader Hadia Nazem Samaha Team Leader Michel Muvudi Lushimba Team Leader Wezi Marianne Msisha Team Leader Kadiatou Balde Financial Management Specialist Lydie Madjou Financial Management Specialist Guy Kiaku Kindoki Procurement Specialist Mamata Tiendrebeogo Procurement Specialist Rahmoune Essalhi Procurement Specialist Lanssina Traore Procurement Specialist Jean-Claude Azonfack Procurement Specialist Christine Foma Environmental Specialist Christophe Ngongo Muzyumba Environmental Specialist Abdoulaye Gadiere Environmental Specialist Joelle Nkombela Mukungu Environmental Specialist Claude Lobo Environmental Specialist Cyrille Valence Ngouana Kengne Environmental Specialist Moise Bolamu Social Specialist Lydia Kanyembo Social Specialist Donat Vema Tunamau Social Specialist Lucienne M. M'Baipor Social Specialist Richard Everett Social Specialist Elena Segura Labadia Counsel Isabella Micali Drossos Counsel Siobhan McInerney-Lankford Counsel Joelle Mudi Nke Procurement Team Page 55 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) Mohammad Ilyas Butt Procurement Team Gil Shapira Team Member Lucie Lufiauluisu Bobola Team Member Yasmine Binti Sangwa Team Member Karine N. MOUKETO-MIKOLO Team Member Faly Diallo Team Member Claudia Rokx Team Member Supriya Madhavan Team Member Antoine V. Lema Team Member Meta Mobula Team Member Ghulam Dastagir Sayed Team Member Julie Luvisa Bazolana Team Member Sariette Jene M. C. Jippe Team Member Jeannine Kashosi Nkakala Team Member Saidou Diop Team Member Gyorgy Bela Fritsche Team Member Lombe Kasonde Team Member Laurence Elisabeth Marie-Paule Lannes Team Member Nikolai Alexei Sviedrys Wittich Team Member Amba Denise Sangara Team Member Luc Laviolette Team Member Christopher Thomas Andersen Team Member Marion Jane Cros Team Member Alice Museri Team Member Christelle Epuza Tandundu Team Member B. STAFF TIME & COST @#&OPS~Doctype~OPS^dynamics@icrannexstafftime#doctemplate A. STAFF TIME & COST Stage of Project Cycle Staff Time & Cost Page 56 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) No. of Staff Weeks US$ (including travel and consultant costs) Preparation FY14 3.350 77,346.25 FY15 36.369 255,409.43 FY16 16.200 135,941.01 FY17 0.000 88.18 FY18 0.000 .00 Total 55.92 468,784.87 Supervision/ICR FY15 8.450 93,033.72 FY16 72.757 631,264.97 FY17 99.355 754,659.81 FY18 121.977 736,245.83 FY19 144.098 1,239,712.60 FY20 133.240 1,565,733.06 FY21 110.126 995,540.04 FY22 119.327 1,186,477.37 FY23 164.428 1,323,811.62 FY24 58.353 472,735.83 FY25 18.230 158,031.37 Total 1,050.34 9,157,246.22 Page 57 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ANNEX 3. PROJECT COST BY COMPONENT 1. While the total approved financing amount 13 was US$714.5 million, the total disbursed amount was US$705.0 million. The difference is due to exchange rate fluctuations between the USD and Special Drawing Rights (SDR), as some of the financing agreements were expressed in SDR (Table 11). TABLE 11: PROJECT COMPONENTS AND COSTS Amount at Approval Amount at Actual spending Component of Parent Project Approval of AF4 at closing (US$ million) (US$ million) (US$ million) 1. Improve Utilization and Quality of Health 120 226.6 348.6 Services at Health Facilities through PBF (49.7%) 2. Improve Governance, Purchasing and 65.2 102.7 127.8 Coaching and Strengthen Health (18.1%) Administration Directorates and Services through PBF 3. Strengthen Health System Performance - 41.3 140.3 38.0 Financing, and Health Policy (18.1%) 4. Disease Surveillance Strengthening and 0 244.9 181.4 Response and CERC (25.7%) 5. Reimbursement of the Project Preparation 0 0 8.9 Advance (PPA) (1.3%) 226.5 714.5 705.1 2. The Project cost by component at the time of closing were calculated by the PIU. Except for EVD response, the disbursement categories do not map directly into the Project components. Compared to the amounts allocated at the approval stages, spending was much more concentrated in the PBF components and in direct support to front-line facilities through PBF grants (Components 1 and 2, and reimbursement of the PPA). The total spending can be broken down as shown in Table 12. TABLE 12: COSTS BY INTERVENTION Component Actual spending at closing % (US$ million) 1. PBF 120 68.8 2. EVD response 181.4 25.7 3. System strengthening 38.0 18.1 Total 714.5 100 13 Not including the partial cancellation of TFA6945. Page 58 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ANNEX 4. TRIANGULATION OF DATA AND IMPACT EVALUATION OF PBF A. PBF INVESTMENTS 1. In addition to the Project’s indicators, the ICR considers the results of the Project’s impact evaluation, which fielded baseline household and facility surveys in 11 provinces (2015) and end-line household and facility surveys in 6 provinces (2021-2022). The impact evaluation included randomized controlled assignment of PBF and “control” status among 58 health zones in 6 provinces (30 PBF and 28 DFF zones).14 Health facilities in the control zones received DFF consisting of the same amount of funding as the PBF facilities (with adjustments for population size), but their funding was not tied to their performance. (Table 13) TABLE 13: COMPARISON BETWEEN THE PBF AND DFF INTERVENTIONS PBF health zones DFF health zones Quarterly financial transfers Conditional on quality and quantity of services Unconditional on facility performance. to health facilities Transfer equals average transfer to PBF facilities in the same province, adjusted for catchment population. Supervision Enhanced supervision with structured quality Routine supervision by health zone and checklists, verification of registries, province. community verification. Spending guidelines Quarterly business plan. Minimum of 20% of transfer to be used No more than 50% of total revenue to be for medication and consumables. spent on personnel bonuses. Minimum of 20% of transfer to be used for medication and consumables. Determination of personnel Use of project “indexes” tool is mandatory. No guidelines. bonuses Startup investment bonus US$2500 for health centers Same as PBF US$3000 for referral centers US$5000 for hospitals Provision of centrally Yes Same as PBF procured medications and family planning inputs Financing of health zone Transfers according to performance contract Unconditional transfer equal to average teams tied to verification. received by PBF health zones in the same province. Source: Adapted from Fink and Shapira (2022). (Annex 7, ref. 13) 2. It is important to note that the objective of the impact evaluation was to evaluate the impact of linking financing to performance (the PBF model) as opposed to unconditional additional funding (the DFF model). There was no “pure control” group that did not receive funding. The case-control PBF-DFF analysis shows a modest positive impact of PBF compared to DFF in most dimensions, though only a few are statistically significant (Annex 7, ref. 13 and 48). The study found that PBF increased coverage of incentivized services by 3 percentage points on 14 Although the original design included 100 health zones in 11 provinces (50 PBF and 50 DFF health zones), the impact evaluation was scaled down due to EVD 9 and higher-than-expected survey costs. Page 59 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) average compared to DFF. The largest improvements were in the use of modern family planning methods and in the initiation of antenatal care during the first trimester of pregnancy. TABLE 14: IMPACT OF PBF ON COVERAGE OF INCENTIVIZED SERVICES, COMPARED TO DFF Service Mean in DFF Impact and 95% N.Obs. group confidence interval Early antenatal care initiation (first trimester) 0.18 0.08 (0.03–0.13) 4135 At least 4 antenatal care visits during the last pregnancy 0.34 0.03 (− 0.06 to 0.12) 4134 Antenatal care with tetanus shot 0.78 0.03 (− 0.02 to 0.09) 4135 Antenatal care with anti-malarial 0.71 0.01 (− 0.07 to 0.09) 4135 Institutional delivery 0.91 0.03 (− 0.02 to 0.07) 4089 Any postnatal care 0.39 0.03 (− 0.06 to 0.12) 4135 Modern family planning method among women aged 15–49 0.05 0.03 (0.01–0.06) 9585 Growth monitoring in the past 6 months for children under 5 0.03 0.01 (− 0.02 to 0.04) 7247 Children aged 13–24 months with all basic vaccinations 0.51 0 (− 0.12 to 0.11) 1540 Source: Shapira et al. (2023) (Annex 6, D 48) 3. The Project financed both PBF and DFF, and many results framework indicators include PBF and DFF facilities. When evaluating PBF, one would like to capture both the income effect and the incentive effect. Therefore, to evaluate the PDSS project (as opposed to evaluating the PBF model), one would need to compare PDSS-funded facilities with “pure control” facilities. However, such a group does not exist because non-PDSS provinces were generally supported by other donors. Still, for the ICR, the relevant comparison is PDSS vs. no PDSS, as opposed to PBF vs. DFF. 4. In absence of a control group for PDSS, the only option is a before-after comparison in PDSS provinces, triangulated with findings from other studies. Case-control studies in Zambia, Nigeria, and Cameroon that showed that DFF has a substantial impact on RMNCH indicators (Friedman et al. 2016, Khanna et al. 2021, de Walque et al. 2021). Under PDSS, DFF funds to control facilities represented the only Government transfer in many cases, especially in rural areas. In addition, the Project contributed to strengthening the provincial management (technical and financial), M&E systems which also benefitted DFF facilities. Starting in 2020, DFF facilities were required to use 20 percent of their proceeds to purchase medicines and received investment funds for infrastructure. The drawback from the before-after comparison is that one cannot account for other factors that contributed to improvements during the same period. Certainly, other partners also contributed to specific areas. For example, Global Fund, Gavi and the United Nations Children’s Fund (UNICEF) supported vaccination efforts with vaccines, solar power systems, and cold-chain equipment. Other donors supported interventions in certain health facilities, but no other large- scale province-wide initiatives were implemented in the provinces and technical areas covered by the Project. 5. A before-after comparison using the IE household surveys shows a very substantial increase in the quantity of RMNCH services between 2016 and 2021, both in PBF and DFF facilities. On the utilization side, skilled ANC increased from 81 to 93 percent, institutional delivery improved from 81 to 92 percent, and postnatal care within 2 days after birth increased from 9 to 41 percent. The process quality (content) of maternal care also changed markedly. (Figure 7, Panel A) The likelihood of tetanus shots increased from 67 to 80 percent, receipt of antimalarial medicines during ANC increased from 44 to 72 percent, the likelihood of receiving postnatal care within 2 days increased from 9 to 41 percent, and the likelihood of discussing family planning with health providers increased from 45 to 77 percent. (Figure 7, Panel B) Page 60 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) FIGURE 7: EVOLUTION OF MATERNAL CARE INDICATORS IN SIX IE PROVINCES 6. The IE facility surveys also confirm the positive achievement of the Project on the quality checklist (PDO 5). Between the baseline and endline, the availability of basic equipment, vaccines, family planning products, handwashing stations, incinerators, clinical protocols, and infection control procedures improved markedly, both in health centers and hospitals. (Figure 2 and Figure 8) The only areas that did not see much improvement were improved sources of water and the availability of containers for sharps and needles. The lack of improvement in sources of water can be traced to the timing of infrastructure work under the Project, which mostly started after 2020, not long before the IE endline survey in 2021. As reported under the IRI, the availability of 22 essential drugs increased from 45 to 69 percent in health centers and from 70 to 84 percent in hospitals. This is, incidentally, a higher level of achievement than the one reported under IRI 8. The impact evaluation surveys also included direct observations of child curative care, first ANC visit, and family planning consultations. These direct observations show a more modest improvement in the content of child curative care and first ANC visits between the baseline and endline surveys. 7. The results were also triangulated using two rounds of the Demographic and Health Surveys (DHS 2014 and 2023-2024) and the 2017-2018 Service Provision Assessment (SPA) survey. The results are outlined in Annex 1 under each indicator. 8. Socioeconomic disparities in quality of care. The impact evaluation surveys were used for additional research on the quality of health services. Based on household surveys, direct clinical observations, and clinical interviews, Fink et al. 2022 demonstrate that there are substantial socioeconomic differences in the quality of care received by women. They find that most of the overall wealth-quality gradient comes from the spatial correlation between household wealth and quality of care. In other words, poor women tend to live in areas with health Page 61 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) facilities that provide poor quality of service. To close the quality gap, specific investment is needed in poorer areas, such as improved supply chains, recruitment and retention of qualified health workers, and better infrastructure and equipment. FIGURE 8: SURVEY-BASED STRUCTURAL QUALITY INDICATORS (FOR HOSPITALS) Source: Author elaboration based on impact evaluation report (Annex 7, ref. 13) B. EVD RESPONSE 9. The EVD results were triangulated using various results from published literature. For contact tracing, a study in Beni Health Zone, Nord-Kivu, is particularly informative. It found that 87 percent of contacts were successfully traced, with a median delay of 4 days between the contact’s last interaction with the primary case- patient and first interaction by the contact tracing teams. A further comparison with EVD outbreaks in Liberia and Sierra Leone suggests that the Democratic Republic of Congo’s response was more successful at identifying contacts for patients (over 85 percent of patients had at least one contact identified, compared to 27 percent in the Liberia outbreak and 44 percent in the Sierra Leone outbreak) (Polonsky et al. 2021). Page 62 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ANNEX 5. EFFICIENCY ANALYSIS 11. The original Project design included a Cost-Benefit Analysis that estimated cumulative discounted costs of US$141,886,424, cumulative discounted benefits of US$164,695,906, and an internal return rate of 16.09 percent. Given the large expansion in Project financing, extension of the effective Project duration to 8.5 years, and change in Project scope, the ICR carries out an updated cost-benefit analysis. The main tenets of the updated analysis are to (i) update the estimated health benefits from reductions in the burden of disease due to the PBF and DFF interventions (jointly as a package) and estimate the savings in OOP payments by households; (ii) review evidence on the cost-effectiveness of the PBF intervention compared to the DFF intervention and (iii) estimate the impact of the successful containment of the EVD 10 outbreak. A. COST-BENEFIT ANALYSIS OF PBF/DFF INVESTMENTS (AS A PACKAGE) 12. The main parameters of the analysis are presented in Table 15. The updated analysis first estimates the health burden of a specific set of conditions that are amenable to care financed under the Project, such as vaccine- preventable diseases, maternal and neonatal disorders, diarrhea, upper respiratory infections, and malaria. The IHME estimated the burden of each of these conditions for each year in 2016 to 2021. For years after 2021, the analysis assumes that the burden of disease remains stable in absence of Project interventions (Table 16). The Project exclusively focused on interventions that are generally accepted as being highly cost effective (Disease Control Priorities, third edition 2018). TABLE 15: PARAMETERS OF THE ECONOMIC ANALYSIS Parameter Value Source Investment value 714,500,000 Project documents GDP per capita (2021) 577 Databank Population benefitted by the Project (% of the country’s estimated population) 33% Project documents Discount rate future benefits and Project costs 3% assumed Years of benefit 12 assumed GDP growth after 2023 3% assumed 298 HIV/AIDS 4,186,062 IHME (2016-2021) 302 Diarrheal diseases 8,557,144 IHME (2016-2021) 321 Other intestinal infectious diseases 23,336 IHME (2016-2021) 328 Upper respiratory infections 845,703 IHME (2016-2021) 329 Otitis media 255,471 IHME (2016-2021) Disease burden from Disability- 338 Diphtheria 74,120 IHME (2016-2021) adjusted Life Years (DALY) 339 Pertussis 3,417,858 IHME (2016-2021) amenable to Project 340 Tetanus 299,143 IHME (2016-2021) interventions 341 Measles 1,549,969 IHME (2016-2021) (2016-2021) 345 Malaria 33,831,305 IHME (2016-2021) 350 African trypanosomiasis 167,465 IHME (2016-2021) 366 Maternal disorders 4,709,381 IHME (2016-2021) 380 Neonatal disorders 32,225,821 IHME (2016-2021) 386 Nutritional deficiencies 6,281,511 IHME (2016-2021) 402 Acute hepatitis B 51,368 IHME (2016-2021) Total 96,475,657 13. The analysis takes three possible scenarios for the maximum impact of Project on this disease burden in any given year: 3 percent reduction (low scenario), 4 percent reduction (medium scenario), and 5 percent reduction (high scenario). The maximum impact of the Project is assumed to apply to 2020, 2021, and 2022, while years Page 63 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) preceding have an impact that gradually builds to this percentage, and years after have an impact that gradually returns to zero (Table 16) The maximum impact of each scenario is below the decrease in the burden of disease estimated by IHME in 2020 and 2021, except for 2019 which was the height of the EVD 10 outbreak. For example, in 2018, IHME estimates that the burden of disease decreased by 5 percent, and the economic analysis assumes that less than half of that is attributable to the Project’s impact. At the height of Project impact in 2020-2022, the assumed reduction in burden of disease attributable to the Project is approximately two thirds of the reduction estimated by IHME. TABLE 16: PROJECT DISBURSEMENTS AND ASSUMED IMPACT, BY YEAR Assum Assumed % impact on disease ed burden** DALY disease project burden of imple Actual conditions Yearly men- Medium disbursements amenable to reduction GDP per GDP per tation Low scenario excluding Ebola project in disease capita, US$ capita level scenario (max. High scenario Year response interventions burden* current growth by year (max. 2.5%) 3%) (max. 4%) 2015 10,000,000 482 0 0.0% 0.0% 0.0% 2016 25,974,912 6,073,304 456 -5% 10% -0.3% -0.4% -0.5% 2017 46,672,905 5,794,730 -5% 451 -1% 20% -0.6% -0.8% -1.0% 2018 70,759,577 5,464,900 -6% 546 21% 50% -1.5% -2.0% -2.5% 2019 103,697,440 5,345,999 -2% 576 5% 75% -2.3% -3.0% -3.8% 2020 97,028,200 4,907,898 -8% 525 -9% 100% -3.0% -4.0% -5.0% 2021 112,438,903 4,571,721 -7% 577 10% 100% -3.0% -4.0% -5.0% 2022 34,901,451 4,572,156 665 15% 100% -3.0% -4.0% -5.0% 2023 31,761,214 4,572,156 649 -2% 75% -2.3% -3.0% -3.8% 2024 -65,190 4,572,156 669 3% 50% -1.5% -2.0% -2.5% 2025 0 4,572,156 689 3% 25% -0.8% -1.0% -1.3% 2026 0 4,572,156 709 3% 25% -0.8% -1.0% -1.3% 2027 0 4,572,156 731 3% 25% -0.8% -1.0% -1.3% 2028 0 4,572,156 753 3% 10% -0.3% -0.4% -0.5% 2029 0 4,572,156 775 3% 10% -0.3% -0.4% -0.5% *As estimated by IHME for conditions amenable to project interventions ** Multiply the Project implementation level with the maximum Project impact under each scenario *** Cells highlighted in orange/italic are projections based on the latest calculated value. 14. Using these parameters and assumptions, the calculated DALY reduction over 12 years ranges from 815,469 to 1,304,750, while the present value of the stream of DALY reduction ranges from US$394,042,153 to US$630,467,445 (Table 17). In addition to stream of DALY reductions, the economic analysis considers the reduction in OOP payments that was generated by the Project. OOP reductions constitute an economic transfer rather than an economic cost, and therefore they should be subtracted from the costs of the Project. In real terms, OOP fees for child curative care visits decreased by 76 percent in health centers and 58 percent in hospitals, while fees for antenatal care in health centers decreased by 81 percent. Taking into account that the Project financed 20,000,000 visits under PBF, and that the number of visits in DFF facilities was slightly lower (estimated at 18,000,000), the Project generated an estimated US$104 million in OOP fee savings (Table 17), equivalent to US$89 million in present Page 64 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) value. The low scenario estimates an internal rate of return of 17.28 percent from the PBF intervention, which is in line with the original estimate at Project design. TABLE 17: ECONOMIC ANALYSIS RESULTS Low scenario Medium scenario High scenario Peak reduction in DALYs from Project investments 3% 4% 5% DALY reduction (total) 978,562 1,268,172 1,630,937 Present value of stream of DALY reduction (US$) 479,890,930 639,854,574 799,818,217 Present value of OOP savings for households (US$) (88,592,269) (88,592,269) (88,592,269) Present value of Project expenditures excl. EVD response 460,508,234 460,508,234 460,508,234 Benefit-cost ratio 1.29 1.72 2.15 Internal rate of return 17.28% 41.07% 66.60% B. COST-EFFECTIVENESS OF PBF VS. DFF INTERVENTION 15. A yet-unpublished cost-effectiveness study of the PBF model (Annex 6, ref. 54) estimated that the costs of PBF and DFF were US$2.05 and US$1.71 per person per year, respectively. The incremental cost-effectiveness ratio was estimated to be $1187 per Quality-Adjusted Life Year (QALY) (95% confidence interval: -9,794 - 12,071), not accounting for improvements in the quality of care. Taking into account the improvement of quality of care, the ICER of PBF compared to DFF was US$201.0 per QALY. C. EVD RESPONSE 16. The Project contributed financially to the response to the EVD 9, 10 and 11 outbreaks. Among the three outbreaks, ECD10 was, by far, the more widespread and challenging response (Box 1). BOX 1: TIMELINE OF EVD OUTBREAKS IN THE DEMOCRATIC REPUBLIC OF CONGO, 2018-2020 EVD 9: May-July 2018 • Species: Zaire ebolavirus • Reported number of cases: 54 • Reported number of deaths and percentage of fatal cases: 33 (61%) The Government declared the outbreak on May 8, 2018, in the Bikoro region of Équateur Province in the northwestern part of the country, after two cases were confirmed by laboratory testing at the Institut National de Recherche Biomédicale in Kinshasa. On July 24, 2018, WHO declared the end of the ninth outbreak of EVD in the DRC. EVD 10: August 2018-June 2020 • Species: Zaire ebolavirus • Number of cases: 3,481 • Reported number of deaths and percentage of fatal cases: 2,299 (66%) The Government declared the outbreak on August 1, 2018, in the Nord-Kivu province of eastern Democratic Republic of Congo. Cases were also reported in Ituri and South Kivu provinces and in Uganda. The WHO declared the outbreak over on June 25, 2020. EVD 11: June-November 2020 • Species: Zaire ebolavirus Page 65 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) • Reported number of cases: 130 • Reported number of deaths and percentage of fatal cases: 55 (42.3%) The Government declared the outbreak on June 1, 2020, in Mbandaka, Équateur Province of western Democratic Republic of Congo. This outbreak was distinct from the EVD 10 outbreak which was still ongoing when this one began. On November 18, 2020, the MoPHHSP and WHO announced the outbreak was over. Laboratory sequencing suggests that most cases in this outbreak were likely the result of a new spillover event (i.e. a new introduction of the virus into the community from an animal reservoir) followed by person-to-person spread. Sequencing efforts also identified a few cases that appeared to be linked to the prior Équateur Province outbreak in 2018, possibly due to sexual transmission or relapse of a survivor. Source: Adapted from Centers for Disease Control, https://www.cdc.gov/ebola/outbreaks/index.html 17. The costs and benefits of the EVD 10 investments under the Project were affected by (i) unique circumstances that drove up the cost of the intervention; (ii) uncertainty on the spread of the disease in absence of Project financing; (iv) uncertainty as to the overall impact of an even larger epidemic. TABLE 18: BREAKDOWN OF EVD PUBLIC HEALTH RESPONSE SPENDING Interventions Amount (US$ million) Percentage (%) Coordination 144 19 Clinical management of EVD patients 141 18 Surveillance, contact tracing, points of entry, 122 16 vaccination Risk communication and community engagement 105 14 Infection prevention and control/WASH 106 14 Operational support (incl incentive payment) 59 7.7 Safe and dignified burials 40 5.3 Psychosocial support 34 4.4 Laboratories 15 2.0 Source: Zeng et al. (2023) 18. The amount of spending on the EVD 10 response was estimated at US$1.18 billion, of which US$181 million was financed from the Project. The four pillars of the response were (i) public health response (68%), community engagement (19.8%), security (10.1%), and regional preparedness (2%) (Table 18). Zeng et al. (2023) also estimate the unit costs of the response, which were high across the board: US$4435 for contact tracing and surveillance per identified case, US$59 per laboratory test, US$1464 per identified case in treatment center. Especially at the beginning of the epidemic, international staff and staff from other provinces were playing a leading role in the response as members of the coordination teams, rapid response teams, and vaccination teams. This was due to several factors including (i) no experience with EVD in the affected areas, (ii) the use of an experimental vaccine that required adherence to a strict protocol, (iii) the need to build the capacity of the laboratory network, (iv) low capacity and staffing of local health facilities. Treatment of EVD cases in treatment centers was primarily done by international NGOs. This reliance on external staff generated higher costs (due to higher salaries and benefits and deployment costs) as well as friction with local health staff, humanitarian response agents, and communities (Kalenga et al. 2019). As local capacity increased, a larger share of the work was done by local teams. Ring vaccination became more successful when it was transitioned to local vaccinators that were better able to win the trust of communities. Another important factor affecting the cost of the EVD 10 response was the endemic violence Page 66 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) in the affected areas, coupled with large population movements (Figure 9). The violence not only directly increased the cost of the response, but also increased the transmission rate of the epidemic (Wannier et al. 2019, Jombart et al. 2020, Insecurity Insight 2020,). FIGURE 9: CONCENTRATION CHART OF VIOLENT EVENTS, BY PROVINCE, 2018-2024 Source: Autjor’s elaboration using www.acled.org 19. The 2014 EVD epidemic in Sierra Leone, Guinea and Liberia gives a window into the “what if” – what may have been the impact of an even larger outbreak cause by lack of epidemic control. The three countries, which had an estimated combined population of approximately 22 million at the time, experienced over 26,000 cases of EVD. Huber et al. (2018) estimated that the economic and social burden of the outbreak was over US$56 billion. Of this, the most significant component, $ 18 billion, was deaths from other causes. Significant impacts were estimated from loss of health workforce due to the disease, diversion of resources towards EVD response, and closure of health facilities due to lack of staff, which led to additional deaths from maternal and neonatal conditions, HIV/AIDS, tuberculosis, malaria, and vaccine-preventable diseases. 20. This lesson on the significant social cost of foregone “regular” care during an outbreak, was one of the areas where the Project made a significant contribution during EVD 10. The Government responded by using the Project to temporarily institute free health care in Nord-Kivu using the PBF tools that had already been developed and applied in other Project provinces. Given the low baseline of Government funding reaching those facilities, a lack of response may have led to a general flight of health workers towards the more lucrative EVD response. By temporarily introducing PBF as a mechanism for free care in Nord-Kivu, the Project was able to provide a basic level of financing to the front lines of regular care, thereby ensuring its continuity. Wisniewski et al. (2023) show that the free care policy had a positive impact on overall clinic attendance rates for regular care such as malaria and pneumonia, relative to comparison sites. 21. While it is difficult to extrapolate the estimated cost of the West Africa epidemic to the EVD 10 outbreak, there is no doubt that a full-blown outbreak like the 2014 one, without support for regular care, would have had social costs of well over US$100 billion. Page 67 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) Operational efficiency of Project financing for EVD 10 response. 22. The efficiency of spending on the response was suboptimal, as outlined in the report of a June 2019 mission for strengthening financial management. Weaknesses included: • Lack of consolidated human resources database, which led to irregularities in employment and payments to staff, including overlapping full-time appointments. (WHO SRP4 report) • Differing payment schedules for personnel on the ground led to poaching of MoPHHSP staff (WHO SRP4 report) • Weaknesses in logistics management, including excessive leasing of vehicles at high overall cost but lack of ambulances and motorcycles, poor coordination of stocks of inputs between partners, absence of logistics standard operating procedures (SOP), high use of air transport, etc. (WHO SRP4 report) • High levels of corruption and inappropriate transactions (including SEA) in the selection of local staff (multiple reports by the New Humanitarian). Page 68 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ANNEX 6. SUPPORTING INFORMATION AND DOCUMENTS FIGURE 10: CPF STRUCTURE FY22-26 Source: WB Report No. 168084-ZR TABLE 19: DEMOCRATIC REPUBLIC OF CONGO HEALTH PORTFOLIO Project Health System Multisectoral COVID-19 Strategic Regional Disease Strengthening for Nutrition and Preparedness and Surveillance Systems Better Maternal Health Project Response Strengthening Phase IV Child Health (PDSS) (PMNS) (With AF) (REDISSE IV) Internal code P147555 P168756 P173825 and P167817 P176215 Approval date 18 December 2014 28 May 2019 2 April 2020 and 29 1 October 2019 June 2021 Effectiveness 30 May 2016 21 April 2020 23 June 2020 and 7 13 March 2020 date June 2022 Closing date 30 June 2024 4 July 2026 June 30, 2024 31 May 2025 Project amount US$714.50 million US$561 million US$247.2 million US$280 million (US$150 million for the Democratic Republic of Congo) Disbursement (as 100% 61% 100% 83% of Nov. 12, 2024) Project Improve utilization To increase the To prevent, detect (i) to strengthen national Development and quality of utilization of and respond to the and regional cross-sectoral Objective (latest) maternal and child nutrition-specific threat posed by capacity for collaborative health services in and nutrition- COVID-19, and disease surveillance and targeted areas sensitive strengthen national epidemic preparedness in Page 69 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) Project Health System Multisectoral COVID-19 Strategic Regional Disease Strengthening for Nutrition and Preparedness and Surveillance Systems Better Maternal Health Project Response Strengthening Phase IV Child Health (PDSS) (PMNS) (With AF) (REDISSE IV) within the interventions systems for public the Participating Countries; Recipient's Territory targeting children 0- health preparedness. and (ii) in the event of an 23 months of age Eligible Crisis or Emergency, and pregnant and to provide immediate and lactating women in effective response to said the project regions Eligible Crisis or Emergency. and to respond to an eligible crisis or emergency Geographic scope Haut-Kananga, Kwilu, Kassaï, Kassaï Kinshasa, Kongo Équateur, Nord-Kivu, Haut-Lomami, Central, Sud-Kivu Central, Kwango, Kwango, Kassaï Central, Lualaba, Sud- Kwilu, Haut-Katanga, Kassaï Oriental, Tshopo, Ubangi, Tshuapa, Nord-Kivu, Sud Kivu, Tshuapa Mongala, Kwango, Ituri, Maï-Ndombe, Kwilu, Maï-Ndombe, Kassaï Nord-Kivu, Maniema, Kinshasa (after 2020), Equateur (up to 2019) Page 70 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ANNEX 7. LIST OF CONSULTED DOCUMENTS A. WORLD BANK DOCUMENTS 1. Document Type: Project Appraisal Document; Report No.: PAD1088; Document Date: November 25, 2014; Congo - Health System Strengthening for Better Material and Child Health Results (English). Washington, D.C. World Bank Group. http://documents.worldbank.org/curated/en/797381468248430170/Congo-Health- System-Strengthening-for-Better-Material-and-Child-Health-Results 2. Document Type: Project Paper; Report No.: PAD2136; Document Date: March 10, 2017; Congo, Democratic Republic of - Health System Strengthening for Better Maternal and Child Health Results Project: additional financing (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/334761491184862554/Congo-Democratic-Republic-of-Health- System-Strengthening-for-Better-Maternal-and-Child-Health-Results-Project-additional-financing 3. Document Type: Project Paper; Report No.: RES30061; Document Date: October 31, 2018. Samaha,Hadia Nazem. Disclosable Restructuring Paper - Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) - P147555 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/359761541009330561/Disclosable-Restructuring-Paper-Health- System-Strengthening-for-Better-Maternal-and-Child-Health-Results-Project-PDSS-P147555 4. Document Type: Project Paper; Report No.: RES34663; Document Date: December 6, 2018. Samaha,Hadia Nazem. Disclosable Restructuring Paper - Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) - P147555 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/412601544125241730/Disclosable-Restructuring-Paper-Health- System-Strengthening-for-Better-Maternal-and-Child-Health-Results-Project-PDSS-P147555 5. Document Type: Project Paper; Report No.: PAD3263; Document Date: February 13, 2019. Congo, Democratic Republic of - Health System Strengthening for Better Maternal and Child Health Results Project : Third Additional Financing (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/809421551582095952/Congo-Democratic-Republic-of- Health-System-Strengthening-for-Better-Maternal-and-Child-Health-Results-Project-Third-Additional- Financing 6. Document Type: Project Paper; Report No.: RES37424; Document Date: August 19, 2019. Samaha,Hadia Nazem. Disclosable Restructuring Paper - Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) - P147555 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/216121566243464247/Disclosable-Restructuring-Paper- Health-System-Strengthening-for-Better-Maternal-and-Child-Health-Results-Project-PDSS-P147555 7. Document Type: Project Paper; Report No.: RES38501; Document Date: February 26, 2020. Samaha,Hadia Nazem. Disclosable Restructuring Paper - Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) - P147555 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/936581582755213125/Disclosable-Restructuring-Paper- Health-System-Strengthening-for-Better-Maternal-and-Child-Health-Results-Project-PDSS-P147555 8. Document Type: Project Paper; Report No.: PAD3799; Document Date: May 29,2020. Democratic Republic of the Congo - Fourth Additional Financing for the Health System Strengthening for Better Maternal and Child Health Results Project (English). Washington, D.C. : World Bank Group. Page 71 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) http://documents.worldbank.org/curated/en/913191592186507411/Democratic-Republic-of-the-Congo- Fourth-Additional-Financing-for-the-Health-System-Strengthening-for-Better-Maternal-and-Child-Health- Results-Project 9. Document Type: Project Paper; Report No.: RES43110; Document Date: August 27, 2020. Samaha,Hadia Nazem. Disclosable Restructuring Paper - Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) - P147555 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/234651598531769385/Disclosable-Restructuring-Paper-Health- System-Strengthening-for-Better-Maternal-and-Child-Health-Results-Project-PDSS-P147555 10. Document Type: Project Paper; Report No.: RES46486; Document Date: December 2, 2021. El Kadiri El Yamani,Fatima. Disclosable Restructuring Paper - Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) - P147555 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/099640012022126656/Disclosable0Re0ect00PDSS0000P147555 11. Document Type: Project Paper; Report No.: RES56640; Document Date: June 21, 2023. El Kadiri El Yamani,Fatima. Disclosable Restructuring Paper - Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) - P147555 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/099062123122511984/P1475550b651f80d10963b078a6c76a22 c0 12. Document Type: Project Paper; Report No.: RES60330; Document Date: March 28, 2024. El Kadiri El Yamani,Fatima. Disclosable Restructuring Paper - Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) - P147555 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/099032824064067038/P1475551bee1010021a174127adf168f1 5e 13. Document Type: Health, Nutrition and Population Discussion Papers; Report No.: None; Document Date: October 2022. World Bank Group. 2022. Performance-based Financing in the Health Sector of the Democratic Republic of Congo: Impact Evaluation Report. © World Bank, Washington, DC. http://hdl.handle.net/10986/38132 License: CC BY 3.0 IGO. https://hdl.handle.net/10986/38132 14. Document Type: Internal Document; Report No.: None; Document Date: June 2019. Samaha, Hadia and team. Democratic Republic of Congo (DRC) – Health System Strengthening Project – PDSS (P147555). Mid Term Review Issues Paper. 15. Document Type: Internal Document; Report No.: Report No. 168084-ZR; Document Date: February 22, 2022. Congo, Democratic Republic of - Country Partnership Framework for the Period FY22-26 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/214221646062568502/Congo-Democratic-Republic-of-Country- Partnership-Framework-for-the-Period-FY22-26 16. World Bank Guidance on Corporate Results Indicators, OPS5.06-GUID.132, April 17, 2017. 17. World Bank. 2022. The World Bank’s Social Response to Ebola and Beyond in DRC: How a Rights-based Approach Helped Secure Community Engagement and Participation. https://worldbankgroup.sharepoint.com/sites/DNRINT/Shared%20Documents/Forms/AllItems.aspx?id=%2Fsi tes%2FDNRINT%2FShared%20Documents%2FP171186%2Dfcb74ba7%2Da0f6%2D4a83%2Dbace%2Df3a40433 eb1c%2Epdf&parent=%2Fsites%2FDNRINT%2FShared%20Documents 18. Mécanisme de Financement Mondial/Banque Mondiale 2022. Le contrat unique dans le secteur de la santé en République Démocratique du Congo. Etude de cas : Connaissances & Apprentissage. Page 72 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) https://www.globalfinancingfacility.org/sites/default/files/Contrat-Unique-RDC-Etude-de-Cas-Connaissances- Apprentissage_0.pdf B. OTHER REPORTS 19. Ministère de la Santé Publique, Hygiène et Prévoyance Sociale, Unité de Gestion du Programme de Développement du Système de Santé. Rapport d’achèvement du projet ID Projet PDSS: P147555, Juin 2024. 20. Ministère de la Santé Publique, Secrétariat Général. Mise en œuvre de la riposte contre l’Epidémie de la Maladie à Virus Ebola (10ème) dans le Nord Kivu et Ituri, Rapport de l’Appui du PDSS, Aout 2018-Juin 2020. 21. République Démocratique du Congo, Ministère de la Santé Publique, Secretariat Général, PRONANUT. Consultation Prescolaire – Manuel D’Orientation. (National Guidelines for growth monitoring visits of children 0-59 months), https://www.medbox.org/document/drc-consultation-prescolaire-cps-manuel-dorientation 22. (EVD 10-SRP 1) WHO Health Emergencies Programme. National Plan for the response to the ebola virus disease epidemic in North Kivu Provice (SRP1). https://www.who.int/docs/default-source/documents/spr- ebola-2019/srp1-drc-ebola-disease-outbreak-response-plan.pdf?sfvrsn=40799796_4 23. (EVD 10-SRP 2) WHO Health Emergencies Programme. National Plan for the response to the ebola virus disease epidemic in North Kivu Provice (SRP1). https://www.who.int/docs/default-source/documents/srp3- en-drc-ebola.pdf 24. (EVD 10 -SRP3) WHO Health Emergencies Programme. Strategic Response Plan for the Ebola Virus Disease Outbreak in the Provinces of North Kivu and Ituri, Democratic Republic of the Congo, February-July 2019. https://www.who.int/docs/default-source/documents/srp3-en-drc-ebola.pdf 25. (EVD 10-SRP4) WHO Health Emergencies Programme. Strategic Response Plan for the Ebola Virus Disease Outbreak in the Provinces of North Kivu and Ituri, Democratic Republic of the Congo, July-December 2019. https://www.who.int/docs/default-source/documents/drc-srp4-9august2019.pdf?sfvrsn=679e4d26_2 26. WHO Health Emergencies Programme. Ebola Response in North Kivu, South Kivu & Ituri, Democratic Republic of the Congo, Annual Report July 2019 to June 2020. 27. WHO Health Emergencies Programme. Post-Ebola Transition in North Kivu, South Kivu & Ituri, Democratic Republic of the Congo, July to September 2020. 28. WHO Health Emergencies Programme. Ebola Virus Disease, Democratic Republic of the Congo, External Situation Reports, various numbers. 29. World Health Organization. WHO's response to the 2018–2019 Ebola outbreak in North Kivu and Ituri, the Democratic Republic of the Congo, Report to donors for the period August 2018 – June 2019. https://www.who.int/docs/default-source/documents/emergencies/drc-ebola-response-srp-1-3- october2019.pdf 30. World Health Organization. Final Report of the Independent Commission on the review of Sexual abuse and exploitation during the response to the 10th Ebola virus disease epidemic in the provinces of North Kivu and Ituri in the Democratic Republic of the Congo (DRC), 2021. https://www.who.int/publications/m/item/members-of-the-independent-commission https://www.who.int/publications/m/item/final-report-of-the-independent-commission-on-the-review-of- sexual-abuse-and-exploitation-ebola-drc Page 73 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 31. UNICEF Cellule d’Analyses en Sciences Sociales (CASS). Recommandations clés de la Cellule d’Analyse en Sciences Sociales pour les équipes de réponse contre la MVE en Equateur (juin 2020). https://reliefweb.int/report/democratic-republic-congo/recommandations-cl-s-de-la-cellule-d-analyse-en- sciences-sociales-3 32. UNICEF Cellule d’Analyses en Sciences Sociales (CASS). Suivi des recommandations issues des analyses intégrées de la CASS dans le cadre de la réponse à la résurgence d’Ebola (Butembo, RDC 2021). https://www.unicef.org/drcongo/media/5981/file/COD-CASS%20-resurgence-%20utilisation-evidences.pdf 33. Congo Research Group. Ebola in the DRC. The Perverse Effects of a Parallel Health System. https://library.alnap.org/help-library/ebola-in-the-drc-the-perverse-effects-of-a-parallel-health-system C. PUBLISHED AND GREY LITERATURE 34. de Walque D, Robyn PJ, Saidou H, Sorgho G, Steenland M. Looking into the performance-based financing black box: evidence from an impact evaluation in the health sector in Cameroon. Health Policy Plan. 2021 Jul;36(6):835-847. doi: 10.1093/heapol/czab002. 35. de Walque D, Kandpal E. Reviewing the evidence on health financing for effective coverage: do financial incentives work? BMJ Glob Health. 2022;7:e009932. doi: 10.1136/bmjgh-2022-009932. 36. de Walque D, Kandpal E, Wagstaff A, Friedman J, Neelsen S, Piatti-Fünfkirchen M, Sautmann A, Shapira G, Van de Poel E. Improving Effective Coverage in Health: Do Financial Incentives Work?. Washington, DC: World Bank; 2022. http://hdl.handle.net/10986/37326. License: CC BY 3.0 IGO. 37. Fink G, Kandpal E, Shapira G. Inequality in the Quality of Health Services: Wealth, Content of Care, and the Price of Antenatal Consultations in the Democratic Republic of Congo. Econ Dev Cult Change. 2022;70(3):1295-1336. https://www.journals.uchicago.edu/doi/full/10.1086/713941 38. Friedman J, Qamruddin JN, Chansa C, Das AK. Impact evaluation of Zambia’s health results-based financing pilot project. Washington, DC: World Bank Group; 2018. http://documents.worldbank.org/curated/en/798081509456632349/Impact-evaluation-of-Zambia-s-health- results-based-financing-pilot-project 39. Fritsche GB, Soeters R, Meessen B. Performance-Based Financing Toolkit. Washington, DC: World Bank; 2014. http://hdl.handle.net/10986/17194. License: CC BY 3.0 IGO. 40. Huber C, Finelli L, Stevens W. The Economic and Social Burden of the 2014 Ebola Outbreak in West Africa. J Infect Dis. 2018;218(Suppl 5):S698-S704. https://doi.org/10.1093/infdis/jiy213 41. Insecurity Insight. Attacks on Health Care During the 10th Ebola Response in the Democratic Republic of the Congo. Geneva, Switzerland: Insecurity Insight; November 2020. https://insecurityinsight.org/wp- content/uploads/2020/11/Attacks-on-Health-Care-During-the-10th-Ebola-response-in-the-DRC-November- 2020-FINAL.pdf 42. Jamison DT, Gelband H, Horton S, Jha P, Laxminarayan R, Mock CN, Nugent R. Disease Control Priorities, Third Edition: Volume 9. Improving Health and Reducing Poverty. Washington, DC: World Bank; 2017. http://hdl.handle.net/10986/28877. License: CC BY 3.0 IGO. 43. Jombart T, Jarvis CI, Mesfin S, Tabal N, Mossoko M, Mpia LM, Abedi AA, Chene S, Forbin EE, Belizaire MRD, de Radiguès X, Ngombo R, Tutu Y, Finger F, Crowe M, Edmunds WJ, Nsio J, Yam A, Diallo B, Gueye AS, Ahuka- Mundeke S, Yao M, Fall IS. The cost of insecurity: from flare-up to control of a major Ebola virus disease Page 74 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) hotspot during the outbreak in the Democratic Republic of the Congo, 2019. Euro Surveill. 2020 Jan;25(2):1900735. doi: 10.2807/1560-7917.ES.2020.25.2.1900735. PMID: 31964460; PMCID: PMC6976886. 44. Kalenga OI, Moeti M, Sparrow A, Nguyen VK, Lucey D, Ghebreyesus TA. The Ongoing Ebola Epidemic in the Democratic Republic of Congo, 2018–2019. N Engl J Med. 2019;381:373-383. doi: 10.1056/NEJMsr1904253. 45. Kandpal E, Loevinsohn BP, Vermeersch CMJ, Pradhan E, Khanna M, Conlon MK, Zeng W. Impact Evaluation of Nigeria State Health Investment Project. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/589301552969360031/Impact-Evaluation-of-Nigeria-State- Health-Investment-Project 46. Khanna M, Loevinsohn B, Pradhan E, Fadeyibi O, McGee K, Odutolu O, Fritsche GB, Meribole E, Vermeersch CMJ, Kandpal E. Decentralized facility financing versus performance-based payments in primary health care: a large-scale randomized controlled trial in Nigeria. BMC Med. 2021 Sep 21;19(1):224. doi: 10.1186/s12916- 021-02092-4. PMID: 34544415; PMCID: PMC8452448. 47. Neelsen S, de Walque D, Friedman J, Wagstaff A. Financial Incentives to Increase Utilization of Reproductive, Maternal, and Child Health Services in Low- and Middle-Income Countries: A Systematic Review and Meta- Analysis. Policy Research Working Paper No. 9793. Washington, DC: World Bank; 2021. http://hdl.handle.net/10986/36344. License: CC BY 3.0 IGO. 48. Polonsky JA, Böhning D, Keita M, Ahuka-Mundeke S, Nsio-Mbeta J, Abedi AA, Mossoko M, Estill J, Keiser O, Kaiser L, Yoti Z, Sangnawakij P, Lerdsuwansri R, Vilas VJDR. Novel Use of Capture-Recapture Methods to Estimate Completeness of Contact Tracing during an Ebola Outbreak, Democratic Republic of the Congo, 2018-2020. Emerg Infect Dis. 2021 Dec;27(12):3063-3072. doi: 10.3201/eid2712.204958. PMID: 34808076; PMCID: PMC8632194. 49. Shapira G, Clarke-Deelder E, Booto BM, et al. Impacts of performance-based financing on health system performance: evidence from the Democratic Republic of Congo. BMC Med. 2023;21:381. doi: 10.1186/s12916-023-03062-8. 50. Shen GC, Nguyen HT, Das A, Sachingongu N, Chansa C, Qamruddin J, Friedman J. Incentives to change: effects of performance-based financing on health workers in Zambia. Hum Resour Health. 2017 Feb 28;15(1):20. doi: 10.1186/s12960-017-0179-2. PMID: 28245877; PMCID: PMC5331731. 51. Wannier SR, Worden L, Hoff NA, Amezcua E, Selo B, Sinai C, Mossoko M, Njoloko B, Okitolonda-Wemakoy E, Mbala-Kingebeni P, Ahuka-Mundeke S, Muyembe-Tamfum JJ, Richardson ET, Rutherford GW, Jones JH, Lietman TM, Rimoin AW, Porco TC, Kelly JD. Estimating the impact of violent events on transmission in Ebola virus disease outbreak, Democratic Republic of the Congo, 2018-2019. Epidemics. 2019 Sep;28:100353. doi: 10.1016/j.epidem.2019.100353. Epub 2019 Jul 26. PMID: 31378584; PMCID: PMC7363034. 52. Wisniewski J, Worges M, Lusamba-Dikassa PS. Impact of a free care policy on routine health service volumes during a protracted Ebola virus disease outbreak in the Democratic Republic of Congo. Soc Sci Med. 2023 Apr;322:115815. doi: 10.1016/j.socscimed.2023.115815. Epub 2023 Mar 1. PMID: 36889222. 53. Zeng W, Samaha H, Yao M, Yao M, Ahuka-Mundeke S, Nsio-Mbeta J, Mossoko M, Estill J, Keiser O, Kaiser L, Yoti Z, Sangnawakij P, Lerdsuwansri R, Vilas VJDR. The cost of public health interventions to respond to the 10th Ebola outbreak in the Democratic Republic of the Congo. BMJ Glob Health. 2023;8:e012660. doi: 10.1136/bmjgh-2023-012660. 54. Zeng W, Shepard DS, Nguyen H, Chansa C, Das AK, Qamruddin J, Friedman J. Cost-effectiveness of results- based financing, Zambia: a cluster randomized trial. Bull World Health Organ. 2018 Nov 1;96(11):760-771. doi: 10.2471/BLT.17.207100. Epub 2018 Aug 29. PMID: 30455531; PMCID: PMC6239017. Page 75 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 55. Zeng W, Shapira G, Gao T, Li G, Jarawan E, Bouey J, El Kadiri El Yamani F. Cost-effectiveness of the performance-based financing (PBF) program in the Democratic Republic of the Congo (DRC), manuscript. D. SURVEYS AND DATA 56. (DHS 2023-2024) RDC-Institut National de la Statistique, École de Santé Publique de Kinshasa et ICF. 2024. RDC, Enquête Démographique et de Santé 2023–24 : Rapport des indicateurs clés. Kinshasa, RDC et Rockville, Maryland, USA : ICF. https://dhsprogram.com/pubs/pdf/PR156/PR156.pdf 57. (DHS 2013-2014) Ministère du Plan et Suivi de la Mise en œuvre de la Révolution de la Modernité (MPSMRM), Ministère de la Santé Publique (MSP) et ICF International, 2014. Enquête Démographique et de Santé en République Démocratique du Congo 2013-2014. Rockville, Maryland, USA : MPSMRM, MSP et ICF International. https://dhsprogram.com/publications/publication-FR300-DHS-Final-Reports.cfm 58. (SPA 2017-2018) Ecole de Santé Publique de l’Université de Kinshasa (ESPK) [République Démocratique du Congo] et ICF. 2019. République Démocratique du Congo: Evaluation des Prestations des Services de soins de Santé (EPSS RDC) 2017- 2018. Kinshasa, RDC et Rockville, Maryland, USA : ESPK et ICF. https://dhsprogram.com/publications/publication-SPA30-SPA-Final-Reports.cfm 59. (Global Burden of Disease) Global Burden of Disease Collaborative Network. Global Burden of Disease Study 2021 (GBD 2021). Seattle, United States: Institute for Health Metrics and Evaluation (IHME), 2024. https://vizhub.healthdata.org/gbd-results/ 60. (WDI) World Bank, World Development Indicators, https://databank.worldbank.org/source/world- development-indicators, accessed December 4, 2024. Page 76 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) ANNEX 8. BORROWER’S PROJECT COMPLETION REPORT - EXECUTIVE SUMMARY A. BORROWER’S PROJECT COMPLETION REPORT - EXECUTIVE SUMMARY 1. The Health System Development Project (PDSS) is a project of the Government of the Democratic Republic of Congo, co-financed by the Democratic Republic of Congo, the World Bank and other partners. 2. The Project targeted 13 provinces, namely Kwilu, Kwango, Maï-Ndombe, Equateur, Tshuapa, Sud-Ubangi, Mongala, Haut-Lomami, Haut-Katanga, Lualaba, Maniema, Nord-Kivu and Kinshasa through support to more than 3,000 health facilities in 178 health zones (HZ), serving about 36 million people, or nearly 35% of the Congolese population. 3. Its development objective is to improve the use and quality of maternal and child health services in targeted health zones. It supports the implementation of the reframed National Health Development Plan PNDS 2019-2022. It finances the health system within the framework of universal health coverage (UHC) in the Democratic Republic of Congo, according to the strategic procurement of care system or "performance-based financing (PBF)", whose services to be purchased are previously verified and evaluated to determine their quantity and quality. 4. It is structured around four components, namely: (i) Component 1 : Improving the use and quality of health services in health facilities through results-based financing; (ii) Component 2 : Improve governance, support and strengthening of health administration services through results-based financing; (iii) Component 3 : Strengthening the performance of the health system – financing and health policy through impact studies, training (UHC), workshops (UHC, Global Financing Facility), surveys, etc.; (iv) Component 4 : Strengthening epidemiological surveillance (epidemic control). 5. The project focused on MCH, improving health service delivery through the expansion and strengthening of PBF in targeted health zones. In addition to increasing the use and quality of RMNCAH-NUT services, PBF has responded to health system shortcomings by improving the way health sector activities are financed and its policy capacity through action focused on (i) the human resources required to achieve health outcomes (motivation, distribution, etc.), (ii) affordability of health services, (iii) availability of affordable and good quality medicines, (iv) community participation, (v) availability of more data, (vi) improvement of the quality of care through more investment, (vii) and improved governance at all levels of the health system. 6. It is expected that through the interventions carried out (human resources training, purchase of services, quality evaluation through quality grids, vignettes and evaluation of perceived quality, flat-rate pricing, security of medicines through the retention of 20% of subsidies and the development of the public-private partnership approach, the supply of medicines and vaccines, a single contract for the structuring of regulations at all levels of the health pyramid, free childbirth, the mechanism for identifying and caring for the indigent (equity fund), the decentralized investment approach, the support provided in the context of the fight against HAT, support for the various responses to epidemics, the registration of households, the use of management plans and index tools, ...), the PDSS should contribute to the achievement of the Development and Intermediate Objectives of the Project, which are: 1) the percentage of pregnant women who have had the 4th prenatal visit (55% ); 2) the new curative consultations per capita per year (50%); 3) fully vaccinated children (70%); the average quality of care score at the level of health facilities under PBF (60%); the number of eligible people vaccinated during EVD (cumulative # of 120,000); the number of people who received essential health, nutrition and population services (annual # of 7,819,376). Page 77 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) 7. The PDSS should also contribute to the achievement of the targets of the six (6) impact indicators of the PNDS, namely: Target 1 - reduction of maternal mortality; Target 2 - Reduction of infant and child mortality; Target 3 - Control of the epidemic of HIV, tuberculosis, malaria and neglected tropical diseases; Target 4 - Control of non- communicable diseases; Target 5 - Reducing chronic malnutrition from 43% to 33%; and Target 6 - Reducing out- of-pocket payments and catastrophic expenditures. 8. Several interventions and mechanisms have been developed and implemented: a. as part of component 1, such as the training of PBF actors at all levels, which has been able to capacity 10,882 people; the contracting with health care institutions, the maximum number of which was 5,547 health facilities in 2020; the purchase of care services (MPA and CPA), in quality and quality, including for the vulnerable, which have made it possible to take care of people who have received essential health services, 59,678,585; the use of quality checklists, vignettes and the "small dose" "high frequency" method to assess the quality of care and the knowledge and skills of health workers; the conduct of community verification to confirm the existence of patients and their care and to assess their level of satisfaction (voice of the community); the introduction of flat-rate pricing for the purchase of services and free childbirth in order to better control the financial accessibility of health care; the development of the investment unit approach and the decentralization of the management of these investments; the provision of medical equipment and materials to 1,565 health facilities (1,424 PHC centers and 123 hospitals), for the solarization of 613 health facilities and for the transfusion safety at the national center for blood transfusion, 09 provincial centers for blood transfusion and 163 hospital and health center blood transfusion centers. b. component 2, such as the development of single contracts with central directorates, provincial management teams and health zone management teams; contracting with EUP for the function of contracting and purchasing services with health care institutions, support for the management of 20% deductions for the purchase of medicines, for support in the organization of rehabilitation and construction in the provinces, for contracting with local associations for community verifications; the establishment of a system of external cross-checking ensured at the beginning by an external verification agency and from 2022 by the General Health Inspectorate (Inspection Générale de la Santé - IGS); the strengthening of the inspection and control function at the level of the IGS and the IPS; c. component 3, support provided to the Government for certain public health interventions such as the response to the yellow fever epidemic, the top-up of counterpart funds for the purchase of vaccines, the purchase of solar refrigerators for health centers in the PDSS target health zones through a contract with UNICEF, support for the organization of annual reviews; support for the conduct of public finance reforms such as the establishment and strengthening of the DAF, the introduction of "program budgeting", support for the improvement of budget execution, flagship training on health financing, mapping of financing and donors, support for the process of rejuvenating health sector staff; the strengthening of the SNAME and the establishment of the public-private partnership approach; the supply of medicines as inputs, the provision of medicines free of charge to health facilities as part of the strengthening of their working capital; support for pharmaceutical regulatory activities and certification of private drug distributors; the acquisition of family planning inputs and the strengthening of the working capital of the Regional Distribution Centers for Medicines; d. component 4, to strengthen epidemiological surveillance (epidemic control) and which has made it possible to support, through the WHO, multisectoral coordination and logistical support, the detection and isolation of cases and their management, the confirmation of cases at the laboratory, rapid access to quality care for patients, the vaccination of high-risk people in a loop strategy (health professionals and contacts), logistical support for the response, security assessment of response intervention sites; through UNICEF, communication; awareness-raising and community engagement, implementation of a WASH strategy, child protection and psychosocial support, CEP of severe malnutrition, support for operations and coordination; through the Page 78 The World Bank ICR DOCUMENT Health System Strengthening for Better Maternal and Child Health Results Project (PDSS) (P147555) International Organization for Migration (IOM), surveillance activities at points of entry (POE) and checkpoints (POC) and strengthening the capacity of the National Program for Border Hygiene (Programme National d’Hygiène aux Frontières – PNHF) to coordinate the prevention of the spread of the epidemic and finally with the International Federation of the Red Cross the organization of dignified and safe burials; and which has also made it possible to develop other approaches that have strengthened the usual interventions of the response such as the subsidization of alerts of suspected cases of EVD, the financing of free EVD care and emergencies, the supply of drugs for the working capital of health facilities and the development of the 20% approach and finally the support of means of transport. 9. The results of the impact study as well as the information provided by the mission to capitalize on the experiments carried out during the project corroborate these results. 10. The details related to the project's implementation actions, the lessons learned, the challenges encountered during the implementation of the project are specified in the body of [the completion] report and we invite you to go through it in detail because the works posed by the various actors involved in the PDSS, starting with the population, including health providers, health zone management teams, provincial management teams and their partners as well as all central level executives deserve to be defended. Many thanks. At the end of the implementation period of the various interventions mentioned above, the PDSS was able to achieve all the development objectives of the project with a percentage of pregnant women who had the 4th prenatal consultation estimated at 65% out of 55% expected; new curative consultations per inhabitant and per year estimated at 55% compared to the expected 50%; fully vaccinated children assessed at 78% compared to the targeted 70%; an average score of the quality of care at the level of health centers under Performance- Based Financing evaluated at 67% compared to the expected 60%; a cumulative number of eligible people vaccinated during EVD obtained at 390,697 people against a cumulative # of 120,000 and finally a number of people who received essential health, nutrition and population services estimated at 10,040,346 people for 2022, the last year of implementation of PBF against an annual # of 7,819,376. u project with B. BORROWER’S COMMENTS ON THE DRAFT ICR No comments were received. Page 79