Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005034 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-53240 and TF-15872, TF-16618, and TF-A0565) ON A IDA CREDIT IN THE AMOUNT OF SDR13.1 MILLION (US$ 20 MILLION EQUIVALENT) AND A GRANT FROM THE MULTI-DONOR HEALTH RESULTS INNOVATION TRUST FUND IN THE AMOUNT OF US$20 MILLION TO THE REPUBLIC OF SENEGAL FOR A Senegal Health & Nutrition Financing Project {Date of Regional Director’s approval of ICR} Health, Nutrition & Population Global Practice Africa Region CURRENCY EQUIVALENTS Exchange Rate Effective June 30, 2019 Currency Unit = CFAF (XOF) CFAF 576.26 = US$1 US$ 1.39021 = SDR 1 FISCAL YEAR January 1 - December 31 Regional Vice President: Hafez M. H. Ghanem Country Director: Nathan M. Belete Regional Director: Amit Dar Practice Manager: Gaston Sorgho Task Team Leader(s): Maud Juquois ICR Main Contributor: Laure Mercereau ABBREVIATIONS AND ACRONYMS ACMU Universal Health Insurance Agency (Agence pour la Couverture Maladie Universelle) AEC Community Execution Agency (Agences d’Exécution Communautaire) ANC Antenatal Care ANSD National Institute of Statistics (Agence National des Statistiques et de la Démographie) BCC Behavioral Change Communication BSF Family Safety Net cash transfer (Bourse de Sécurité Familiale) CBHI Community Based Health Insurance (Mutuelle) CBO Community Based Organization CCT Conditional Cash Transfer CLM Unit Against Malnutrition (Cellule de Lutte contre la Malnutrition) CMU Universal Health Insurance (Couverture Maladie Universelle) CPS Country Partnership Strategy CSO Civil Society Organization DAGE Fiduciary Directorate in the MSAS (Direction de l’Aménagement et de la Gestion des Equipements) DGPSN Delegation for Social Protection and National Solidarity (Délégation Générale à la Protection Sociale et à la Solidarité Nationale) DES Directorate for Hospitals in the MSAS (Direction des Etablissements de Santé) DHIS2 District Health Information Software 2 (web-based platform for reporting and analyzing health data used by the health facilities and the Ministry). DHS Demographic and Health Survey DIEM (Direction des Infrastructures, des Equipements et de la Maintenance). DPRS Directorate of Planning, Research and Statistics (Direction de la Planification, de la Recherche et des Statistiques) DSME Directorate of Maternal and Child Health (Direction de la Santé de la Mère et de l’Enfant) – formerly: DSRSE DSRSE Directorate of Reproductive Health and Child Survival (Direction de la Santé de la Reproduction et de la Survie de l’Enfant) – Later: DSME ESPS Poverty Survey in Senegal (Enquête de Suivi de la Pauvreté au Sénégal) FM Financial Management GFF Global Financing Facility for Every Woman Every Child GHED Global Health Expenditure Database (WHO) HRBTF Health Results Based Financing Trust Fund (USAID) HRITF Health Results and Innovation Trust Fund IC Investment Case IE Impact Evaluation IPF Investment Project Financing IVA Independent Verification Agency ISR Implementation Status and Results report MDG Millenium Development Goal MEFP Ministry of Economy, Finance and Planning (Ministère de l’Economie, des Finances et du Plan) M&E Monitoring and Evaluation MSAS Ministry of Health and Social Affairs (Ministère de la Santé et des Affaires Sociales) MTR Mid-Term Review NGO Non-Governmental Organization NHA National Health Accounts PAD Project Appraisal Document PER Public Expenditure Review PDO Project Development Objectives PNBSF National Family Safety Net Program (Programme National de Bourses de Sécurité Familiale) PNFBR National Results Based Financing Program (Programme National du Financement Basé sur les Resultats – FBR) PNDSS National Sanitary and Social Development Plan (Plan National de Développement Sanitaire et Social) PRN National Nutrition Strengthening Program (Programme de Renforcement de la Nutrition) PSE Emerging Senegal Plan (Plan Sénégal Emergent) QER Quality Enhancement Review RBF Results-Based Financing RMNCAH Reproductive, Maternal, Newborn, Child and Adolescent Health RF Results Framework SAFI Strategy for mobile midwives (Stratégie des Sages-Femmes itinérantes) SCD Systematic Country Diagnosis SIGICMU Integrated Management Information System for the CMU (Système d’Information de Gestion Intégré de la CMU) SPA Service Provision Assessment SSA Sub-Saharan Africa UHC Universal Health Coverage WHO World Health Organization TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .................................................................................................................... 5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ..................................................................... 12 II. OUTCOME .................................................................................................................... 19 A. RELEVANCE OF PDOs ....................................................................................................................... 19 B. ACHIEVEMENT OF PDOs (EFFICACY)................................................................................................ 20 C. EFFICIENCY........................................................................................................................................ 27 D. JUSTIFICATION OF OVERALL OUTCOME RATING............................................................................ 28 E. OTHER OUTCOMES AND IMPACTS (IF ANY) .................................................................................... 28 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 30 A. KEY FACTORS DURING PREPARATION ............................................................................................ 30 B. KEY FACTORS DURING IMPLEMENTATION ..................................................................................... 31 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 34 A. QUALITY OF MONITORING AND EVALUATION (M&E) ................................................................... 34 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ........................................................... 36 C. BANK PERFORMANCE ...................................................................................................................... 37 D. RISK TO DEVELOPMENT OUTCOME ................................................................................................ 39 V. LESSONS AND RECOMMENDATIONS ............................................................................. 39 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 41 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 56 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 59 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 60 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 63 The World Bank Senegal Health & Nutrition Financing (P129472) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P129472 Senegal Health & Nutrition Financing Country Financing Instrument Senegal Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Government of Senegal Ministry of Health and Prevention Project Development Objective (PDO) Original PDO The objective of the project is to increase utilization and quality of maternal, neonatal and child health and nutritional services, especially among the poorest households in Targeted Areas in the Recipient's territory. Page 1 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 20,000,000 20,000,000 18,423,116 IDA-53240 20,000,000 20,000,000 17,158,326 TF-15872 450,000 194,321 194,321 TF-16618 828,000 784,085 784,085 TF-A0565 Total 41,278,000 40,978,406 36,559,848 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 41,278,000 40,978,406 36,559,847 KEY DATES FIN_TABLE_DAT Approval Effectiveness MTR Review Original Closing Actual Closing A 11-Dec-2013 14-May-2014 01-Jun-2017 29-Jun-2018 30-Jun-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 04-May-2016 4.00 Change in Components and Cost Change in Disbursements Arrangements Change in Financial Management Other Change(s) 10-Apr-2018 15.85 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Page 2 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 28-Mar-2014 Satisfactory Satisfactory 0 02 05-Oct-2014 Satisfactory Satisfactory 1.45 03 15-Apr-2015 Satisfactory Satisfactory 1.85 04 24-Oct-2015 Moderately Satisfactory Moderately Satisfactory 2.41 05 09-May-2016 Moderately Satisfactory Moderately Satisfactory 4.00 06 21-Nov-2016 Moderately Satisfactory Moderately Satisfactory 6.92 Moderately 07 29-Jun-2017 Moderately Unsatisfactory 9.97 Unsatisfactory Moderately 08 10-Jan-2018 Moderately Unsatisfactory 13.64 Unsatisfactory 09 06-Sep-2018 Moderately Satisfactory Moderately Satisfactory 21.06 10 13-Apr-2019 Moderately Satisfactory Moderately Satisfactory 24.94 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 5 Central Government (Central Agencies) 5 Financial Sector 16 Insurance and Pension 16 Page 3 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Health 79 Public Administration - Health 6 Health 73 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 100 Health Systems and Policies 88 Health System Strengthening 71 Reproductive and Maternal Health 17 Nutrition and Food Security 12 Nutrition 6 Food Security 6 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Country Director: Vera Songwe Nathan M. Belete Director: Ritva S. Reinikka Amit Dar Practice Manager: Trina S. Haque Gaston Sorgho Task Team Leader(s): Christophe Lemiere Maud Juquois ICR Contributing Author: Laure Mercereau Page 4 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Country context. Located in the western-most part of Africa’s Sahel region, Senegal is one of the most stable countries in Africa. Its national territory spans 196,712 km2, with 700 km of coast by the Atlantic Ocean, and its population was estimated at 12.8 million in 2011. Almost half (43 percent) of the population lived in urban areas, especially in the greater Dakar region. At appraisal, the country’s political system had further been strengthened by the 2012 presidential election. Although deadly protests preceded the first round of voting, the electoral process was peaceful and consolidated Senegal’s democracy. Macky Sall was elected against incumbent President Abdoulaye Wade and sworn in as President in April 2012. With a GNI per capita estimated at 1,0401, and a population growth averaging 2.5 percent a year, Senegal still faced significant demographic, economic and social challenges. After a decade of moderate growth, the percentage of people living below the national poverty line, at 46.7 percent, had barely changed since 2005. While inequality remained moderate, geographical disparities were pronounced and broadly unchanged, with two-thirds of poor households living in rural areas. While notable progress had been observed on child health, health and nutrition outcomes were still mixed. Among all the African countries that carried out Demographic and Health Surveys (DHS), Senegal had experienced the fastest reduction in child mortality (-8 percent per year since 2005), in large part thanks to the success of the malaria program. Similarly, the national nutrition program had been very effective, and chronic malnutrition had fallen sharply: stunting rates stood at 16 percent of under-five children, almost half the rate in 2000 (30 percent) and the lowest in Africa. However, acute malnutrition (wasting) had remained high at 10 percent since the early nineties. Besides, almost no progress had been made in maternal and neonatal health. In 2011, maternal mortality (MDG 5) still stood at about 400 deaths for 100,000 live births, virtually the same level as in 2005. The proportion of deliveries attended by skilled staff had only slightly increased from 52 to 65 percent. Immunization coverage had not increased substantially, and neonatal mortality, at 36 per thousand births in 2011, was still high, reflecting the persistent weaknesses of the health system. The Senegalese health sector was suffering from several inefficiencies. In 2011, total health spending was estimated at 6 percent of GDP by the Global Health Observatory2, about the average for SSA countries. But that amount was not sufficient to adequately fund the system, especially in rural areas. In the public sector, the allocation of resources was skewed in favor of hospitals, while PHC remained underfunded. Out-of-pocket spending represented 55 percent of current health expenditure in 2013 (GHED). Other major weaknesses included the low density in health centers in rural areas and poor distribution of the work force, and the inadequate performance of health workers (as indicated by indicators such as high absenteeism rates, for instance3). Overall the disappointing performance of the health sector was analyzed as the 1 In 2011. Atlas method, current US$ (source: World Bank). 2 Global Health Expenditure Database (GHED). 3 20 percent in average in 2010 (Service Delivery Indicator survey). Page 5 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) result of weak incentives and accountability mechanisms, rather than of lack of funding or other issues (initial training, etc.). There were few mechanisms for pooling risks in the health sector, which contributed to limiting access to care, especially for the poorest households. Compulsory insurance schemes only covered a minority of the population (workers in the public sector and the formal private sector). The voluntary Community-Based Health Insurance Schemes (CBHIs or “mutuelles”) had similarly limited coverage, as they were not subsidized by the Government. A free maternal care program was set up in 2006, but it seemed to do little to improve accessibility. In 2011, according to the DHS, only 30 percent of women in the lowest wealth quintile were assisted by skilled personnel during delivery (the proportion for the highest quintile, by contrast, was 95.5 percent); the first reason for not seeking care, cited by 51 percent of women (and 68 percent in the lowest quintile) was the lack of money.4 Senegal health priorities in 2012-2013. After the elections, Senegal developed an ambitious plan to reduce poverty and accelerate growth, Emerging Senegal Plan 2014 - 2035 (Plan Sénégal Emergent - PSE). The PSE consisted of three pillars, with the second focused on expanding access to social services and social protection. In the health sector, the government had adopted an ambitious Universal Health Coverage (UHC) strategy combining demand-side and supply-side interventions in health and nutrition. The President launched the CMU (Couverture Maladie Universelle) a national health insurance program in September 2013, in accordance with his campaign promise. The objective was to increase health insurance coverage, especially among the rural and informal groups, by subsidizing voluntary participation in CBHIs (mutuelles). On the supply side, the government was determined to address the long-standing issues in the health sector; the RBF strategy was thus adopted to improve the quality of health service and complement the efforts to improve financial accessibility to health care. A pilot was conducted in two regions (Kaffrine and Kolda) with USAID support from 2012, which showed promising results. Rationale for Bank engagement and higher objectives to which the Project contributed. In that context, the Bank’s engagement aimed at supporting the two national strategies (RBF and CMU) that could be expected to be transformational and support the achievement of the government’s goals in the health sector. It was fully aligne d with the Country Partnership Strategy (CPS) 2013-2017 adopted in January 2013, especially its second pillar, focused on “assisting the Government to improve the allocation and effectiveness of expenditures and the impact on results, particularly in the social sectors”. It was consistent with the World Bank’s Health, Nutrition and Population Strategy (2007) (strategic focus on health, nutrition and population results, contribution to client–country efforts to strengthen health systems, intersectoral approach, collaborative division of labor with global partners). The Bank was in a good position to support the Government of Senegal on RBF, building on the considerable experience gathered on successful RBF programs in other African countries. The engagement could also benefit from the lessons learned on demand-side interventions, both in other countries (health vouchers) and in Senegal (support to the National Nutrition Strengthening Program – Programme de Renforcement de la Nutrition or PRN – and to the National Family Safety Net Program - Programme National de Bourses de Sécurité Familiale or PNBSF, a Conditional Cash Transfer (CCT) program targeting vulnerable families). 4Overall, access to health care services varied significantly according to the economic situation of the household. For instance, the percentage of children fully immunized was 56 percent for children in households belonging to the lowest wealth quintile, compared with 70 percent for children in the highest quintile; utilization of modern contraception services varied between 24.5 percent for the richest women and 4.4 percent only for the poorest (all numbers, DHS 2010-11). Page 6 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Theory of Change (Results Chain) The theory of change presented in Figure 1 illustrates how the Project’s activities and their outputs aimed at achieving the expected outcomes (PDO). Three main series of activities were identified by the Project to achieve the PDO: (1) the transfer of funds to health facilities after the provision of a defined package of health services (RBF scheme) and associated capacity building activities to strengthen the supply of health services; (2) a series of activities to support demand for health and nutrition services and; (3) various capacity building activities of the MSAS and other entities involved in the strategic development of health policies in Senegal. The Project also intended to contribute to longer-term health outcomes for the population, especially for pregnant women and children. Key assumptions under which the operation was designed included the following: (1) the government of Senegal had the capacity to implement RBF intervention; (2) there was strong support for reform, including in the public health sector and in hospitals; (3) demand- and supply-side RBF, together with other interventions, are needed to address the multiple issues hindering the utilization of health and nutrition services, especially by the poorest. Figure 1: Theory of change at appraisal Page 7 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Project Development Objectives (PDOs) The Project Development Objective (PDO) was “to increase utilization and quality of maternal, neonatal and child health care and nutritional services, especially among the poorest households in targeted areas in the Recipient’s territory” (Financing Agreement signed on March 28, 2014). Key Expected Outcomes and Outcome Indicators The three expected outcomes were (1) increased utilization of maternal and child health and nutrition services; (2) increased equity in access to those services; and (3) improved quality of health services. Six indicators were selected to measure the success of the Project; four measured progress in utilization of services, while one measured progress in equity and one improved quality, respectively. Table 1: PDO indicators and associated outcomes (at appraisal) PDO indicators Outcome measured 1. Proportion of poorest people among the total direct beneficiaries (percentage) Increased equity in access to maternal, child health and nutritional services 2. New acceptors of modern contraceptive methods (number) 3. Births (deliveries) attended by skilled health personnel (number) Increased utilization of maternal, child health and nutritional services 4. Pregnant women receiving at least four ANC visits at a health facility (percentage) 5. Severely malnourished detected children who are referred and received at the health center for all necessary visits (percentage) 6. Average score of health quality index (based on a subset of the RBF Checklist). Improvement in quality of maternal, child health and nutritional services The Project targeted six out of the fourteen administrative regions in Senegal: Kaffrine, Kedougou, Kolda, Tambacounda, Sédhiou, Tambacounda, and Ziguinchor, with a total population of about 3.5 million people. The regions were selected among the poorest in the country, with the worst health results (in terms of maternal and child mortality) and lowest rates in service coverage for basic services (ANC, assisted deliveries, immunization rates, etc.). In addition, the PAD defines 15 intermediate outcome indicators: 1. Number of health facilities that have an RBF contract 2. Percentage of the RBF envelop that has been effectively allocated to the health facilities (%) 3. Percentage of facilities with verified results and having received their payments (%) 4. Children Immunized (number) (IDA Core Indicator) 5. Pregnant women receiving antenatal care during a visit to a health provider (number) (IDA Core Indicator) Page 8 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) 6. Direct project beneficiaries (number), of which female (percentage) (IDA Core Indicator) 7. Number of poorest households that are identified within the National Unified Registry (number) 8. People with access to a basic package of health, nutrition, or reproductive health services (number) (IDA Core Indicator) 9. Children aged 0-24 months who attended at least 1 Growth Monitoring and Promotion service during the 2 preceding months (%) 10. Pregnant/lactating women, adolescent girls and/or children under age five reached by basic nutrition services (number) (IDA Core Indicator) 11. Number of beneficiaries of maternal health vouchers 12. Percentage of eligible women who have received a maternal health vouchers (%) 13. Health personnel receiving training (number) (IDA CORE Indicator) 14. Number of hospitals where the cost accounting system has been implemented 15. One continuous DHS report and one continuous Health Service Delivery survey report produced every year. Page 9 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Components At appraisal, the Project was funded by three sources5: (1) an IDA credit (IDA-53240, $20 million equivalent); (2) a grant from a multi-donor trust fund, the HRITF (TF-15872, $20 million); and (3) a grant from the HRBF trust fund, financed by USAID (TF-A0565, $2.3 million). It consisted in three components: Component 1. Strengthening the supply-side of health system through Results-Based Financing for health and nutrition services (estimated cost at appraisal: $22.3 million (HRITF $18 million; IDA $2 million; and USAID HRBFTF $2.3 million); actual at project closing: $14.6 million)6. This component financed the extension of the RBF pilot in six selected regions, and associated capacity strengthening activities for the MSAS. The extension was planned in phases, to allow for an impact evaluation. The main activities supported by this component included: - Financing the RBF grants to health facilities and health district/regional teams in the targeted regions. - Supporting the implementation and supervision of the RBF scheme: - Capacity building activities for the RBF teams at central, regional and district levels and support to the RBF unit to ensure a smooth implementation of the RBF program (PNFBR) - Payment of the Independent Verification Agencies (IVAs) and Community Based Organizations (CBOs), tasked with the verification and counter-verification of the reality of services provided to women and children before the payment of the RBF bonuses. Component 2. Improvement to accessibility of maternal, nutrition and child health services (estimated cost at appraisal: US$14 million (IDA $12 million; HRITF $2 million); actual at project closing: $15.6 million). This component supported several mechanisms whose main purpose was to subsidize demand for healthcare and nutrition services. The main activities, which were partly implemented at the national level, were the following: (1) Support to the Universal Health Insurance Scheme (CMU) - Activities to strengthen the capacities of the national entities involved in the implementation of the CMU (Training about regulatory, financial and strategic issues related to the CMU; study tours, international and regional workshop, etc.); equipment (including for the information systems), consulting services (notably for the design of the Equity Fund, the national entity that was to fund the subsidies to the CBHIs), and support (equipment, training) for the creation of the mutuelles (equipment, training and workshop). - Support to the national targeting system (Unified Registry), that would be used to identify the poorest households eligible for the higher subsidies in the CMU scheme (consultant services and equipment). (2) Support to the National Nutrition Strengthening Program (PRN) The activities funded aimed at supporting demand for nutrition services and to foster behavioral change for better nutrition outcomes. It financed the demand-side activities of the existing PNR in the six targeted regions, implemented locally by the AECs (Community Execution Agency). AECs are NGOs that work with and on behalf of local governments to implement a package of nutrition services and behavioral change communication (BCC) activities at the community level: (i) growth promotion and monitoring for children between 0 and 24 months in communities; (ii) (BCC) activities; and (iii) detection and community care of malnutrition. Page 10 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) (3) The component also funded the maternal health vouchers (demand-side RBF) The scheme was a pilot, implemented in the six regions of the Project for the first time (half of the local collectivities of the six regions). The vouchers incentivized two services: (i) four antenatal care (ANC) visits by poor pregnant women; and (ii) assisted deliveries at the health center level. The component financed the vouchers (cash transfers) to pregnant women, the operational costs of the AECs implementing the program (managing the cash transfer, verification and communication), and the verification of the indicators incentivized by the scheme (by the same IVAs as in the supply-side RBF scheme, as the idea was to test the efficacy of combining demand-side and supply-side incentives to improve results in antenatal and maternal health service coverage). Component 3. Institutional strengthening and Project Implementation (IDA, estimated cost at appraisal: US$6 million; actual at project closing: $9.3 million). This component, entirely funded by the IDA credit, was implemented nationwide; it included the following activities: (1) Capacity building for better regulation of the hospital sector: consulting services, training and IT equipment to: - support (a) the roll out of a Unified Hospital Information System in the main hospitals (to better track production, revenues and costs), (b) the piloting of a new hospital payment system, (c) the development of an improved planning tool for hospital care (regional hospital master plans); - strengthen the capacity of the MSAS’s Directorate in charge of hospital regulation (DES). (2) Capacity building for better M&E of the health system: support to the ANSD for the “Senegal continuous survey”, an annual survey combining a household survey (DHS) and a health facility survey (SPA). The Bank was taking over the financial support provided by USAID. (3) Capacity building activities for better general and financial management within the health system. The main objective of this activity was to implement the recommendations of the joint (Bank-USAID) fiduciary assessment, to strengthen the capacities of the DAGE. 5 In addition, a small trust fund (TF-16618) provided funding ($450,000) for a demand-side RBF pre-pilot in the health district of Gossas in the region of Fatick. The trust fund closed on March 31, 2016; the remaining balance was cancelled, and the financial incentives pregnant women for pregnant women in the district were funded under the same rules agreed upon for the Maternal Health Vouchers (Part B.3 of the Project) after the 2016 restructuring (see paragraph 1 B). 6 The total amount disbursed by the Project as reflected in the data sheet ($36.3 million) is lower than the amounts stated here, based on disbursements on FCFA. The difference ($3.2 million) reflects the weakening of the USD against the Euro (and the FCFA) during the period of implementation. Page 11 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION The Project was restructured twice, in May 2016 and in April 2018. The main changes introduced by those restructurings are summarized below. First restructuring (May 2016). The Bank received a letter (Ref # 9895 MEFP) dated September 15, 2015, requesting the Association to restructure the Project, to allow for better implementation. It also requested the financing of the activities undertaken by the government to fight the Ebola epidemic7. Consequently, the restructuring, formally adopted on May 4, 2016: - Included in Component 3 activities related to institutional strengthening for Ebola preparedness; - Introduced changes in the implementation and FM arrangements (inclusion of central-level entities in the beneficiaries of the RBF bonuses; creation of two sub-accounts; and changes in disbursement arrangements); - Introduced a minor extension of the geographical scope of the Project, with the addition of the health district of Gossas (region of Fatick) as a targeted area for the demand-side RBF (as it was an area of a pre-pilot). Second restructuring (April 2018). The second restructuring was implementing after a Mid-Term Review, conducted in 2017 (in two phases: February and May-June) brought to light persisting issues, and the need to make more significant changes to ensure the success of the Project. A letter (Ref # 3058 MEFP) dated April 2, 2018, requested an extension of the Project and a restructuring of its financing. The objectives were to accelerate the implementation, to speed up disbursements and to redirect the funding towards priority activities, as identified by the GFF Investment Case8. The changes introduced by the restructuring, adopted on April 10, 2018, included the following: - A reallocation of funds between components and the addition of new activities; - The modification of the results framework; - Changes in the disbursement categories; - Change in disbursements estimates; - The modification of the closing date to June 30, 2019. Revised PDOs and Outcome Targets The PDO remained the same. The results framework was changed as a result of the second restructuring; however, the changes affecting the PDO-level targets were relatively minor (see section below). 7 As soon as the World Health Organization (WHO) reported cases of Ebola Virus Disease (EVD) in southeastern Guinea in March 2014, the MSAS developed and implemented and intervention plan. The epidemic ended in Senegal in October 2014. However, the Government requested the support of the Bank to implement prevention and surveillance measures in the 8 regions where the risk of resurgence was highest. Six were already targeted by the Project (all but Kaffrine; the Ebola interventions were also implemented in Dakar and Djourbel). 8 The Global Financing Facility for Every Woman Every Child (GFF) leading to the adoption of a costed Investment Plan (IE) began in Senegal in 2017 and led to discussions with partners in parallel with the implementation of the Project; the IC was adopted technically in March 2018. See Section IV D for a short description of its five priority areas. Page 12 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Revised PDO Indicators As part as the 2018 restructuring, the results framework was modified to (1) reflect the one-year extension, (2) take into account the new activities funded by the Project and the changes in implementation of the government’s CMU strategy since 2013, and (3) adjust indicators to available information. The changes impacting the PDO-level indicators are detailed in Table 2 below. Table 2: PDO indicators and targets (restructured, April 2018) PDO indicator Target for the PDO indicator Revised target Initial definition initial closing date, Revised definition for the new closing (2013) June 2018) (2018) date, June 2019) 1. Proportion of poorest people among 25% Definition changed to a cumulative 2,800,000 the total beneficiaries (percentage). number: Number of poor people reached by the Project 2. New acceptors of modern 67,434 Unchanged (but the primary source Higher target: contraceptive methods (number) used was changed from the RBF portal 107,041 to DHIS2). 3. Births (deliveries) attended by 271,679 Unchanged (primary source changed Target maintained skilled health personnel (number) to DHIS2) 4. Pregnant women receiving at least 67% (baseline: Unchanged (primary source changed Lower target: 35% four ANC visits at a health facility at 50%) to DHIS2) Baseline rectified standard quality (percentage) (0%) 5. Severely malnourished detected 80% (baseline: 0) Definition changed to reflect available 6,745 children who are referred and received data: percentage replaced by a at the health center for all necessary cumulative number visits (percentage) 6. Average score of health quality 65% (baseline:0) Unchanged Higher target: 70% index (based on a subset of the RBF checklist) (percentage) The intermediate outcome indicators were also revised: (1) Three indicators were dropped: “Percentage of the RBF envelop that has been effectively allocated to the health facilities” (# 2 in the list above), “Direct project beneficiaries (number)” (#6), and “Number of poorest households that are identified within the National Unified Registry” (#7), the first because the design of the RBF program was to be changed, thus this indicator was not relevant anymore, the second because it was not on the list of corporate indicators anymore, and the third because the National Unified Registry was not supported by the Project as had been planned initially (see below on the Revised components). (2) Two indicators were added, to reflect the funding of new activities to support the implementation of the Universal Health Insurance Scheme (CMU) at the national level, under component 3 (“Number of people effectively targeted by the communication activities of the CMU”, and “Number of operational SIGICMU9 modules”). 9 Integrated Management Information System for the CMU (Système d’Information de Gestion Intégré de la CMU) Page 13 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) (3) Several changes were made to update the definitions of some indicators (either to align them with new corporate indicators, or to clarify the definitions, in reference to the RBF Manual or to reflect available data). The targets were revised accordingly. The revised Results framework can be found in Annex 1. Revised Components Changes were made to the components through the restructurings. The 2016 restructuring added Ebola-related activities, as mentioned above. In 2018, the restructuring revised the components as follows: (1) New activities were added to support the provision of quality health services: - A new sub-component 1.3 was added to fund activities to improve quality. This sub-component provided funding for interventions of the GFF Investment Case under Priority 4 (“Strengthening the Pillars of RMNACH supply-side with quality human resources, availability of drugs and quality service delivery”). Activities funded included notably: transition to scale of the pilot Strategy for mobile midwifes (SAFI), support for the revision of the Human Resources for Health National Strategy, support for mechanisms for quality control of drugs, procurement of obstetrical and maternal health equipment, RMNACH training of health workers and community health workers, and strengthening the geographical accessibility of essential medicines, etc.; - A sub-component 2.4 was added, relating to adolescent health (also one of the 5 priorities of the GFF Investment Case). Activities under this new sub-component included behavioral change interventions and communication strategies to support demand for reproductive health services targeting adolescents; - New capacity building activities were added under sub-component 3.3 (“Capacity building for better general and financial management within the health system”) to finance major procurement for equipment of the different implementing entities and capacity building interventions for the DSME (Maternal Health Directorate), DPRS (Planning, Research and Statistics Directorate). - A sub-component 3.4 relating to health financing was introduced, to fund studies, training and dissemination activities relating to health financing reforms (on domestic resources mobilization, strategic purchasing mechanisms, the role of public-private partnership, etc.). Those activities reflected the adoption in June 2017 of the Health Financing Strategy, also as part of the GFF process in Senegal. (2) Some activities were removed or modified to adapt to the development of the Government’s universal health insurance policy. - The activities financed under subcomponent 2.1 (support to the implementation of the CMU) were slightly updated. They still supported the implementation of the universal health insurance scheme (communication, technical assistance, comprehensive and integrated information system, training, etc. for the mutuelles) but also supports the ACMU’s effort to strengthen the efficiency and equity of the system (pilots to integrate the policy of free health care for children under 5 within the National Insurance mechanisms, enrollment of the beneficiaries of the BSFs in the CBHIs). - The activity relating to the scale-up of the National Unified Registry through the provision of technical assistance, training and equipment to the MSAS was removed. Soon after the Project became effective, the responsibility to carry out the scale-up of the National Unified Registry was transferred to another administrative entity, the DGPSN (and supported by a Social Protection Project funded by the World Bank). Page 14 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) (3) The funds were reallocated between components, and total funding reduced by $1.5 million. Considering the delays in implementing the RBF approach, the amount allocated to component 1 was reduced by $6.6 million. $5.1 million were reallocated to components 2 and 3. The total amount of funding was reduced from $42.3 million to $40.8 million, as it was not possible to disburse the total amount of the USAID grant before the closing of the HRBFTF (on December 31, 2016). The amount allocated to the strengthening of the hospital regulation system (component 3.1) was decreased to $1.2 M. The activities implemented by the Hospital Directorate (DES) had suffered major delays; while the Hospital Information System feasibility study had been finalized, it was clear that the procurement of IT equipment and training activities would not be carried out before the end of the Project. The overall reallocation of funds between the components and sub-components is summarized below. Table 3 Project Components and estimated cost at approval and after the 2018 restructuring Initial Revised Components allocation allocation (2013) (2018) Component 1: Result-Based Financing for health and nutrition services and capacity 22.3 15.7 building 1.1. Financing of Result Based Financing 14.4 9.2 1.2. Building capacities for RBF extension 7.9 5.5 1.3. Improving quality of health services (drugs, HR, maternal and child health) – (new) 1 Component 2: Improvement of accessibility to maternal, nutrition, and children health 14 15.6 services 2.1 Support to the implementation of the Universal Health Insurance 7 7 scheme 2.2 Strengthening demand for nutrition services and behavior change 5 6.3 2.3 Setting up demand-side RBF (i.e. maternal health vouchers) to improve access to 2 2 health care 2.4 Adolescent health - (new) 0.3 Component 3: Institutional strengthening and project implementation 6 9.5 3.1 Capacity building for better regulation of the hospital sector 3 1.2 3.2 Capacity building for better M&E of the overall health system 2 2 3.3 Capacity building for better general and financial management within the health 1 5.5 system 3.4 Support to health financing reforms - (new) 0.8 TOTAL 42.3 40.8 NB: the difference between the initial allocation and the revised allocation (1.5M) is recorded in this table, but not in the Financing Table p.2 (where the amount of USAID grant from the TF-A0565 trust fund is adjusted to its actual number at closing in 2016, $828,000). The table above includes the amount as estimated at appraisal (2013). In addition, it does not take into account the small trust fund (TF-16618) that provided funding for the demand-side pre-pilot in Gossas (see footnote 5 above). Page 15 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Other Changes The 2016 restructuring introduced the following changes: (1) Changes in disbursement arrangements. The delays in processing the administrative agreement between USAID and the World Bank resulted in the Grant Agreement between the WB and Senegal for this TF being signed on January 28, 2016, almost two years after the other Agreements (signed in March 2014). Given this delay, and the fact that the first tranche of the USAID grant ($828,000 after subtracting Bank management fees) was to be spent by December 30th, 2016, the total amount could not be disbursed under the initial disbursement arrangements (under which the USAID funds were spent jointly with the HRITF /IDA funds). The co-financing approach was thus changed from joint to parallel: HRBF TF funds were reallocated to selected activities and regions not to be financed by the IDA credit or the HRITF grant (RBF interventions in Kolda and Kaffrine). The other two Financing Agreements were also adapted, with the addition of new categories and changes in disbursement percentages. The ceiling amount for the HRITF was increased (from CFAF100,000,000 to CFAF300,000,000) to facilitate disbursements. (2) Changes in Financial Management. Sub-accounts were created for the CLM (Cellule de Lutte contre la Malnutrition) and ANSD (Agence Nationale de la Statistiques et de la Démographie) in order to facilitate and speed up project activities implemented by these entities which had sufficient fiduciary capacities to manage the funds. (3) Other changes. Another change implemented through the restructuring was the revision of implementation arrangements for Results-Based Financing (RBF), with the addition of central-level units directly in charge of implementing RBF as beneficiaries of performance bonuses. Finally, as mentioned above, the health district of Gossas in the region of Fatick was added as a targeted area for the Project in order to continue funding the financial incentives for pregnant women who had been piloted there. The 2018 restructuring introduced additional changes: (1) The modification of the closing date, from June 29, 2018, to June 30, 2020. The disbursement estimates were also revised to reflect the one-year extension. (2) Changes in the expenditure categories, in order to:  Follow the reallocation of funds between interventions as part of the restructuring;  Reintegrate the activities (RBF interventions in Kolda and Kaffrine) financed by the USAID grant until December 2016. The RBF would thus be entirely financed for the six regions by the HRITF grant;  Add a specific expenditure category for the intervention related to subsidies for enrollment of the poorest under five children in CBHIs (CMU small-scaled pilot). (3) Changes in the results framework (as explained above). Finally, the ceiling amount for the HRITF was increased again (From CFAF 300,000 to CFAF 600,000) to remove implementation bottlenecks due to lack of cash flow for the Project. Page 16 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Rationale for Changes and Their Implication on the Original Theory of Change The changes introduced by the first restructuring were meant to allow the Project to support the fight against Ebola, and to address the issues that prevented the Project to be implemented effectively. The successive Aide-Memoires had highlighted the necessity to strengthen the PNFBR and to overcome the delays in recruiting the verification agency, setting up the portal, etc. During the July 2015 mission, it became clear that one of the issues was the lack of motivation of the staff managing the program. It was therefore decided to include them as beneficiaries of bonuses; in addition, it was agreed to create two sub-accounts for the ANSD and the CLM, as the initial arrangements seemed to generate significant delays, hindering the implementation of activities under component 2. Those measures were expected to improve significantly project implementation, allowing the rating of the Project to be raised again (both progress toward Development Objectives and Project Implementation had been downgraded to Moderately Satisfactory after the summer 2015 (ISR #4)). However, delays subsided, and in November 2016, the two MS ratings were maintained. While progress had been made in the implementation of the RBF (with the first payments of bonuses in October 2016), issues persisted, and the World Bank team continued to advocate for major changes to align the program onto international best practices. In November 2016, despite progress, the disbursement rate still stood at 11 percent for the trust funds (against 1 percent in April 2015). Disbursements were picking up for the IDA credit (22 percent, against 8 percent in April 2015) but not as fast as anticipated, revealing the existence of other problems. Those issues, pertaining to general project management and financial management, were discussed during the successive missions and addressed by the government, with some delays, in 2017 (with the hiring of several staff, to strengthen the fiduciary management of the Project in particular) (see Section III. B for more details). The decision to further restructure was made after a Mid-Term Review (MTR), conducted in two phases in early 2017 and June 2017), confirmed the severity of the problems encountered by the Project (justifying a further down-grading of the ratings, to Moderately Unsatisfactory in the June 2017 ISR (#7). In May 2017, disbursement overall was 24.1 percent (34.2 percent IDA but only 17.4 percent for the TFs). The Mid-Term Review identified solutions to the main issues, namely (1) the adoption of a three-month road map to improve the PNFBR; (2) the strengthening of the DAGE by the recruitment of a FM consultant to address delays, while additional staff would be hired; and (3) the reallocation of funds initially meant for the RBF program to activities that had been implemented successfully (demand-side RBF and nutrition activities, CMU, etc.) and/or that could help improve the quality of care and support other priorities highlighted by the GFF Investment Case, which was being developed at the time. Some large investments (in the SIGICMU, the new information system of the ACMU for instance) were also made to support the Government’s CMU strategy. The changes thus aimed at ensuring the success of the Project, while preparing a new operation in health (with IDA- GFF funding).10 It was thus agreed to continue implementing the RBF (with significant reforms and the adoption of a roadmap to accelerate implementation) and to use the funding that remained available for other activities that were expected to improve access and utilization of quality health services. The second restructuring, finally approved in May 201811, enacts those e decisions, and starts a second phase in the implementation of the Project, with disbursements finally picking up in the last year of 10 The Investing in Maternal, Child and Adolescent Health Project (P162042) was approved on September 26, 2019. Funded by a $140 equivalent IDA credit and a $10 M grant from the GFF, it will scale up the successful approaches tested under the Project to improve maternal and child health and nutrition (community nutrition platform, health insurance for the poorest, cash transfer for poor pregnant women, support to ensure the availability of critical maternal and child health inputs) and pilot innovative approaches on adolescent health and quality of care. It focuses on the same six regions as the Project. 11 The delay between the MTR and the restructuring was mostly due to the fact that continued funding to the PNFBR was conditioned by the Bank to changes to the PNFBR (on the modalities of verification, notably, which had been a stumbling block for long); the revised Manual enacting the changes was only finalized and submitted for Non-Objection in February 2018. Page 17 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) implementation. The September 2018 ISR (#9), noting the progress made during 2017 (especially, the hiring of new staff to manage the fiduciary aspects of Project management), and the improvement of disbursement rates (51 percent in August 2018), upgraded the Project’s rating to MS. On balance, the changes introduced in the Project during its implementation constituted a shift in emphasis between the different types of interventions (the classic supply-side RBF receiving less funding, while system strengthening activities and demand-side interventions received more); the PDO remained unchanged. The revised theory of change - in which activities are changed in component 2 and 3, but not the expected outputs and outcomes - is represented below. The main underlying assumption of the revised theory of change was that the new activities, which consisted mostly in capacity strengthening (trainings, procurement of equipment, etc.) would have a similar effect on the PDO (utilization, quality of services, and equity) as the one they replaced (the supply-side RBF especially, which saw its funding significantly reduced). Figure 2: Revised theory of change (after the April 2018 restructuring) Page 18 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating - High The relevance of the PDO is rated High, as it remains in line with the current national priorities in Senegal and with the World Bank’s strategies. The PDO is consistent with the objectives of the Bank’s Country Partnership Framework (CPF) and the national priorities of Senegal. The Senegal CPF (FY19-FY23) currently under finalization focuses on: (i) building up human capital throughout the life cycle, (ii) optimizing social development and (iii) creating an ecosystem for innovation. The PDO, “to increase utilization and quality of maternal, neonatal and child health care and nutritional services, especially among the poorest households in targeted areas” is aligned with the first strategic objective, related to building human capital. Its activities specifically contributed to the achievement of objectives 1.1 (“Improve early years outcomes for children ages 0-5”) and 1.4 (“Empower adolescent girls and women to have more control over their child-bearing and productivity”) of the CPF. Similarly, the PDO is consistent with the strategic goals of Senegal in the health and social sectors, as outlined in the National Sanitary and Social Development Plan (Plan National de Développement Sanitaire et Social (PNDSS) 2019 – 2028) (reinforce governance and financing in the health sector; develop the supply of health and social services; and promote social protection). The PDO is also fully in line with the World Bank strategy. It supports the twin goals of reducing poverty and promoting shared prosperity, and directly contributes to the development of Senegal’s human capital, in the areas where the country still lags behind (persistent regional imbalances in terms of access to basic and reproductive health services, and high fecundity rates).12 It also supports the achievement of the Sustainable Development Goals (SDGs) adopted by the United Nations in 2015, especially Goal 3: “Ensure healthy lives and promote well-being for all at all ages”, and its targets: the reduction of maternal mortality (target 3.1), the reduction of under-five mortality (3.2), the achievement of universal access to sexual and reproductive health-care services (3.7), and Universal Health Coverage (3.8). 12 CPF 2019-2023, p. 17. Page 19 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) B. ACHIEVEMENT OF PDOs (EFFICACY) Although the Project was restructured twice, the PDO wasn’t changed. The scope of the Project remained mostly the same throughout implementation.13 The outcome targets were increased or maintained, except for one, but the change was meant to correct a mistake.14 Thus, as per the ICR guidelines, a split rating is not required, and all indicators are assessed against the outcome targets as formally revised through the April 2018 restructuring. Efficacy rating: Modest By the end of the project, five out of the six PDO-level indicators surpassed their targets, and one was substantially achieved. Eight out of the twelve Intermediate Result Indicators (IO) either surpassed or substantially reached their target; one was partially achieved; and three were not achieved. Table 4: Summary of indicator achievement Rating Outcome Indicators Intermediate Outcome Total Indicators Surpassed (100 percent+) 5 8 13 Substantially achieved (85 percent+) 1 0 1 Partially achieved (65-85 percent) 0 2 2 Not achieved (<65 percent) 0 2 2 Total 6 12 18 Percentage of targets surpassed/substantially achieved 100% 67% 78% It should be noted that the Results framework comprises few indicators measuring progress on quality and equity in access: four out of six outcome indicators (and seven out of twelve Intermediate Indicators) measure the progress in utilization of services. For this ICR, alternative sources of information (surveys or administrative data) were considered, especially for those two aspects. However, they do not always provide the information needed to assess whether the Project reached its goals. No end line survey was conducted for the Impact Evaluation15, so that data to measure the success of the two RBF interventions are missing. The Senegal continuous surveys do not provide much information to assess the Project’s results, as the 2018 DHS survey is not representative at the regional level, and the SPA survey does not comprise clinical observations; the 2019 data are not available, and data from the 2017 surveys were collected at a time when the project had not been fully implemented yet. The assessment of the achievement of the PDOs is organized around each outcome indicated in the PDO statement: (1) Increase equity in access to basic health and nutrition services; (2) Increase utilization of maternal and child health and nutritional services in the six targeted regions; (3) Improve quality of health care in the selected regions. 13 The 2016 restructuring expanded slightly its geographical scope and added the Ebola activities; the 2018 restructuring removed the activity related to the scale-up of the National Unified Registry (under component 2.1) but replaced it with activities with the same objective – to support the implementation of the CMU. 14 Three of the outcome targets were increased: the number of poor people reached by the project, the new acceptors of modern contraceptive methods, and the index of quality of care, and one was maintained (deliveries attended by skilled birth personnel). One target was lowered: the percentage of women having four ANC visits (from 67 percent to 35 percent), but the Page 20 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Assessment of Achievement of Each Objective/Outcome (1) Increase equity of access to maternal, child health and nutrition services. Rating: Substantial One of the objectives of the project was to improve access to health and nutrition services especially for the poorest, who suffer the most from the lack of financial coverage and other barriers. To that effect, the project supported interventions that were implemented in the regions of the Project (and benefited directly poor people, though not exclusively). It also supported the CMU strategy of the Government and other interventions that were implemented at the national level. As shown in the table below, all indicators measuring the progress in equity of access to services reached or exceeded their targets. The PDO-level indicator measures the number of poor people reached by the Project. At completion, the corresponding target was fully met: 4,408,425 poor women and children had benefited from maternal health vouchers, nutrition services, and financial coverage through the free affiliation to a CBHI (157 percent of the target, 2,800,000 people). The two intermediate indicators related to that PDO also reached their targets or exceeded them: 239,840 people were effectively targeted by communication activities of the ACMU (146 percent of the target, or 165,000 people), and the project supported the procurement of five modules of the ACMU’s management information system (the SIGICMU), as planned. The activities measured by those indicators aimed at strengthening institutions at the national level. Some are foundational, for example the financing of the SIGICMU, which will allow for the identification of the beneficiaries, the enrollment, the management of claims, etc. They probably had a limited effect on access to services for the poor and vulnerable in the short term, but they are likely to eventually improve the system’s capacity to provide them with financial protection. Indicators measuring increased equity in access to health and nutrition services Indicators Target Achieved In Percentage of the target PDO level Number of poor people reached by the Project 2,800,000 4,408,425 157% indicators Intermediate Number of people effectively targeted by communication 165,000 239,840 146% outcome activities of the ACMU indicators Number of modules of the SIGCMU operational 5 5 100% change reflected the need to correct a mistake in the baseline, which was lower than stated in the PAD; the indicator related to the “number of severely malnourished children who are referred at the health center for all necessary visits” was changed (from a percentage to a cumulative number) to reflect available data. 15 The Impact Evaluation (P145230, CN Review meeting January 2015) was initially planned and designed to study both supply and demand-side incentives. It was funded by a separate grant from the Health Results Innovation Trust Fund (HRITF). Baseline data collection took place in 2015 (health facilities and households surveys, with also qualitative tools). Quantitative and qualitative reports were produced, with specific analyses on several aspects (iniquities in access to maternal services; (ii) absenteeism of health workers; (iii) determinants of health services utilization and (iv) determinants of discontinuity of care between antenatal care and delivery. However, implementation issues prevented studying the supply-RBF intervention, as it was not implemented in the way that was initially envisaged before 2018 (and then was implemented with considerable delays). After internal discussions, it was therefore decided to give up the Impact Evaluation, as it was unlikely to contribute much to the learning agenda of the HRITF. The funding was reallocated to the evaluation of RBF interventions in other countries. Page 21 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Aside from the indicators of the results framework, other elements point to an impact of the project in terms of equity of access to health and nutrition services. The geographical targeting of the project is one of those elements. Most of the project interventions were implemented in some of the poorest regions of Senegal. As shown on the map below, in all six regions except Zinguinchor, the percentage of households who are poor (defined as those belonging to the lowest wealth quintile) is more than double the national average. Given that focus on disadvantaged regions, there is a strong likelihood that the interventions supported by the Project, compared with the status quo, improved equity of access to health and nutrition services in Senegal. Percentage of the population belonging to the lowest wealth quintile, by region (source: DHS 2017) DHS data suggest that enrollment in the mutuelles (CBHIs) has been progressing, especially for the poor. The number of people declaring that they are covered by a CHBI doubled from 2011 to 2017 (from 1.5 to 3 percent). For the lowest wealth quintile, the proportion rose from 0.3 to 4 percent, a percentage higher than for the richest (1.7 percent in 2017). Although those proportions remain very low, they point to a progress, which can probably be attributed at least in part to the support provided by the Project. 16 Besides, perceived barriers to seek care seem to be decreasing over time, especially for the poorest households. The percentage of women declaring that they faced significant barriers to access health care, and that the lack of money was the main factor, decreased from 51 percent in 2011 to 43 percent in 2018 (DHS). The reduction was larger for the women in the households belonging to the lowest quintile (from 68 to 61 percent) than for the richest (from 30 to 27 percent), which suggests an impact of the different initiatives to limit the cost of health care (including the mutuelles). 16Administrative data from the ACMU’s annual report (Rapport de Performance de l’année 2018, March 2019) are not consistent with survey data. But they also suggest a progress, even if coverage may be overestimated overall. The reports thus states that at the end of 2018, almost half of the Senegalese population (49.6 percent) was covered by a financial protection mechanism (public or private insurance, mutuelle, free health care scheme for children under five, etc.). It is more than twice the percentage estimated at appraisal, when the CMU strategy had just been launched (18 percent). For the mutuelles more specifically, the ACMU’s 2018 report estimates the share of the target population (informal sector) effectively covered at 45.8 percent. Page 22 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Overall, it seems reasonable to assume that the Project contributed, at least in part, to improving access to health and nutrition services for the poorest. Although the progress is modest in the limited timeframe of the Project, financial accessibility should continue to increase in the future, allowing for improved utilization of services by the poor. The achievement of the objective “Increase equity in access to maternal, child health and nutrition services” is therefore rated Substantial. (2) Increase utilization of maternal and child health and nutritional services in the six regions targeted by the project. Rating: Modest One key objective of the Project was to increase the utilization of maternal, neonatal and child health and nutritional services, using a Results-Based Financing approach and supply-side and demand-side interventions. As per the revised RF, the four following PDO indicators were used to assess achievement of the objective: (1) number of new acceptors of modern contraceptive methods; (2) number of births attended by skilled health personnel; (3) percentage of pregnant women receiving at least four ANC visits in a health facility: and (4) percentage of detected severely malnourished children who are referred and received at the health center for all necessary visits. By the closing date, most indicators measuring progress in service coverage surpassed or achieved their targets at least substantially (see table below). Three PDO indicators surpassed their targets: - The total number of new acceptors of modern contraceptive methods was 128,648, against a target of 107,041 (120 percent of the target); - The number of births attended by skilled health personnel was 292,572 (against a target of 271,679, i.e. achievement against the target of 108 percent); - The percentage of pregnant women having at least four antenatal care visits (at standard quality) was 44 percent; 126 percent of the revised target (35 percent). The last PDO indicator substantially achieved its target: the total number of severely malnourished detected children who are referred and received at the health center for all necessary visits was 5,721, or 85 percent of the target (6,745). Out of the seven intermediate results indicators linked to this PDO, four achieved or surpassed their targets: people receiving essential HNP services (3,908,118 people, or 130 percent of the target – 3,016,899 people); pregnant women receiving ANC during a visit to a health provider (528,114, or 113 percent of the target – 468,884 women); children under 24 months who attended at least one growth monitoring and promotion service in the last two months (629,555, or 108 percent of the target – 585,456), and the number of health facilities that have an RBF contract (373, 117 percent of the target). The second of the two indicators measuring the progress of the implementation of the supply-side RBF was not achieved, as no facility had received their payment within the timeframe required by the manual, or three months (the objective was 85 percent of contracted facilities). The two indicators pertaining to the maternal voucher program were partially achieved: the number of vouchers distributed (39,164) represented 78 percent of the target (50,245), while the percentage of eligible women who had received a voucher, at 23 percent, was well below the target (72 percent - achievement rate against the target: 32 percent). Page 23 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Indicators measuring progress in utilization of health and nutrition services Indicators Target Achieved Percentage of the target PDO level Number of new acceptors of modern contraceptive methods 107,041 128,648 120% indicators Number of births attended by skilled health personnel 271,679 292,572 108% Percentage of pregnant women having at least four ANC visits 35% 44% 126% Severely malnourished children who are referred and received 6,745 5,721 85% at a health facility for all necessary visits Intermediate Number of people receiving essential HNP services 3,016,899 3,908,118 130% outcome Number of pregnant women receiving ANC during a visit 468,884 528,114 113% indicators Children ages 0-23 months who attended at least one growth 585,456 629,555 108% monitoring and promotion service in the last two months Number of health facilities that have an RBF contract 318 373 117% Percentage of facilities with verified results and having received 85% 0 0% their payment within three months Number of maternal health vouchers distributed 50,245 39,164 78% Percentage of eligible women who had received a voucher, at 23 72% 23% 32% percent, was well below the target (72 percent - achievement rate against the target: 32 percent). Other sources of information seem to corroborate the results achieved in terms of service coverage. Data from the DHS surveys are difficult to mobilize to assess the project’s results. As of December 2019, data available at the regional level date back to 2017, when implementation of the program had barely begun in four out of six regions. For 2018 however, data are available for four grand regions. The great South region covers five of the six regions covered by the Project (Tambacounda, Kédougou, Kolda, Sédhiou, and Ziguinchor)17, and there is a positive trend on most indicators measuring coverage in basic maternal and child health services in that great South region. The trend is confirmed in the data collected for the Government’s ICR18 (from the Ministry’s routine information system ), which are more recent, and disaggregated at the regional level: they show that the regions covered by the project saw an improvement on many coverage related indicators (completely vaccinated children, assisted deliveries, ANC4, new acceptors of modern family planning methods). On nutrition outcomes, the DHS surveys provide more detailed information, and suggest that greater South outperformed the rest of the country: between 2012 and 2018, the stunting rate decreased from 26 to 17 percent in the corresponding regions, while the rate remained stable at the national level, at 19 percent. The interventions of the Project probably contributed to that result. However, severe difficulties and delays were encountered in the implementation of the supply-side RBF intervention, which casts some doubt on the attribution of the results in service coverage to the Project. The RBF program, coupled with the demand-side activities, was supposed to be the cornerstone of the Project and received a significant amount of funding. Its credibility was probably affected by the delays in its implementation, which would impact its effectiveness (as the underlying theory of change of RFB programs is based on the idea that monetary incentives increase the motivation of health workers and managers, which stimulates them to improve the quantity and quality of services). The difficulties faced by the 17 The other regions are the North (Louga, Saint Louis, and Matam), the West (Thiès and Dakar), and Central regions (Diourbel, Fatick, Kaffrine, and Kaolack). 18 Rapport d’Evaluation et Capitalisation des interventions du PFSN, Ministère de la Santé et de l’Action sociale, November 2019, p. 32-35. Page 24 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) RBF program thus raise some doubt regarding the attribution of the results achieved to the interventions of the Project. As a result, the Efficacy of the Project for the PDO relating to service coverage is rated Modest. (3) Improve the quality of health care in the selected regions. Rating: Modest One of the objectives of the Project was to support interventions promoting the quality of health care. The scaling-up of the supply-side RBF program was initially viewed as an important intervention to support that objective; however, complementary interventions (training of health professionals and community health workers, procurement of equipment, capacity building for hospital policies, support for mechanisms for quality control of drugs, etc.) were also planned. While the support to Senegal’s hospital policy was essentially dropped after the second restructuring, many of those interventions were implemented. The progress toward that objective is measured by the PDO level indicator “average score of health quality index” (based on the RBF checklist). To assess health facilities’ performance, quality checklists were developed; they covered several dimensions of quality: availability of essential equipment, hygiene, financial and drug management, monitoring, maternal care, immunizations, family planning, and infectious diseases management. The quarterly payment to the health facilities depended on the level of achievement of quantitative targets, deflated with the quality score (expressed in percentage). The Project achieved a majority of the quality objectives. The average score of health quality index was 77 percent, higher than the target (70 percent). Out of the three intermediate indicators associated with that target, two achieved or surpassed their targets (One continuous and SPA surveys produced every year – eight in total), and 5,414 health personnel received training (172 percent of the target, 3,144). One was not achieved, the number of hospitals where the cost accounting system has been implemented (the target was 34, but no accounting system was implemented). Indicators measuring improved quality of health and nutrition services Indicators Target Achieved In Percentage of the target PDO level Average score of health quality index (percentage) 70% 77% 109% indicators Intermediate One continuous and SPA surveys produced ever year 8 8 100% outcome Health personnel receiving training (number) 3,144 5,414 172% indicators Number of hospitals where the cost accounting system has 34 0 0 been implemented However, the attribution to the Project of the results observed in terms of improved quality of services is somewhat problematic. In the absence of an end line survey for the Impact Evaluation, there is not much information to corroborate the results and to link them to the Project’s interventions (RBF). In addition, many of the activities contributing to improved quality of care (training of health personnel, procurement of equipment, etc.) were implemented very late (in the last year of implementation) and are therefore unlikely to have strongly contributed to the results observed in the health facilities under RBF contracts. Therefore, despite the fact that most indicators achieved or surpassed their targets, the efficacy of the Project for the PDO “Improve the quality of health care in the selected regions” is rated Modest. Page 25 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Justification of Overall Efficacy Rating As shown in table 4 above, 78% of all indicators measuring progress on the Project’s objectives surpassed or substantially achieved their targets. Only four of the intermediate outcome indicators did not achieve their targets, or only partially achieved them (at 65-85 percent). However, the attribution of some of the results (in terms of service utilization, and quality of care) to the Project’s activities is somewhat uncertain. As a result, the overall efficacy of the Project is rated Modest. Efficacy - outcomes Ratings Outcome 1: To increase equity in access to basic health and nutrition services Substantial Outcome 2: To increase utilization of maternal and child health and nutritional services Modest in the six targeted regions Outcome 3: To improve quality of health care in the selected regions Modest Overall rating Modest Page 26 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) C. EFFICIENCY Assessment of Efficiency and Rating - Substantial A cost-benefit analysis was carried out in 2013 at appraisal. It predicted that the Project would yield significant economic returns to Senegal by improving the utilization and quality of reproductive, maternal, child health and nutrition services. The analysis considered only the RBF (supply-side and demand-side) and the nutrition interventions; it estimated that they would reduce maternal mortality and under-five mortality in the six targeted regions by 29 and 12 percent respectively. In terms of monetary value, the project benefits were estimated at $35.6 million, compared to costs estimated at $18.2 million. The benefit-to-cost ratio was therefore 1.95 (one US dollar invested in the project was expected to generate benefits equivalent to almost two). A cost-benefit analysis was not carried out for this ICR, as the interventions added during implementation (by the second restructuring) consisted mainly in capacity building and institutional strengthening activities, which are complex and difficult to include in simulation models. For the evaluation of the Project’s efficiency, both allocative and implementation efficiency were considered. Details are provided in Annex 4. Allocative efficiency is considered Substantial. Global evidence shows that high impact maternal and child health and nutrition interventions as the ones supported by the Project are extremely cost-effective. For instance, the cost- effectiveness of a standard maternal and child health service package is estimated between $24 and $585 per DALY19 averted, while that of a standard package of prenatal and delivery care ranges from $92 to $148 per DALY averted. 20 Training of community health workers and midwives show a cost of $150 to 1000 per DALY depending on the national context. Nutrition interventions are also highly cost-effective. For instance, the community management of severe and acute malnutrition has a 26 to $39 cost per DALY averted; participatory women’s groups on health outcomes show a cost of $150 to 1000 per DALY averted (Figures from DCP3, volume 2, chapter 17). The efficiency of the interventions financed by the Project (supporting the delivery of basic services in health and nutrition targeted on mother and young children, in mostly poor regions) is thus high, and higher than most interventions in the health and nutrition sectors. The analysis of the costs and benefits of the Project, both direct and indirect, yields the same conclusion. The Project allocative efficiency is thus rated Substantial. Implementation efficiency is rated Modest. The Project encountered long delays in the first years, and implementation and disbursements really began to accelerate very late (in the last 18 months of the Project); it had to be extended by a year. The implementation of the RBF activity experienced the longest delays, but all activities suffered from general Project management and fiduciary issues. As a result, the pace of disbursements began to increase in 2018 only. However, those delays were eventually resorbed in the second phase of implementation. Apart from the activity on hospital reform, most planned activities were implemented; new activities were added through the second restructuring. In July 2018, as already mentioned, the disbursement rate stood at 51% - but at completion, this rate reached 97%. There is a very stark contrast between the first phase of implementation (four years, during which half the funding was disbursed) and the second (one year, during which an almost equal amount was disbursed. While implementation efficiency could be rated Negligible during the first phase of implementation, it would be rated High during the second phase. On average, implementation efficiency is therefore rated Modest. On balance, considering the very high efficiency of the second phase of implementation, and the substantial allocative efficiency due to the type of interventions financed, the Efficiency of the Project is rated Substantial overall. Page 27 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) D. JUSTIFICATION OF OVERALL OUTCOME RATING Following the Bank Guidance on ICR for IPF operations, the overall outcome rating is derived by combining the assessments of Relevance, Efficacy, and Efficiency. The Relevance of the PDO was rated High, and the Efficiency was assessed as Substantial. With the efficacy of the Project rated Modest, the overall Outcome rating is consequently Moderately Unsatisfactory. Dimension Rating Relevance High Efficacy Modest Efficiency Substantial Outcome rating Moderately Unsatisfactory E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender By supporting the provision of quality maternal and child health services, the Project directly targeted women and young children who are the primary beneficiaries of those services. In addition, it aimed at increasing demand for services through interventions that targeted pregnant women and mothers (maternal health voucher program, community-based promotion of maternal and child health and nutrition services). Those approaches have proven effective in expanding access to health and nutrition services for women. Institutional Strengthening Institutional strengthening was one of the main aspects of the Project, and one of its components (component 3), was specifically focused on it. Over the course of the Project, many institutions benefited from capacity building activities. Some interventions were not implemented as planned, especially in the hospital sector, but training was provided to staff of the Ministry, at the central and decentralized levels, in areas ranging from financial management to mother, child and adolescent health. Strong support was provided to the newly created ANACMU, both for the deployment of mutuelles in the regions covered by the Project, and for the development of its information system (SIGCMU). Support to the ANSD for the Senegal continuous survey helps the Agency provide reliable information for decision-making and helped strengthen the institutional capacity to design and implement reforms in the sector. 19 DALYs measure the overall burden of disease, expressed as the number of years lost due to ill-health, disability or early death. 20 Laxminarayan, Ramanan, Anne J. Mills, Joel G. Breman, Anthony R. Measham, George Alleyne, Mariam Claeson, Prabhat Jha et al. 2006. “Advancement of Global Health: Key Messages from the Disease Control Priorities Project.” The Lancet 367 (9517): 1193–1208. Page 28 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) The RBF program, despite the difficulties experienced, contributed to capacity strengthening at the health facility level. The external evaluation of the PNFBR conducted as part of the Mid-Term Review in May 2017 pointed to several successes of the program: a high level of engagement of health workers in the program, increased autonomy and management capacity at the health facility level, investments in equipment (fridges for vaccines, trash can), training of health staff, better communication between teams, etc. Those observations contributed in the decision not to discontinue the program before the end of the Project. Poverty Reduction and Shared Prosperity The Project was directly pro-poor as it aimed at increasing utilization of health and nutrition services especially among the poorest households. It targeted some of the poorest regions in Senegal. Demand-side RBF mechanisms (maternal vouchers) removed financial obstacles to seeking maternal care and benefited poor households; the CMU, by allowing the poorest (beneficiaries of the BSFs) to be affiliated to a CBHI without paying any contribution, also contributed to protect poor people against the financial risk associated with illness, thus reducing poverty and improving shared prosperity. Page 29 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION Overall, the following factors impacted the outcome of the Project:  The government of Senegal was politically committed to achieving UHC, and particularly, to increasing financial protection for the poor and for people working in the informal sector. This impacted positively the outcome of the Project, even as the policy evolved and a new agency was created to implement the CMU strategy.  The collaboration with USAID and its contractor, while fully justified at the design stage, may have hindered the Bank’s effort to implement the RBF program in line with international standards. The collaboration with USAID to provide support to the RBF program meant that the Project could build on experience gathered from an RBF pilot implemented in Senegal. It was part of an effort to overcome the fragmentation of donors’ support in the health sector. However, it also led to the implementation of an RBF model that significantly differed from international standards. As explained by the team during the Quality Enhancement Review (QER) meeting held on August 8, 2013, there were many policy items being discussed with the government at the time of the Project’s preparation. The MSAS wanted to expand the RBF without changing the features of the pilot, and the choice was made to keep certain aspects (the remuneration against targets), focusing on the necessity to change the design of others (the verification mechanisms, notably)21. During implementation, the stakeholders involved in its pilot phase proved somewhat resistant to change, which may have hindered the efforts to implement the program according to international standards, especially on verification22.  The Project was the first Investment Project Financing in the health sector in a decade, which may have led to underestimate the difficulties in engaging with the MSAS. At appraisal, the low implementation capacity of the MSAS was identified as a risk. The capacities of the CLM (managing the nutrition activities and the demand-side FBR) were assessed as very strong, given the unit’s track record in managing Bank projects. The PNFBR’s capacities were assessed as acceptable, and the overall risk linked to the capacity of implementing agency, as “moderate”. In retrospect, that assessment looks too optimistic. In addition, the recurrent tensions within the public health sector, and their potential impact on the availability of data, were not really anticipated.23 21 Due to the limited budget available for verification, the pilot used a peer verification mechanism instead of the standard IVAs used by other RBF schemes, which created a risk of conflict of interest. 22 Adequate independent mechanisms to verify results are a key feature in RBF schemes, as they provide assurance that funds have been used for the intended purpose, mitigating the risk of misuse of funds and ensuring accountability. 23 An “information strike” (i.e. health workers refusing to report any information to health authorities) had already taken place between 2010 and 2012. The issue wasn’t new: the ICR for the previous sector-wide Bank operation (Integrated Health Sector development Project - P002369, Report No: 34252, December 2005) mentions the disruption of the project’s monitoring as the result of a similar strike. Page 30 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) B. KEY FACTORS DURING IMPLEMENTATION Implementation can be divided into two phases: (1) From effectiveness to the revision of the Project after the Mid-Term Review (May 2014 to early 2018); (2) From the second restructuring to the revised closing date (2018 and the first semester of 2019). Many issues were resolved after the second restructuring, leading to improvement in implementation and disbursements (19 percent only in January 2017, when the MTR was initiated, and 96.4 percent at the end of the grace period). The ratings on progress toward PDO and overall implementation progress, which had been lowered to Moderately Unsatisfactory (MU) in June 2017 (with both Financial management and Project management rated MU, and the component 1 – RBF – rated Unsatisfactory), were raised again (to Moderately Satisfactory) in the September 2018 ISR. Factors that positively affected project implementation:  Continued political support for UHC. The Project benefited from a favorable political context, especially for the development of the national insurance program: the CMU strategy remained an important element of the Government’s strategy.  The stability and strong commitment of the World Bank team to the success of the Project. There were only two TTLs during implementation, and both had participated in the preparation of the Project. That low turnover contributed to the success of implementation. In addition, the World Bank team provided a very high level of support for the implementation of the Project and was closely engaged in resolving the challenges that arose. It was a key factor in the results eventually achieved by the Project, especially in its second phase.  Support and leadership at the Ministerial level during the second phase of implementation. Despite the changes in government, the MSAS only had two ministers during implementation; the second one was appointed in September 2017 and confirmed after the 2019 presidential election. His strong commitment to ensuring the success of the Project probably helped accelerate implementation in the last two year.  The commitment of some implementation units. Some interventions benefited from the engagement of the units implementing them, and, in some cases, from their experience in implementing World Bank projects: the DSRSE (for the training activities in the field of maternal and child health), the ANSD (for the “continuous surveys”), the ACMU, the CLM (for the nutrition and demand-side RBF interventions). The results were constrained however by the lack of financing resulting from the issues in overall project coordination and financial management, especially until 2017. Factors that negatively affected project implementation: The main factors that affected negatively implementation and outcomes are presented below; they were most under the control of the implementation entities.  Issues in the implementation of the supply-side Results-Based Financing intervention - The implementation of the RBF program suffered from significant delays. The Project continued financing the RBF approach in the two pilot regions and extended it to four other regions, building on the existing scheme. One of Page 31 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) the key requirements was the recruitment of Independent Verification Agencies (IVAs). The procurement process to recruit the IVAs was delayed, and while the terms of reference had been prepared even before the Project became effective (in July 2013), the contracts were only signed in January 2016, almost two years after effectiveness. The first round of verifications happened in May 2016 and the first payments in October 2016.24 There were also long delays in the development of a designated web-based platform to monitor RBF results (in spite of the multiple reminders from the Bank team), which impacted the Project’s ability to monitor results. Despite the implementation of a roadmap to improve the efficiency of the program in 2018, it continued to suffer from delays until the end of the Project25. - Implementation was negatively affected by the lack of capacities of the PNFBR, and insufficient commitment to improve the design of the program. At the beginning of the Project, the implementation of the RBF program seems to have suffered from the lack of motivation of the staff managing the RBF program at the central level. The first restructuring addressed that issue and added the central-level units directly in charge of implementing RBF as beneficiaries of performance bonuses. However, it did not significantly improve the performance of the PNFBR, which remained an issue until the end. While the World Bank team repeatedly suggested to strengthen national capacities on RBF through the recruitment of an international consultant and training for the staff on RBF, those suggestions were ignored. Finally, the PNFBR was not responsive to feedback from the field, and slow in addressing issues: for instance, the updates on the RBF operational manual, which had been recommended by the external evaluation of the program as part of the MTR process in May 2017, was only finalized in January 2018. - The lack of real ownership of the RBF by the Ministry of Health probably doomed the approach. Despite the stated support for the program, technical issues should have been addressed during implementation; failure to do so point to a more fundamental issue: the program was not perceived by high level officials within MSAS as one of the key strategies to improve the performance of the health sector. There was little if any communication with the other technical directorates of the Ministry (reproductive health, human resources, malaria program, directorate of hospitals, information system…). Lack of real support was also underlined by the fact that there was not allocation to the program from the national budget. During the MTR, given the issues faced by the RBF program, the option of closing component 1 was discussed. However, the decision was made to restructure the Project, to focus on the interventions that had been most successful so far, and to prepare the next operation (which was being considered as part of the GFF process, as explained before). Concerning the RBF program, it was agreed that a three-month roadmap would be implemented to substantially revise the operating manual of the FBR, taking into account the lessons from international experience (as had been advised by the Bank team for two years). The new approach was finally implemented in 2018; however, it was decided to close the RBF program at the end of 2018, no new contract being signed by the health facilities for 2019. As a result, the RBF intervention was implemented for less than two years, with considerable delays. 24 In the two regions already under RBF, Abt Associates was tasked with the verification for 2014 and 2015. 25 The component 1 was still rated MU in the last ISR; by contrast, component 2 (demand-side interventions) was rated Moderately Satisfactory, and component 3 (Institutional strengthening and project implementation), Satisfactory. Page 32 of 67 The World Bank Senegal Health & Nutrition Financing (P129472)  Weaknesses in the fiduciary management of the Project The financial management assessment conducted at the beginning of the Project concluded that the DAGE had adequate capacities to manage the financial and procurement aspects of the project, provided a few issues be addressed through a mitigation plan (involving designating an accountant to perform accounting tasks for this Project, adopting a project Implementation Manual, reinforcing the internal audit department, and recruiting a consultant in procurement to help with the additional workload resulting from the Project, and build capacities). However, financial management proved to be a bottleneck in the implementation of the Project. The first restructuring, in 2016, tried to address the slow pace of disbursements by changing the institutional arrangements and creating two sub-accounts, for the CLM and for ANSD, as the Project coordination and the DAGE could not manage the flow of paperwork associated with the implementation of the components implemented by those entities. Those adjustments did not fully address the FM issues, and the rating for Financial management of the Project was downgraded to MU in June 2017. The workload was too heavy for the financial and procurement aspects of the Project to be managed efficiently by the DAGE; besides, delays were compounded by inefficiencies and delays within the Ministry of Finance. Several measures were taken to address these issues during the second semester of 2017: additional financial management and procurement staff were hired (FM specialist, accountants), a training was conducted by the Bank’s disbursements unit, and the ceiling of the designated account increased. In addition, the World Bank team worked very closely with the Project coordination to ensure smooth disbursements until the end of the project. The July 2018 implementation support mission noted significant progress on financial management, and a decrease in cash flow issues and delays in the implementation of the Project’s activities due to the lack of financial resources. However, despite those improvements, FM remained a bottleneck. The delays at the Ministry of Finance remained substantial, and the change in government after the presidential election (and the subsequent split between the Ministries of Economy and Finance) exacerbated the issue in the last months of implementation.  Weaknesses in project coordination Technical and fiduciary issues experienced in the implementation of the Project could have been addressed more quickly. For instance, the development of the RBF portal took far too much time, despite the multiple reminders from the World Bank team, and the fact that it made it difficult to monitor the project’s results. Similarly, staffing issues in the FM and procurement team should have been addressed sooner. The multiple responsibilities taken on by the Coordinator may have been an obstacle to the timely resolution of implementation issues, penalizing the whole project. Given the issues noted in project management (delays in request processing at the coordination level, recurring coordination and communication problems within the coordination team, a lack of leadership in project management, departure of key staff – project officer and procurement specialist – who were not replaced in a timely manner), and their impact on the delays in the implementation of some activities, Project Management rating was downgraded Moderately Unsatisfactorily by the last ISR (April 2019). Page 33 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design The design of the monitoring and evaluation system was robust overall. The Project’s theory of change behind the design of the M&E was clear, with a straightforward PDO and indicators defined to measure progress on the different interventions financed by the Project. However, and as mentioned above in the Efficacy section, most indicators measured the results of the Project in terms of service coverage, and few indicators measured the impact of the Project on equity in access to those services, and on quality. As in other projects with an important RBF component, indicators were to be collected through the information system used by the program: the quantitative indicators bought through RBF contracts would be verified by third parties (the Independent Verification Agencies) and counter-verified by independent Community Based Organizations. PDO-level indicator 6 (index of quality of care) would also be collected through the RBF program. The development of a web-based platform was planned to improve the reliability and speed of reporting. Only one PDO-level indicator, the one pertaining to equity of access, required specific data collection for the identification of the poorest among the beneficiaries of each intervention (using the information recorded in the Unified Registry). A majority of Intermediate Indicators similarly relied on data produced by the PNFBR. The monitoring system comprised training and capacity building, including support to the Senegal “continuous survey”, and an impact evaluation of the RBF interventions. The design of the result framework was modified in 2018. The restructuring added new indicators, and modified others, adjusting the definitions to available information (some indicators expressed in percentages were converted into indicators in cumulative numbers, for instance). Targets were also modified, either to adjust to the new closing date, or to correct mistakes (in the baseline for the PDO-level indicator “percentage of pregnant women receiving at least four ANC visits” for instance). The outcome indicator related to equity was notably modified, as identifying the poorest in the beneficiaries had proved challenging. The Ministry’s routine information system (DHIS2) became the primary source of data collection, replacing the PNFBR-produced data. Issues remained however, notably the limited number of indicators to measure progress on equity of access and quality. M&E Implementation M&E implementation suffered from a series of issues, resulting in the failure to provide any value for some of the indicators until after the initial closing date. Data on Project results was not available until late: the first values for the PDO-level indicators (except the one on equity) and for 4 Intermediate results indicators were reported in November 2016. There was no monitoring for the other indicators (1 PDO and 11 intermediate indicators) until after the second restructuring (September 2018). Challenges included the following:  Delays in implementing the RBF supply-side intervention are the main reason why results were not monitored for most of the supply-side interventions until mid-2018. The performance contracts with the facilities were signed during the third trimester of 2015, more than a year after the Project became effective; the contracts with the IVAs were signed at the beginning in January 2016. Even after the MTR and the measures adopted to speed up Page 34 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) the implementation of the Project in 2017, significant delays were encountered in the implementation of the revised approach. The signature of the contract modification for the two IVAs was delayed; in April 2019, the results verification for the four quarters of 2018 had not been completed yet. The RBF platform that was supposed to improve transparency and to help disseminate the results of the RBF wasn’t operational until 2018.  The impact evaluation of the RBF program was not completed. The evaluation initially planned to study both supply and demand-side incentives and assess their impact (P145230) would have compared the situation as captured by the baseline survey in 2015, with the situation two years later, in the four regions where the program was extended. By that time however, implementation had barely started, and while RBF programs often encounter implementation delays, in Senegal the implementation diverged too much from what was originally planned for the impact evaluation to be conducted. Therefore, the decision was made not to collect end line data.  There were also issues with data collection for indicators relative to other interventions. For instance, the CLM was not able to directly identify the beneficiaries of maternal health vouchers; the number needed to be inferred from the number of vouchers – which was a source of recurring mistakes when computing and recording the values for the corresponding indicators  The capacity of the Coordination to compile data and communicate on results was extremely low. Documents communicated by the Government to the World Bank were riddled with errors, did not comprise clear definitions of the indicators measured, and mixed values for different years or indicators in the same columns. As a consequence, the RF was very difficult to read, and took an inordinate amount of time to verify and update. The annual reports of the Project were not communicated to the different implementation units.  The RF was not updated at the end of the Project. The last ISR was due to be updated after the last supervision mission in April 2019. However, due to the short timeframe to complete the tasks linked with the Project’s closure (including the finalization of procurement processes, and the close monitoring of disbursements), there wasn’t enough time to do so. As a result, the PDO-level and intermediate indicators at closing show updating dates in December 2017, June 2018 or December 2018. M&E Utilization During the first phase of implementation, there was no data produced to monitor the Project, so that decisions had to be based on alternative sources of information. When the MTR was conducted, the decision to continue implementing the RBF was based on an external evaluation report on the RBF and on the qualitative appreciation of the World Bank team during a field trip to Ziguinchor and Sédhiou (when the results of the RBF proved to be more positive at the facility level than could be assumed from the degree of implementation at the national level). The data produced during the second phase of the Project implementation were not really used. That was in part because of the delays in updating the indicators, but also because towards the end of the Project, the focus of the discussions between the Bank and the MSAS was still on implementation issues and on the bottlenecks to disbursement. Page 35 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Justification of Overall Rating of Quality of M&E While overall the results framework’s design was adequate, with some weaknesses, the implementation and utilization of the data produced suffered from severe shortcomings. The overall quality of M&E is therefore rated Modest. Dimension M&E Design M&E Implementation M&E Utilization Overall M&E Rating Substantial Negligible Negligible Modest B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social Safeguards The Project’s environmental assessment category was C. It did not trigger any Safeguards Policy. The limited environmental risk was managed through the updating of the Ministry’s Medical Waste Management Plan. The Project was expected to have a positive social impact by strengthening the role of communities in the monitoring of health services (as part of the RBF approach, through the surveys conducted by CBOs). Access to health and nutrition services was expected to improve, especially for the poor; grievance mechanisms were developed for the RBF programs. Fiduciary The Project’s ISRs rated Financial Management performance as Moderately Satisfactory (MS) during the first phase of its implementation. The initial fiduciary assessment had concluded that the DAGE could manage the Project (with some mitigation measures). For procurement, it would also rely on the MSAS structures existing to manage the procurement process (Contract Committee, Procurement Unit). The rating of FM was downgraded to Moderately Unsatisfactory in June 2017, despite the changes implemented by the first restructuring in 2016. 2017 was marked by significant fiduciary difficulties. At the end of 2017, disbursements were still low (35 percent and only 23 percent for the TF, six months before the initial closing date) and the needs expressed by the different components (CLM, ACMU, ANSD…) were not adequately met. The issues faced by the DAGE were compounded by delays at the Ministry of Finance regarding the management of the designated account, and it was clear that staffing was an issue. Progress was noted in 2017 and FM again rated MS at the end of the year. In 2018, supervision missions noted the important efforts and significant progress made concerning FM. The Bank’s recommendations were followed satisfactorily, with the transmission of the Annual Work Program and Budget and the recruitment of the internal auditor was finalized. On procurement, there were still shortcomings (delays for the replacement of the procurement specialist; problems with the update and completeness of procurement information in STEP; long delays for approval and payment, notably). Contract management was also an issue, especially in the last months of implementation, when timelines became very short. Procurement was rated Moderately Satisfactory for the duration of the Project. Page 36 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) As detailed in Section III, fiduciary issues affected implementation and outcomes of the Project. C. BANK PERFORMANCE Quality at Entry Analytical work and experience from previous engagement. The project preparation in 2012-2013 was informed by extensive analytical work. The Public Expenditure Review conducted in 201126 underlined the major issues in the public health sector. The technical assistance to hospital reform (P107480) had enabled the Bank to enhance its knowledge of the sector and develop the policy dialogue. In the field of nutrition, the project policy dialogue built on the experience gained from several projects. On RBF, the Bank has helped the Government design and implement the RBF pilot in its second year of implementation. Through the Poverty Reduction Support Credits27, the Bank had provided support to the Government in five priority areas, including health. The series aimed at strengthening health services, with a focus on improving information, financial and human resources management in the sector (including the implementation of performance-based contracts for hospitals), and on increasing access to basic health services. Project preparation and timetable: The Project was prepared over a nine-month period, between the Project Concept Review held in March 2013 and the approval by the Board of Directors in December 2013. It was planned to be implemented over a four-year period which seemed reasonable given the success of the RBF pilot. Besides, similar RBF program had been successfully implemented in a similar timeframe in other West African countries, in more difficult contexts. Overall, international knowledge was considered in Project design, as well as the Senegalese experience in RBF. The design of the Project was addressing the main issues identified in the health sector, was fully consistent with the sector’s strategic goals, and was in line with the overall Bank CPS. As mentioned in Section III, a QER meeting was held in August 2013. The team took into account many comments and suggestions of the panel, limiting the number of the PDO-level indicators, for instance, or taking care to design the Project with sufficient flexibility to tailor its support to the Government’s evolving strategy on UHC. However, the team may have underestimated the risks associated with the Project. The overall risk was rated as Substantial in the PAD, with the governance and fraud risk identified as most critical (for the RBF component, mainly). The PAD also noted that implementing RBF mechanisms and universal health insurance programs were inherently risky as they were transformational. However, the team considered the risks associated with the capacity of the implementing agency moderate, on the basis of the Bank’s experience in implementing similar programs in countries in West Africa, and Senegal’s experience in RBF mechanisms. Technical assistance was viewed as sufficient to alleviate concerns about the lack of experience of the implementing agency (MSAS); other risks were identified (on financial sustainability, stakeholders28) but they were rated Moderate. In retrospect, and as detailed in section III A, other issues might have been considered, which turned out to have a significant impact on the implementation of the Project: 26 Rapport No. AAA82-SN. Développer les outils des institutions de l'Etat pour une gestion plus efficiente de la dépense publique au Sénégal. Revue des Finances Publiques, World Bank, June 2012. 27 P074065, P091051 and P098964. 28 Linked with the creation of the CAPSU, the Independent Fund for Social protection, which was considered to fund the CMU and the PNBSF but was later dropped by the Government. Page 37 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) - The lack of strong internal support and buy-in for the RBF within the Ministry (despite the stated political support); - The history of engagement with the MSAS, which provided evidence of the difficulty to implement reforms in the public health sector; - The inadequate capacity of the PNFBR; - The stakeholder risk associated with the partnership with USAID. Overall, the quality at entry is rated Moderately Satisfactory. Quality of Supervision The World Bank team provided very close support throughout implementation. As explained above in the paragraph III. B, the task leadership changed only once during the life of the Project, in 2017. The incoming TTL had been part of the team from the preparation stage, ensuring continuity in the approach. In addition, the Project benefited from the presence of a team member based in-country, which allowed for close supervision. Ten ISRs have been produced, which is adequate given the duration of the Project. They were detailed on the implementation status and clear on the implementation issues. The Mid-Term review produced quality information to support the decisions concerning the project. Throughout implementation, the Bank team provided strong technical support on Results-Based Financing and other issues, from hospital management at the beginning to the priority issues of the GFF starting in 2017. The World Bank team also led a strong policy dialogue on health financing and the integration of the CMU strategy and other health financing mechanisms and collaborated with other partners involved in the sector. A lot of support was also provided on procurement, financial management and disbursements, the Bank team working very closely to resolve the issues that arose until the end of the grace period. The intensity of the support provided is reflected by the frequent missions (especially from 2017, when they average six a year). The Bank Management also provided key support to the team, providing guidance and conveying strong messages to the Government during the MTR. However, the team struggled to monitor the results of the Project. While this is entirely understandable given the delays in the implementation of the RBF program, the ISRs did not contain information on the progress toward the targets of many results indicators until mid-2018. The focus on solving implementation issues and on ensuring a high level of disbursement resulted in the RF and indicators to be somewhat neglected in the regular monitoring missions (all the more as the updating process was extremely time-consuming, as explained in Section IV.A) even in the second phase of the implementation. In the last ISR (April 2019), some of the indicators’ values had not been updated since December 2017. Most delays and implementation issues during implementation were beyond the Bank’s control; despite moderate shortcomings, the Bank provided very close implementation support to the Government from 2013 to completion. The overall quality of supervision is thus rated Moderately Satisfactory. Justification of Overall Rating of Bank Performance Quality at entry Moderately Satisfactory Quality of supervision Moderately Satisfactory Overall rating of Bank performance Moderately Satisfactory Page 38 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) D. RISK TO DEVELOPMENT OUTCOME Despite the persistently low level of prioritization of health in the national Budget, the risk to development outcome is considered to be Moderate given the continuing commitment of the Government of Senegal to reforms in the health and nutrition sectors, and the renewed support of the Bank through the GFF process and new operations. In the health sector, the ambitious CMU program still benefits from strong political support after Macky Sall’s reelection in 2019. In April 2019, the ACMU became an autonomous Agency under the Ministry of Community Development, and Social and Territorial Equity, institutionalizing a complete split between provision and financing of health care. Similarly, The Government is still committed to fighting malnutrition, and improving health outcomes as part of the second pillar of the PSE 2014 – 2035. The World Bank will continue to support Senegal in its efforts to improve results in terms of maternal, child health and nutrition. The GFF process, which began in 2017, led to the development of an RMNCAH Investment Case (IC) in 2018, which focused on five key priority areas: (1) Provision of a high-impact RMNCAH package; (2) Enhanced financial access to and socio-cultural acceptability of the RMNCAH package through demand side financing; (3) Improved adolescent health through multi-sectoral approaches; (4) Strengthened supply of healthcare services by scaling up high-impact human resources and supply chain interventions to address low RMNCAH effective service coverage; and (5) Strengthening health system governance. The IC targets the five priority regions in the South that were already supported by the Project (Sedhiou, Kolda, Tambacounda, Kedougou and Kaffrine), ensuring they will benefit from coordinated support from the donors to improve quality of care and accessibility of adolescent, maternal and child health services. The Investing in Maternal, Child and Adolescent Health Project (P162042), approved in September 2019, will take over the support to some of the interventions funded by the Project since the second restructuring in 2018 (to ensure the availability of human resources and key inputs in the priority regions, strengthen the capacity of health professionals, especially in maternal, child and adolescent health, support the community nutrition interventions, etc.). The new Early Years for Human Development Project (P161332 – approved in September 2018) also provides support to nutrition interventions in other regions. V. LESSONS AND RECOMMENDATIONS  Ensuring the success of a project requires carefully designed institutional arrangements, and a level of support proportionate to the capacity of the entities involved. Institutional arrangements must be designed in a way that meet the cash needs of each entity if separate activities are being implemented, and strike a balance between the aim of strengthening government institutions (by using government systems to implement the project) and effective implementation (which might require some degree of adaptation – for instance, additional incentives for government staff, even if those incentives are difficult to reconcile with World Bank rules on the employment of civil servants).  National ownership at all levels of government is essential to ensure the success of RBF programs (and more generally, of Bank-supported reforms in the health sector). The MTR of the Health Results Innovation Trust Fund (HRITF)29 notes that government ownership for RBF is a critical enabling factor for effective implementation. That 29 Final report, DFID, Norad and World Bank, April 2018 Page 39 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) lesson is particularly well illustrated by the Senegal example. The RBF program was accepted, but not really owned by the Government; slow progress in implementation can be largely explained by that weak government ownership. It led to the eventual decision to discontinue the program. But the Project was a learning experience for the Bank team as well as for the Government. During the five years of implementation, policy dialogue was strengthened, overcoming the initial difficulties. Preparation for the new operation began well in advance and included long discussions on the Government’s policy objectives and on the modalities of the Bank’s support to achieve them, in accordance with the broader goals articulated in the CPF. That process is more likely to promote agreement between all stakeholders but takes time and commitment on the Bank’s side.  Despite implementation challenges, an IPF operation can lay the foundation for stronger partnerships with governments and more effective modes of support. Focusing on the most successful aspects of a project and building on the experience acquired allows to develop an approach better tailored to the client’s needs. For instance, the implementation of the RBF program in Senegal was challenging. However, tools were developed as part of the RBF approach (quality check-lists, web-based platform) and methods tested (results monitoring, linking results to payments...) that will be used by the MSAS to improve management practices in the sector, and to strengthen the quality of health care. Some of the interventions introduced as part of the 2018 restructuring will also continue to be implemented by the Ministry and can be expected to yield important results (strategy for mobile midwifes, for instance, or the support to supply chain reforms). The support offered to the Government of Senegal through the Project thus went from the implementation of a standard RBF program (mostly perceived as a donor program) to interventions better suited to the needs of the Ministry. The new operation approved in September 2019 will build upon that approach and continue to support the principles of result-based financing (the focus on quality and accountability, especially), even if the program itself was discontinued.  The financing of foundational investments (in information systems, health workers training, etc.) and the support to innovations and pilots (maternal vouchers, the integration of free care mechanisms in the CMU) through IPF operations can foster the policy dialogue on health sector issues and support the development of strong health financing strategies by client countries.  On Universal Health Insurance and health financing more specifically, the Project offered a few lessons on how to work on solving issues faced by national health insurance schemes such as Senegal’s CMU (increased fragmentation resulting from the juxtaposition of insurance and free health care schemes, lack of financial sustainability, limits of voluntary community-based insurance schemes in terms of coverage and capacities, etc.). The Project helped the ACMU test and pilot approaches merging the free health care programs for children and the CBHIs into the universal, state-subsidized health insurance scheme, thus preparing the exploration of policy options to support the strengthening and institutionalization of CBHIs as part of a more sustainable and more equitable health financing system. . Page 40 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increase equity of access to maternal, child health, and nutrition services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of poor people Number 0.00 25.00 2800000.00 4408425.00 reached by the project 01-Feb-2011 30-Jun-2018 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): The original indicator was expressed as a percentage: 25% of poorest people among the total direct beneficiaries. However, the definition had to be changed, as the poorest among the beneficiaries could not be identified accurately. The revised indicator counts the poor among the beneficiaries of maternal vouchers, nutrition services, and the beneficiaries of the PNBSF affiliated to a CBHI. Achievement against target: 157% Page 41 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Objective/Outcome: Increase utilization of maternal, neonatal and child health services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New acceptors of modern Number 0.00 67324.00 107041.00 128648.00 contraceptive method 05-Jun-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Achievement against target: 120% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Births (deliveries) attended Number 0.00 271679.00 271679.00 292572.00 by skilled health personnel (number) 01-Feb-2011 30-Jun-2018 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Achievement against target: 108% Page 42 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of pregnant Percentage 0.00 67.00 35.00 44.00 women having 4 antenatal care visits (at standard 01-Feb-2011 30-Jun-2018 30-Jun-2019 30-Jun-2019 quality) Comments (achievements against targets): The baseline was adjusted in 2018 (the 50% rate recorded in the PAD was incorrect). Achievement against target: 126%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Severly malnourished Number 0.00 80.00 6745.00 5721.00 detected children who are referred and received at the 01-Feb-2011 30-Jun-2018 28-Jun-2019 30-Jun-2019 health center for all necessary visits Comments (achievements against targets): The original target was expressed as a percentage (80%). The new indicator reflect data available at the CLM. Page 43 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Achievement against target: 85% Objective/Outcome: Increase quality of RMNACH services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Index of quality of health care Percentage 0.00 65.00 70.00 76.60 01-Feb-2011 30-Jun-2018 30-Jun-2019 30-Jun-2018 Comments (achievements against targets): Achievement against target: 109%. A.2 Intermediate Results Indicators Page 44 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Component: 1. Result-Based Financing for health and nutrition services and capacity building Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of health facilities Number 0.00 383.00 318.00 373.00 that have an RBF contract 01-Jul-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Achievement against target: 117%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of facilities with Percentage 0.00 100.00 85.00 0.00 verified results and having received their payments 01-Jul-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): The RBF bonuses for the years 2017 and 2018 were paid after June 30, 2019 (the RBF manual states that the payment must be made in a maximum of 72 business days - 3 months). Achievement against target: 0%. Page 45 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion People who have received Number 0.00 1563887.00 3016899.00 3908118.00 essential health, nutrition, and population (HNP) 01-Jul-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 services People who have received Number 0.00 1778108.00 1778108.00 2286355.00 essential health, nutrition, and population (HNP) services - Female (RMS requirement) Number of children Number 0.00 445220.00 445220.00 517039.00 immunized 31-Dec-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 Number of women and Number 0.00 962163.00 2300000.00 3098507.00 children who have received basic nutrition services 31-Dec-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 Number of deliveries Number 0.00 271679.00 271679.00 292572.00 attended by skilled health personnel 31-Dec-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 Page 46 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Comments (achievements against targets): Definition updated to reflect the new corporate indicator definition (sum of intermediate indicators "number of children immunized" and "women and children who have received basic nutrition services", and PDO "number of deliveries attended by skilled health personnel") instead of "women of child- bearing age and under-five children among beneficiaries". Achievement against target: 130% Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Pregnant women receiving Number 0.00 468884.00 468884.00 528114.00 antenatal care during a visit to a health provider 31-Dec-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 (cumulative number) Comments (achievements against targets): Achievement against target: 113%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 47 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Health personnel receiving Number 0.00 3144.00 3144.00 5414.00 training (number) 01-Feb-2011 30-Jun-2018 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Achievement against target: 172%. Component: 2. Improvement of accessibility to maternal, nutrition, and children health services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Children aged 0-23 months Number 0.00 50.00 585456.00 629555.00 who attended at least one growth monitoring and 01-Jul-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 promotion service during the 3 precedent months (cumulative number) Comments (achievements against targets): The original target was expressed as a percentage (50%); the new indicator and target reflect available data. Achievement against target: 108%. Page 48 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of maternal health Number 0.00 48631.00 50245.00 39164.00 vouchers distributed (cumulative number) 01-Jul-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Definition changed in 2018; the indicator initially measured the number of beneficiaries of maternal health incentives, but the data collected was not reliable. The revised indicator and target capture the output of this activity more accurately. Achievement against target: 78%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of eligible Percentage 0.00 90.00 72.00 23.00 women that have received a maternal health financial 01-Jul-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 incentive Page 49 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Comments (achievements against targets): The actual rate on June 30, 2018 was 36%. Achievement against target: 32%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of people effectively Amount(USD) 0.00 165000.00 165000.00 239840.00 targeted by the communication activities of 01-Jul-2013 30-Jun-2019 30-Jun-2019 30-Jun-2019 the CMU (cumulative number) Comments (achievements against targets): Achievement against target: 146%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of operationnal Number 0.00 5.00 5.00 5.00 Page 50 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) SIGICMU modules 22-Dec-2017 30-Jun-2019 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Procurement process completed for all 5 modules financed by the project. Achievement against target: 100%. Component: 3. Institutional strengthening and project implementation Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of hospitals where Number 0.00 34.00 34.00 0.00 the cost accounting system has been implemented 01-Jul-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 Comments (achievements against targets): Achievement against target: 0% Page 51 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion One continuous Number 0.00 8.00 8.00 8.00 Demographic and Health Survey (DHS) report and one 01-Jul-2013 30-Jun-2018 30-Jun-2019 30-Jun-2019 health Service Delivery report are produced every year (cumulative number) Comments (achievements against targets): Achievement against target: 100%. Page 52 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Increase equity of access to maternal, child health, and nutrition services Outcome Indicators 1. Number of poor people reached by the Project 1. Number of people effectively targeted by the communication Intermediate Results Indicators activities of the CMU 2. Number of operational SIGCMU modules 1. Support to the development of the health financing strategy, to the mutuelles Key Outputs by Component 2. Major procurement for the ANACMU (5 modules of the (linked to the achievement of the Objective/Outcome 1) Management information system) 3. Pilot to integrate free health care schemes and the CMU 4. Maternal health vouchers (demand-side RBF) Objective/Outcome 2: Increase utilization of maternal, child and nutrition services in the regions targeted by the project 1. New acceptors of modern contraception methods 2. Births (deliveries) attended by skilled health personnel 3. Percentage of pregnant women having four ANC visits (at standard Outcome Indicators quality) 4. Severely malnourished children who are referred and received at the health center for all necessary visits 1. Number of health facilities that have an RBF contract 2. Percentage of facilities with verified results and having received Intermediate Results Indicators their payments 3. People who have received essential, health, nutrition and population services Page 53 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) 4. Pregnant women who receive ANC during a visit to a health provider 5. Children aged 0-24 months who attended at least one growth monitoring and promotion service during the two preceding months 6. Number of maternal health vouchers 7. Percentage of eligible women who have received a maternal health voucher 1. Demand-side RBF program in six regions 2. Support for the development of the mutuelles (CBHIs) 3. Support to the AECs in the six regions: growth promotion and Key Outputs by Component monitoring activities for children under 24 months in the (linked to the achievement of the Objective/Outcome 2) communities, BCC activities; detection and community care of malnutrition cases. 4. Demand-side RBF program (maternal health vouchers) Objective/Outcome 3 Improvement in quality of health care in the selected regions Outcome Indicators 1. Average score of health quality index 1. Health personnel receiving training 2. Number of hospitals where the cost accounting system has been Intermediate Results Indicators implemented 3. One DHS and one continuous SPA report produced every year 1. RBF extension, including verification, of quality related indicators and capacity building activities 2. Transition to scale of the pilot Strategy for mobile midwifes (SAFI), Key Outputs by Component 3. Support for the revision of the Human Resources for Health (linked to the achievement of the Objective/Outcome 2) National Strategy, 4. Support for mechanisms for quality control of drugs, 5. Procurement of obstetrical and maternal health equipment, Page 54 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) 6. RMNACH training of health workers and community health workers, etc. 7. Feasibility study for a Unified Hospital Information System. Support to the ANSD for the Senegal continuous survey Page 55 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Christophe Lemiere Task Team Leader(s) Sidy Diop Procurement Specialist(s) Fatou Fall Samba Financial Management Specialist Sariette Jene M. C. Jippe Team Member Aminata Ndiaye Bob Team Member Melissa C. Landesz Social Specialist Abdoul Ganyi Bachabi Alidou Social Specialist Supervision/ICR Maud Juquois Task Team Leader(s) Ndeye Fatou Mbacke, Mamata Tiendrebeogo, Mountaga Procurement Specialist(s) Ndiaye Fatou Fall Samba Financial Management Specialist Camille Marie Noelle Le Baron Team Member Mamadou Moustapha Ndoye Social Specialist Moussa Dieng Procurement Team Abdoul Ganyi Bachabi Alidou Social Specialist Ndeye Absa Cisse Procurement Team Karine N. MOUKETO-MIKOLO Team Member Nejma Cheikh Team Member Aminata Ndiaye Bob Procurement Team Sylvie Munchep Ndze Procurement Team Sidy Diop Procurement Team Page 56 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Christophe Lemiere Team Member Ndeye Magatte Fatim Seck Procurement Team Astou Diaw-Ba Procurement Team Maman-Sani Issa Social Specialist Ruma Tavorath Environmental Specialist Seynabou Thiaw Seye Procurement Team Anta Tall Diallo Procurement Team Menno Mulder-Sibanda Team Member Aissatou Chipkaou Procurement Team B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 0 22,338.26 FY13 13.698 358,649.18 FY14 39.311 367,419.84 FY16 .780 2,749.24 Total 53.79 751,156.52 Supervision/ICR FY14 11.023 140,109.92 FY15 23.947 144,830.45 FY16 22.615 493,588.78 FY17 17.187 206,275.03 FY18 36.614 253,617.60 FY19 32.597 181,900.78 FY20 10.137 78,809.30 Page 57 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Total 154.12 1,499,131.86 Page 58 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) ANNEX 3. PROJECT COST BY COMPONENT Amount after Amount at Percentage of 2018 Actual at Project Components Approval restructured Restructuring Closing (US$M) (US$M) amount (%) (US$M) 1. Result-Based Financing for health and nutrition 22.3 15.7 14.6 services and capacity building 2. Improvement of accessibility to maternal, nutrition, 14.0 15.6 15.6 and children health services 3. Institutional strengthening and 6 9.5 9.3 project implementation Total 42.3 40.8 39.5 96.830 30There is an undisbursed amount in the HRITF expressed in USD (2.84 M) due to the evolution of exchange rates between the CFA franc and the USD. The amounts above, and the disbursement ratio, are based on the amounts allocated and executed in FCFA. Page 59 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) ANNEX 4. EFFICIENCY ANALYSIS Global evidence shows that investments in maternal and child health have high social returns. Poor health in childhood erodes human capital, harms physical and cognitive development, and results in reduced educational outcomes and economic productivity. Interventions improving health and nutrition outcomes in mothers and children are therefore highly efficient. There is also evidence that focusing on equity is cost-effective: prioritizing services for the poorest and most marginalized could result in sharper decreases in child mortality and stunting and higher cost-effectiveness than mainstream approaches. The efficiency of the Project was estimated at appraisal in 2013. A cost-benefit analysis was carried out. It predicted that the Project would yield significant economic returns to Senegal by improving the utilization and quality of reproductive, maternal, child health and nutrition services. The analysis considered only the RBF (supply-side and demand-side) and the nutrition interventions and was conducted thanks to the LiST tool (part of the “one Health” tool). It estimated that thanks to the Project interventions, maternal mortality and under-five mortality would decrease in the six targeted regions by 29 and 12 percent respectively. Compared with the status quo, the activities financed by the Project were expected to save 2,343 child lives and 324 mother lives. In terms of monetary value, the project benefits were estimated at $35.6 million, compared to costs estimated at $18.2 million. The benefit-to-cost ratio was therefore 1.95; that is to say, one US dollar invested in the project was expected to generate benefits equivalent to almost $2. Both allocative and implementation efficiency are considered below to assess the Project’s efficiency. Allocative efficiency Allocative efficiency is defined as the use of resources to meet strategic development priorities and improve the welfare of the population. The analysis of the Project points to substantial benefits for limited costs. Project Costs. The cost for this Project was minimal: for component 1 (RBF and capacity building interventions), it is estimated at $0.83 per capita per year (considering the actual cost of component 1 – $1.6 million – and the population covered, around 3.5 million people, for a five-year project). For the demand-side interventions implemented in the six targeted regions under component 2 (demand side RBF, nutrition and behavioral change interventions), it is estimated at $0.89 per capita per year (considering actual cost of component 2 – $15.6 million – for a five-year project). The table below compares those costs with the total health expenditures, and the government expenditures in Senegal; it confirms the modest cost of the Project’s interventions. Table 5: Costs of the Project compared with total and domestic health expenditures in Senegal: Page 60 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) Health expenditure per Share of supply-side Share of demand-side capita per year (current interventions (cost per interventions US$), 2015 capita per year US$0.83) (component 2) (cost per capita per year US$0.89) Current health 31.6 2.6% 2.8% expenditures Domestic general 11.5 7.2% 7.7% government health expenditure Source: GHO and project costs, ICR author calculations. Benefits. The project directly impacted the utilization of maternal and child health and nutrition services and contributed to improve the quality of these services. Using as a standard measure cost per Disability- adjusted Life Year (DALY) averted, the literature has established the cost-effectiveness of those interventions. For example, the cost-effectiveness of a standard maternal and child health service package is estimated to range between $24 and $585 per DALY averted, while that of a standard package of prenatal and delivery care ranges from $92 to $148 per DALY averted.31 Training of community health workers and midwives show a cost of $150 to 1000 per DALY depending on the national context32. Nutrition interventions are also highly cost-effective. For instance, the community management of severe and acute malnutrition has a 26 to $39 cost per DALY averted); participatory women’s groups on health outcomes show a cost of $150 to 1000 per DALY.33 Reductions in stunting (as evidenced in Senegal, see the Section on Efficacy) are estimated to increase overall economic productivity by 4 to 11 percent in Africa and Asia34. Nutrition interventions have shown a high rate of return on investment, between $4 and $35 for every dollar invested.35 In addition, many of the interventions financed by the Project are complex and only have an indirect and longer-term impact on coverage. It is the case for instance for the interventions supporting the implementation of the CMU, including the creation and support to the mutuelles. Those interventions are likely to expand financial protection and therefore utilization of a maternal and child health services. Similarly, the training of health workers is likely to have longer term benefits on the quality and therefore the effectiveness of the care provided. However, the efficiency of those interventions cannot easily be measured. The same is true more generally for all health system strengthening activities, which represent 31 Laxminarayan, Ramanan, Anne J. Mills, Joel G. Breman, Anthony R. Measham, George Alleyne, Mariam Claeson, Prabhat Jha et al. 2006. “Advancement of Global Health: Key Messages from the Disease Control Priorities Project.” The Lancet 367 (9517): 1193–1208. 32 Susan Horton and Carol Levin, “Cost-Effectiveness of Interventions for Reproductive, Maternal, Newborn, and Child Health”, in: Disease Control Priorities (third edition): Volume 2, Reproductive, Maternal, Newborn, and Child Health, edited by R. Black, M. Temmerman, R. Laxminarayan, N. Walker. Washington, DC: World Bank. (Chapter 17). 33 Ibid. 34 Horton, S., and R. Steckel. 2013. “Malnutrition: Global Economic Losses Attributable to Malnutrition 1900–2000 and Projections to 2050.” In The Economics of Human Challenges, edited by B. Lomborg, 247–72. Cambridge, U.K.: Cambridge University Press. 35 Shekar, M, Kakietek, J, Dayton Eberwein, J, Walters, D. An investment framework for nutrition: reaching the global targets for stunting, anemia, breastfeeding, and wasting: The World Bank; 2017. Page 61 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) a large share of the funds supporting the Project. While the efficiency of those interventions cannot be precisely measured, they should be considered when assessing the overall efficiency of the Project, as they are likely to be important in the long term. Overall, the allocative efficiency is rated Substantial. Implementation efficiency The following key issues negatively affected efficiency:  Delays in implementation and disbursements. The implementation of the RBF activity (component 1) was significantly delayed until the revision of the approach following the Mid-Term review. The Project also suffered from general Project management and fiduciary issues, which impacted all components and activities. As a result, disbursements only began to accelerate in 2018. However, those delays were eventually resorbed in the second phase of implementation;  Project timeframe. The project was restructured and had to be extended for a year to implement the remaining and new activities. However, apart from the activity of hospital reform, most planned activities were implemented; additional activities were added through the second restructuring. The RBF component, while it faced many difficulties, was eventually implemented. While implementation efficiency could be rated Negligible during the first phase of implementation, it would be rated High during the second phase (disbursement of almost half of the funding in a year). At completion, 96.4 percent of the allocated financing had been disbursed. On average, implementation efficiency is therefore rated Modest. Considering the assessment of allocative and implementation efficiency, overall the efficiency of the Project is rated Substantial. Page 62 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Summary of the Completion Report for the Senegal Health and Nutrition Financing Project, November 2019 (Translation of the executive summary) REPUBLIQUE DU SENEGAL Un Peuple – Un But – Une Foi --------------------------------- MINISTERE DE LA SANTE ET DE L’ACTION SOCIALE According to the 2012-2013 DHS, considerable progress has been made in reducing maternal, neonatal and infant-child mortality between 2005 and 2013. However, efforts remain to be made on the health of children and mothers and on nutrition for children and pregnant and lactating women. In addition, the poverty survey in 2011 estimated a prevalence 46.7% and rapid population growth (around 2.7%) per year, which actually means that the number of poor increases as efforts are being made to alleviate it. This is compounded by geographical disparities that have largely remained unchanged, with two-thirds of poor households living in rural areas. The most affected area includes the regions of Kaffrine, Tambacounda, Kedougou, Kolda, Sedhiou and Ziguinchor.36 This vulnerability poses a problem with access to health services and health insurance, and also with the nutritional status of the populations of the localities mentioned above. Thus, in response to the maternal and nutritional health concerns, the Government of Senegal has received funding from the World Bank for an amount of $ 42 million (N ° credit 5324-SN, March 28, 2014) housed in the Directorate Cooperation and External Finance of the Ministry of Economy, Finance and Planning (DCFE / MEFP). This Project has a long-term goal of reducing infant mortality, but also maternal and neonatal mortality, thereby contributing to the achievement of the Millennium Development Goals (MDGs) No. 1 (Nutrition), 36 Poverty survey 2011 Page 63 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) No. 4 (Child health) and No. 5 (Maternal health). The Project shows the commitment to fall within the new government's priorities, which are to improve the governance and efficiency of social services (as described in the National Strategy for Economic and Social Development 2013-2017). As described in the project document (PAD), most of the problems in the social sectors are less the result of a lack of funding than of weaknesses in incentives and accountability mechanisms in health facilities. The two main national strategies (CMU and RBF) have the potential to be "game changer" elements, by reintroducing adequate accountability, both on the supply side (i.e., health facilities are encouraged to achieve quantitative and qualitative objectives) and on the demand side (support to health mutuelles and maternal health vouchers), not to mention the overall strengthening of the health system. In order to meet that objective, the Project has emphasized its interventions on the supply side, the demand side and the organizational, institutional and skills strengthening aspects of the sector. Firstly, the national program of RBF signed 318 annual contracts between 2015 and 2016, for a total amount of CFAF 3,242,504,032, with the health facilities and the district and regional teams; they resulting in the transfer of FCFA 1,780,238,136 to the accounts of the beneficiaries f,ollowing an external audit and certification, of which 25% for investment and 75% for staff motivation; the absorption rate of the envelope was thus 55%, which was considered low given the considerable needs for investment in health structures. 37 In 2018, 373 RBF beneficiaries are registered, distributed as follows: 327 health posts, 17 health centers, 16 districh health center, 6 hospitals, 6 regional teams, 1 PNFBR. 38 However, despite the setbacks observed in the implementation, this funding mechanism has improved the quality of health care services, as the average quality score increased from 54.6% in 2014 (Kolda and Kaffrine) to 70% in 2015 and 76.63% in 2018 in the 6 regions targeted by the project. 39 This is achieved thanks to the efforts made by the health facilities on the eight aspects of quality. At the same time, all contracted quantity indicators exceeded the national average set in 2016. 40 Then, the ACMU has been supported in its mission since the introduction of the Committees of mutualist initiative, the creation and restructuring of the mutuelles, the establishment of departmental unions, through capacity building of the CBHIs managers on administrative and financial management, and of community actors on the CMU, the sharing of conventions without forgetting communication activities and the CMU élève. The project also helped the Agency through the procurement of the Integrated Management Information System of the CMU (SIGCMU) and the acquisition of motorcycles to support departmental unions, etc. To accompany the implementation of the universal health insurance program, the DGAS identified, in 2016, 9,953 indigents following a survey in the project regions in order to enroll them in the PNBSF program, and in a mutuelle. 41 37 Project annual report 2017 38 RBF contracts, 2018. 39 Quality Verification report, 2018 40 PNFBR Annual report 2016 41 DGAS Annual report 2015 Page 64 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) These combined efforts have made it possible to increase the health insurance coverage rate from 46 to 49.3 percent between 2016 and 2017, compared with a target of 75% in 2017; and the coverage rate of the population by mutual health insurance companies from 16 to 19 percent (with a target of 45 percent in 2017).42 Other community-based interventions led by the CLM reached 389,712 children aged 6 to 59 months of whom 9,760 were diagnosed moderately malnourished and 1,242 severely. 99% of moderately malnourished children were cared for at community level and 93% of them recovered. The 1,242 severely malnourished children were all referred. The financial incentive scheme experienced by the CLM through the project has reached 2,387 pregnant women for a total of 11,937 vouchers paid in 201843. In addition, support for the DES led to the start of the implementation of a hospital information system with a proposed implementation plan after the feasibility study, without taking into account the support to hospitals within the framework of the Pluriannual Contracts of Objectives and Means (Contrats Pluriannuel d’Objectifs et de Moyens). Then, the contribution of the project to the production of the continuous DHS survey reports in recent years is marked by the payment of investigators, following a convention to this effect. Finally, many capacity building activities have been implemented, with both international and national trainings. Indeed, 46 MSAS actors have been trained or participated in international meetings; 5116 people beneficited from national training on various topics such as the RBF mechanisms, the orientations on the Sanitary Committees, the policies and standards on reproductive health, the CMU and the management of the mutuelles, the management of the MSAS information system, monitoring, etc. In summary, the Healh and Nutrition Financing Project has put a lot of emphasis on processes to support the central directorates. This does not highlight the local impact of the interventions, apart from the commissioning of a final evaluation of the project. However, the challenges encountered are based on the decentralization of interventions at the regional level, the documentation of the contribution of the RBF to the health facilities in terms of investment, the implementation of the strategy, and the regular meetings of the coordination bodies. To this end, the evaluation and the capitalization of the project concluded that the expected results, namely the utilization, the equity and the quality of maternal, neonatal, infant and nutritional health services to women and children have been reached. This is explained by the achievement of all the indicators of the performance framework, except the number of financial incentives distributed (out of 50 245 expected only 21 846 were distributed) due to insufficient monitoring of data by the CLM; and the indicator relating to the number of hospitals where the cost accounting system is set up (0 out of the 34 expected), as the related activities were non implemented. 42 ACMU Report 2017, Page 05 43 CLM Annuel Report 2018 Page 65 of 67 The World Bank Senegal Health & Nutrition Financing (P129472) The impact of RBF on the indicators analyzed with routine data is still statistically insignificant compared to those of control regions (Diourbel and Matam) except for Kaffrine. The same is true for the 2011 and 2017 DHS data where no difference is noted between the two groups of regions. However, it would be more relevant to have the EDS 2019 database. This evaluation process thus demonstrated the project’s relevance and effectiveness at 83 and 76 percent respectively. However, the effectiveness could be improved if recommendations or corrective measures are taken to that effect. All indicators are considered relevant for measuring project objectives, according to 93 of respondents to the survey. The following interventions would benefit from being integrated: incentives paid for post-natal care, the community health approach, the gender approach, the support to activity generating activities for the poor and vulnerable groups, the hiring of nurses and midwifes, and hospital governance. The same is true for the following indicators: completeness and promptness of capturing maternal and child data in DHIS2, management of malaria in children, and rate of utilization of health services by people affiliated to a mutuelle. On the other hand, efficiency remains an issue, especially with regard to the availability of resources (0.305), because of an institutional mechanism deemed not flexible for the rapid implementation of interventions. Concerning the sustainability of 69%, even if it is considered satisfactory despite local sustainability initiatives, efforts still need to be made on the mechanisms to ensure the sustainability of the project's achievements; this is linked to an absence of sustainability plan for the project. The capitalization, meanwhile, is focused on the priority package identified through an inclusive and participatory process using an analysis and scoring grid, and with the application of the selection grid. The package generated in that way includes ten priority interventions, namely: the payment of financial incentives for maternal care, growth monitoring, the screening and management of malnutrition, the communication activities in connection with the adolescent health, enrollment initiatives for children and BSF beneficiaries, capacity building of health professionals on the reproductive health of adolescents and on maternal, neonatal and child health acre, and of the actors of the mutuelles on the CMU, the payment of RBF incentives and supervision and post-training follow-up on the CMU. Those interventions are thus described: the process, the results or achievements, the difficulties or constraints, the strengths and areas for improvement and finally the partner or partners of the intervention. Page 66 of 67