Document of The World Bank Report No: ICR00003822 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-77770) ON A LOAN IN THE AMOUNT OF US$30.5 MILLION TO THE DOMINICAN REPUBLIC FOR A HEALTH SECTOR REFORM APL 2 (PARSS2) PROJECT August 29, 2016 Health, Nutrition, and Population Global Practice Caribbean Country Management Unit Latin America and Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective February 29, 2016) Currency Unit = Dominican Republic Peso (DOP$) DOP$1.00 = US$0.022 US$1.00 = DOP$46.00 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ACE Concurrent External Audit (Auditoría Concurrente Externa) APL Adaptable Program Loan CCT Conditional Cash Transfer CERSS Executive Commission for Health Sector Reform (Comisión Ejecutiva para la Reforma del Sector Salud) CPS Country Partnership Strategy CR Contributory Regime DDEI Directorate of Institutional Strategic Development (Dirección de Desarrollo Estratégico Institucional -DDEI) at MISPAS DHS Demographic and Health Surveys DR Dominican Republic ENDESA National Demographic and Health Survey (Encuesta Nacional de Demografía y Salud) FHF Family Health File (Ficha Familiar de Salud) FHI Family Health Insurance (Seguro Familiar de Salud - SFS) FM Financial Management FY Fiscal Year GODR Government of Dominican Republic HMMIS Health Management Monitoring and Information System IBRD International Bank for Reconstruction and Development (World Bank) ICR Implementation Completion Report IS Information System ISR Implementation Status and Results IT Information technology LAC Latin America and the Caribbean LCSHH Unit of Health, Nutrition and Population of the Latin American and the Caribbean Region M&E Monitoring and Evaluation M&I Monitoring and Information MISPAS Ministry of Health (Ministerio de Salud Pública y Asistencia Social) MOEPD Ministry of Economy, Planning and Development (Secretaria de Estado de Economía, Planificación y Desarrollo - SEEPyD) i MOH Ministry of Health (Ministerio de Salud Pública y Asistencia Socia- MISPAS) MRI Monitoring Results Indicators MS Moderately Satisfactory MU Moderately Unsatisfactory NCD Non-Communicable Disease NPV Net present value OM Operational Manual OP/BP Operational Policies/Bank Procedures PAD Project Appraisal Document PARSS2 Phase 2 - Health Sector Reform Project (Proyecto de Apoyo a la Reforma del Sector Salud) PCN Project Concept Note PCU Project Coordination Unit PDO Project Development Objective PIU Project Implementation Unit PROMESE/CAL Program for the Supply of Essential Medications and Office for Logistical Support (Programa de Medicamentos Esenciales y Central de Abastecimiento y Logística) RBF Results Based Financing RHS Regional Health Services. The public sector networks of health care providers that currently belong to the MOH but are in the process of decentralization. SENASA National Health Insurance (Seguro Nacional de Salud) SIUBEN Information System and Beneficiaries Registry of the Cabinet of Socio- Policy Coordination (Sistema de Información y Registro Unico de Beneficiarios) SNIP National Public Investment System SOE Statements of Expenditure SR Subsidized Regime Vice President: Jorge Familiar Country Director: Sophie Sirtaine Practice Manager: Daniel Dulitzky Project Team Leader: Christine Lao Pena ICR Team Leader: Claudia Macías ICR Primary Author: Claudia Macías and Natasha Zamecnik ii DOMINICAN REPUBLIC Health Sector Reform APL2 (PARSS2) Project (P106619) Contents Data Sheet .......................................................................................................................... iv A. Basic Information .......................................................................................................... iv B. Key Dates ...................................................................................................................... iv C. Ratings Summary .......................................................................................................... iv D. Sector and Theme Codes................................................................................................ v E. Bank Staff ....................................................................................................................... v F. Results Framework Analysis ......................................................................................... vi G. Ratings of Project Performance in ISRs ....................................................................... xi H. Restructuring ................................................................................................................ xii I. Disbursement Profile ................................................................................................... xiii 1. Project Context, Development Objectives and Design ................................................... 1 2. Key Factors Affecting Implementation and Outcomes .................................................. 7 3. Assessment of Outcomes .............................................................................................. 18 4. Assessment of Risk to Development Outcome ............................................................. 26 5. Assessment of Bank and Borrower Performance ......................................................... 26 6. Lessons Learned............................................................................................................ 29 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............... 31 Annex 1. Project Costs and Financing .............................................................................. 33 Annex 2. Outputs by Component...................................................................................... 33 Annex 3. Economic and Financial Analysis ..................................................................... 41 Annex 4. Bank Lending and Implementation Support/Supervision Processes................. 45 Annex 5. Beneficiary Survey Results ............................................................................... 47 Annex 6. Stakeholder Workshop Report and Results....................................................... 47 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 48 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 60 Annex 9. List of Supporting Documents .......................................................................... 61 MAP .................................................................................................................................. 62 iii Data Sheet A. Basic Information Country: Dominican Project Name: Health Sector Republic Reform Second Phase APL (PARSS 2) Project ID: P106619 L/C/TF Number(s): IBRD-77770 ICR Date: 08/08/2016 ICR Type: Core ICR Lending APL Borrower: GOVERNMENT Instrument: OF THE DOMINICAN REPUBLIC Original Total USD 30.50M Disbursed Amount: USD 29.56M Commitment: Revised Amount: USD 29.56M Environmental Category: C Implementing Agencies: MINISTRY OF HEALTH CERSS Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept 03/03/2009 Effectiveness: 01/28/2010 01/19/2010 Review: Appraisal: 06/17/2009 Restructuring(s 06/30/2014 ): 10/29/2015 12/04/2015 Approval: 09/17/2009 Mid-term 11/26/2012 11/26/2012 Review: Closing: 10/30/2015 02/29/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Moderately Satisfactory iv C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Moderately Satisfactory Quality of Implementing Satisfactory Satisfactory Supervision: Agency/Agencies: Overall Bank Overall Borrower Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time No None (QEA): (Yes/No): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 5 5 Public administration- Health 95 95 Theme Code (as % of total Bank financing) Health system performance 100 100 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Pamela Cox Country Director: Sophie Sirtaine Yvonne M. Tsikata Practice Manager: Daniel Dulitzky Keith E. Hansen Project Team Christine Lao Pena Fernando Montenegro Torres Leader: ICR Team Leader: Claudia Macias ICR Primary Author: Claudia Macias/Natasha Zamecnik v F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project's Development Objectives are to: (i) improve the capacity of Regional Health Services (RHS) to deliver, in a timely fashion, quality services known to improve the health of mothers, children and people with chronic conditions by public providers at the first level of care; (ii) improve health system responsiveness, defined here as the institutional capacity of public sector health organizations to conduct strategic purchasing of health care services and goods, and to respond to public health emergencies. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Percentage of pregnant women from target population with risk evaluation Indicator 1: completed before the 15th week of pregnancy Value quantitative or 0.43% 50% 40% 50% Qualitative) Date achieved 06/27/2011 10/30/2015 06/27/2014 02/29/2016 Target surpassed at 125% for revised target and 100% for original target. Comments Surpassed meaning achievement between 95-100+%. Target revised in (incl. % June 2014 because of an expansion of the beneficiary population. achievement) Source: DDEI, Ministry of Health (MOH) Percentage of children from target population under 15 months with Indicator 2: vaccination scheme completed according to national protocols Value quantitative or 0% 60% 68% Qualitative) Date achieved 06/27/2011 10/30/2015 02/29/2016 Comments Target surpassed at 113%. (incl. % Source: Directorate of Institutional Strategic Development (DDEI), MOH achievement) Percentage of individuals from target population diagnosed with Indicator 3: hypertension under treatment according to national protocols Value quantitative or 3.54% 50% 35% 38% Qualitative) vi Date achieved 06/27/2011 10/30/2015 06/27/2014 02/29/2016 Comments Target revised in June 2014 due to an expansion of the beneficiary (incl. % population. Revised target surpassed at 110% and original partially achievement) achieved at 74%. Source: MOH Studies on risk factors and burden of disease have been conducted, Indicator 4: disseminated and used by corresponding MOH units (Salud Colectiva) to develop annual operational plans for health promotion and prevention. Results of Studies Value studies used to conducted, quantitative or No studies develop annual disseminated and Qualitative) operation plans used Date achieved 08/20/2009 10/30/2015 02/29/2016 Comments Target achieved. The studies were conducted, disseminated and partially (incl. % used by the MOH units. achievement) Source: DDEI, MOH At least one region is producing monthly reports on the number of poor individuals who were prescribed a medicine at the first level and actually Indicator 5: received medication within 48 hours at the point of consultation service or in a PROMESE/CAL facility Regions VI and VIII are producing reports, At least one including a Value region producing dashboard with quantitative or No monthly reports annual reports information Qualitative) extracted from the database generated by SIMI Date achieved 08/20/2009 10/30/2015 02/29/2016 Comments Target achieved. (incl. % Source: DDEI, MOH achievement) vii (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Percentage of Clinical files with complete information for external audit Indicator 1: to verify RHS reported MRI results Value quantitative or 0% 85% 90% Qualitative) Date achieved 08/20/2009 10/30/2015 02/29/2016 Comments Target surpassed at 106%. (incl. % Source: National Health Insurance (SENASA) achievement) Percentage of post-partum women referred from hospital to their Indicator 2: corresponding first level of care units with proper documentation included in their clinical files of health center Value quantitative or 0.44% 50% 63% Qualitative) Date achieved 06/27/2011 10/30/2015 02/29/2016 Comments Target surpassed at 126%. (incl. % Source: DDEI, MOH achievement) Percentage of children over 14 months and under 24 months with Indicator 3: vaccination scheme completed according to national protocol Value quantitative or 0% 60% 45% 63% Qualitative) Date achieved 06/27/2011 10/30/2015 06/27/2014 02/29/2016 Comments Revised (140%) and original (105%) targets surpassed. Target revised in (incl. % June 2014 because of an expansion of the beneficiary population. achievement) Source: DDEI, MOH Percentage of children under 24 months with growth and development Indicator 4: monitor controls according to national protocol Value quantitative or 0.27% 60% 50% 62% Qualitative) Date achieved 06/27/2011 10/30/2015 06/27/2014 02/29/2016 Comments Revised (103%) and original (124%) targets surpassed. Target revised in (incl. % June 2014 because of an expansion of the beneficiary population. achievement) Source: DDEI, MOH viii Percentage of individuals from target population screened for Indicator 5: hypertension according to national protocol Value quantitative or 0.89% 40% 60% 64% Qualitative) Date achieved 06/27/2011 10/30/2015 06/27/2014 02/29/2016 Comments Revised (106%) and original (160%) targets surpassed. Target revised in (incl. % June 2014 because of an expansion of the beneficiary population. achievement) Source: DDEI, MOH Percentage of individuals from target population screened for diabetes Indicator 6: according to national protocol Value quantitative or 0.07% 40% 35% 43% Qualitative) Date achieved 06/27/2011 10/30/2015 06/27/2014 02/29/2016 Comments Revised (123%) and original (108%) targets surpassed. Target revised in (incl. % June 2014 because of an expansion of the beneficiary population achievement) Source: DDEI, MOH Indicator 7: Percentage of individuals diagnosed with diabetes type II under treatment Value quantitative or 0.79% 40% 30% 34% Qualitative) Date achieved 10/31/2011 10/30/2015 06/27/2014 02/29/2016 Revised target surpassed (114%) and original target achieved (85%). Comments Achieved meaning achievement between 80-94%. Target revised in June (incl. % 2014 because of an expansion of the beneficiary population. achievement) Source: DDEI, MOH Percentage of individuals symptomatic of TB with results from sputum Indicator 8: smears in clinical file of corresponding health center Value quantitative or 0% 40% 35% 45% Qualitative) Date achieved 06/27/2011 10/30/2015 06/27/2014 02/29/2016 Comments Revised (129%) and original targets (113%) surpassed. Target revised in (incl. % June 2014 because of an expansion of the beneficiary population achievement) Source: DDEI, MOH The Health Management System of the MOH includes an automated/electronic module for M&I with all the information needed for Indicator 9: monitoring results on a quarterly basis at the first level of care with data available for RHSs. MOH central level, and SENASA Value Results-adjusted Model developed quantitative or No Module Capitation and used by Qualitative) module used in RHS's ix regions VI and VIII Date achieved 08/20/2009 10/30/2015 02/29/2016 Comments Target achieved (incl. % Source: DDEI, MOH achievement) Risk factors and burden of disease surveys designed, implemented and Indicator 10: results disseminated Results of studies Results of completed studies and completed and disseminated disseminated and and used for Value used for planning planning Studies realized No studies quantitative or public health public health and results Qualitative) interventions interventions disseminated with an emphasis with an on health emphasis on promotion and health prevention promotion and prevention Date achieved 08/20/2009 10/30/2014 10/30/2015 02/29/2016 Comments Target achieved (incl. % Source: DDEI, MOH achievement) Accredited health centers from participating RHS that do not have a Indicator 11: PROMESE/CAL pharmacy and regularly receive medicines procured and distributed by PROMESE/CAL Value quantitative or 0% 80% 100% Qualitative) Date achieved 08/20/2009 10/30/2015 02/29/2016 Comments Target surpassed at 125% (incl. % Source: DDEI, MOH achievement) x G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 12/29/2009 Satisfactory Satisfactory 0.00 2 06/29/2010 Satisfactory Satisfactory 1.42 3 02/23/2011 Satisfactory Satisfactory 2.50 Moderately Moderately 4 08/13/2011 3.69 Satisfactory Satisfactory Moderately Moderately 5 03/22/2012 5.61 Satisfactory Satisfactory Moderately Moderately 6 11/02/2012 8.48 Satisfactory Satisfactory Moderately Moderately 7 05/24/2013 10.75 Satisfactory Satisfactory Moderately Moderately 8 12/27/2013 14.40 Satisfactory Satisfactory Moderately Moderately 9 07/12/2014 17.79 Satisfactory Satisfactory Moderately 10 12/24/2014 Satisfactory 20.90 Satisfactory Moderately 11 06/24/2015 Satisfactory 23.65 Satisfactory Moderately 12 11/05/2015 Satisfactory 25.96 Satisfactory Moderately 13 02/29/2016 Satisfactory 29.03 Satisfactory xi H. Restructuring ISR Ratings Amount Board at Disbursed at Restructuring Reason for Restructuring Approved Restructurin Restructurin Date(s) & Key Changes Made PDO Change g g in USD DO IP millions (a) Revised the Results Framework to modify the baseline data for one indicator and targets for nine indicators; (b) adjusted the cost of two components and 06/30/2014 MS MS 17.49 eliminated the counterpart funding requirement for Component 1; (c) reallocated funds among categories; and (d) updated the disbursement estimates. Extended the Project closing date by four months from October 30, 2015 to February 29, 2016. The restructuring allowed for the completion 10/29/2015 S MS 25.96 of: (a) results-based transfers to participating regions, (b) the external technical audit related to results-based transfers, and (c) technical assistance to the MOH. The third restructuring: (a) modified the Project’s financing plan to increase the share of Bank financing retroactively; (b) reallocated 12/04/2015 S MS 26.09 funds between disbursement categories; (c) updated disbursement estimates; and (d) adjusted the cost of components. xii I. Disbursement Profile xiii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. The Health Sector Reform Program (PARSS2) is the second phase of a 12-year Adaptable Program Loan (APL) which was expected to be implemented in three phases. The total cost of the APL Program was estimated at US$126.71 million, of which the World Bank was to provide loan financing totaling US$90 million. The first phase of the APL, the Health Sector Reform Project (PARSS1), became effective in 2005 and closed on December 31, 2009 with a total cost of US$39.1 million, of which the World Bank provided financing of US$30 million. 2. At the time of Project appraisal, the Dominican Republic (DR) had achieved positive results in key health outcomes. The DR showed a reduction in under-five mortality from 88 to 36 per thousand live births. Infant mortality had dropped from 67 to 32, and neonatal mortality from 40 to 23 per thousand live births. Mortality statistics showed that deaths from communicable diseases had declined with the number of malaria cases, an endemic disease in the country, falling from 3,525 in 2006 to 1,838 in 2008 and 1,643 in 2010 while the prevalence of tuberculosis, another high priority health problem, had also declined since 2000. Chronic malnutrition in children under five also demonstrated a downward trend over the previous ten years. Despite improvements in a number of health outcomes, the Dominican health system achieved lower mother and child health outcomes compared to countries in Latin America with similar or lower levels of economic development, while chronic diseases were emerging as a new challenge. 3. The impact of non-communicable diseases (NCDs) in the country, as in many low and middle low income ones, was steadily growing. NCDs, including cardiovascular disease, cancer, chronic respiratory disease, and diabetes, led among major causes of death, accounting for 32.8 percent of all deaths reported in 2004 and 33.7 percent in 2005. In an effort to strengthen chronic disease prevention and control, the Ministry of Health (MOH) established the National Chronic, NCD Prevention and Control in 2009. 4. The Government of the Dominican Republic (GODR) had previously embarked on an ambitious health sector reform program aimed at improving health system responsiveness. The reform envisioned financial as well as organizational arrangement changes to improve the health system’s responsiveness while enhancing financial protection for the poor. An important milestone at the beginning of this process was the 2001 passage of two key laws: The General Health Law (42-01) and the law establishing the Dominican Social Security System (87-01). Key provisions of the laws included the establishment of a universal health insurance system and the restructuring of public health service provision, including the progressive decentralization of regional health care networks. The health reform configured the National Health System under the direction of the MOH, with a shift toward a separation of functions related to the stewardship, insurance and service provision as a way to enhance health system performance. The passage of the laws also established the provision of a mandatory 1 package of health services known as the Family Health Insurance – FHI (Seguro Familiar de Salud).1 5. Despite the progress achieved, at the time of appraisal, the DR still faced important challenges regarding health outcomes, service quality and expenditure efficiency: a) Lower health outcomes compared to other LAC countries. Compared to other countries in Latin America with similar or lower levels of economic development, the Dominican health system achieved lower mother and child health outcomes, with chronic diseases emerging as a new challenge. Countries such as El Salvador, Peru and Paraguay had achieved significantly lower levels of maternal, infant and child mortality. For example, in 2010 the estimated maternal mortality rate in the DR was 130,000 per 100,000 live births while the maternal mortality rate in El Salvador was estimated at 110 and 120 in Peru. In the same year, the estimated infant mortality rate in the DR was 28 per 1,000 live births while El Salvador had achieved an infant mortality rate of 17 and Peru of 16 per 1,000 live births. El Salvador had a mortality rate for children under five of 20, Peru of 21 and the DR of 34 per 1,000 live births.2 b) Inefficiency and low public health service quality. The Government had made progress in improving access to public health services yet much more needed to be done to achieve better quality. For example, the MOH had identified problems in the quality of prenatal controls as one of the factors that contributed to maternal mortality. c) Weaknesses in health system management. The MOH’s role in defining health sector priorities, formulating policies and norms, and regulating health care activity within a decentralized context required further strengthening. Most managers were not trained to plan for the strategic use of their resources, to manage and use clinical and management IS and to measure outcomes. d) Weaknesses in public spending quality and control. As a result of the global economic crisis and the country’s fiscal difficulties, the GODR adopted measures to improve the quality of public expenditures as a measure to narrow its financing gap. Improving the quality of spending was a precondition for a sustained investment in the country’s social and development program. 6. The World Bank supported the GODR’s long term health sector objective of guaranteeing access to health services to all the Dominican population to allow a satisfactory resolution of their demands and better health outcomes. The World Bank 1 The FHI services covers health promotion, disease prevention and treatment, rehabilitation, and pregnancy and its consequences and treatments resulting from traffic accidents. FHI benefits materialize through the Basic Health Plan which is composed of basic and comprehensive services including: (a) promotion and prevention; (b) primary care, including emergencies, outpatient services and address; (c) specialized care, complex, hospital surgical treatments and assistance; (d) medicines (full coverage for the population in the subsidized regime (SR) and 70 percent coverage for the population in the contributory regime (CR) ); (e) diagnostic tests; Pediatric Dentistry; (G) physiotherapy and rehabilitation; and, (h) supplementary benefits, including prostheses and technical assistance to persons with disabilities 2 World Bank Indicators (2010). 2 supported the GODR’s health sector short and medium term objectives of promoting the separation and decentralization of the four basic functions of health systems: stewardship, financing, insurance and provision that would contribute to the long term objectives of the reform. The World Bank has supported the DR’s health reform process since before its inception in 2001 through its Provincial Health Services Project (4271-DO), which targeted policy and institutional reform through the development of instruments, training of personnel, and improving physical and technological infrastructure in the poorest regions of the country. The first phase of the Program (PARSS1), together with World Bank technical assistance, contributed to critical policy changes and health sector reform interventions. In particular, it contributed to the rapid expansion of health insurance coverage in both the subsidized (SR)3 and contributory (CR)4 regimes5 which are the two main pillars of the country’s health insurance coverage. PARSS1 also backed the institutional strengthening of the MOH (MISPAS - Ministerio de Salud Pública y Asistencia Social), National Health Insurance (SENASA - Seguro Nacional de Salud and the Program for the Supply of Essential Medications (PROMESE/CAL - Programa de Medicamentos Esenciales y Central de Abastecimiento y Logística) by supporting the preparation and implementation of norms, regulations and protocols; and providing training and support to changes within these institutions. It also contributed to strengthen primary care physical infrastructure. 7. The rationale for proceeding with APL2 was strong since the Project represented a natural continuation of World Bank assistance to the APL1. PARSS2 would support some PARSS1 related health reform elements, which remained important challenges, such as increasing health coverage to the FHI, further decentralization, but particularly, improving the quality of care and enhancing the quality of public spending on health care with a focus on the achievement of results. The Project was to prioritize resources for strengthening primary care to improve access to quality services and help meeting the challenge of reducing the burden of disease and out of pocket expenditures of the DR poorest households. 8. The Project was consistent with the World Bank’s Country Partnership Strategy 6 (CPS), the DR National Development Plan and the Ministry of Economy, Planning and Development (MOEPD) Strategic Plan. The Project was closely aligned with two of the strategic development challenges identified in the CPS: (1) strengthening social cohesion and improving access to and quality of social services; and, (2) enhancing quality of public expenditures and institutional development. These objectives responded to the DR national development strategy that called for reducing poverty and providing universal health service access, exploiting synergies resulting from poverty reduction interventions across sector and public agencies and supporting financial sustainability and sound fiscal policy while improving the quality of priority health service delivery known 3 At the time of project closing, about one-third of the total PARSS 1 target population (1.3 million out of 3.2 million) were enrolled, representing approximately 10 percent of the country’s total population. 4 By project closing 2.2 million individuals were enrolled in the contributory system. 5 The SR provides financial protection to the informal sector while the CR focuses on the formal sector. 6 Country Partnership Strategy for the Dominican Republic for the period FY10-FY13, Report No. 49620-DO, dated August 12, 2009.FF. 3 to improve health outcomes for the poor. The Project was consistent with the Ten-Year Health Plan (2006-2015), which set among its priorities the development of the primary health care strategy and enhancement of inter-sector work within the National Health System. The Project was also in agreement with the MOEPD Strategic Plan (2008-2012) that aimed to gradually introduce performance-based budgeting techniques and results agreements to different sectors and public institutions. Therefore, the World Bank’s experience with the Dominican health sector and with the RBF mechanism worldwide made it a natural partner in improving health service delivery and promoting transparency and accountability in the sector. 1.2 Original Project Development Objectives (PDO) and Key Indicators 9. The original Project Development Objectives were: (i) improve the capacity of RHS to deliver, in a timely fashion, quality services known to improve the health of mothers, children and people with chronic conditions by public providers at the first level of care; (ii) improve health system responsiveness, defined as the institutional capacity of public sector health organizations to conduct strategic purchasing of health care services and goods, and to respond to public health emergencies.7 10. The Project had five PDO indicators. The first three were associated with the first PDO and were selected from the list of the FHI basic package of health services and the last two were associated with the second PDO. These indicators were selected since they are a good tracking measure of the burden of disease of the target population and because they were directly linked to the activities financed by the Project.  Percentage of pregnant women from target population with risk evaluation completed before the 15th week of pregnancy;  Percentage of children from target population under 15 months with vaccination scheme completed according to national protocol;  Percentage of individuals from target population diagnosed with hypertension under treatment according to national protocol;  A baseline on risk factors and burden of disease has been conducted, disseminated and used by corresponding MOH units to develop annual operational plans for strengthening public health interventions with an emphasis on health promotion and prevention; and,  At least one region is producing monthly reports on the number of poor individuals who were prescribed a medicine at the first level of care according to national protocols and who actually received medication within 48 hours at the point of consultation service or in a facility of the PROMESE/CAL. 7 The overarching objectives of the APL approved by the Board in 2003 were: (i) improved mother and child health, and poverty reduction; (ii) implementation of new health sector reform laws (42-01 and 87-02), including, strengthening the stewardship role of the MOH; and (iii) consolidating universal health insurance as the public stewardship organization for the health sector; development of regional health care networks; and health insurance mechanisms. 4 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 11. The targets for PDO indicators 1 and 3 were revised during the first restructuring (2014). The targets for PDI indicators one and three were revised due to the Project’s implementation expansion of the RBF mechanism to an additional region (Region VII). The target for PDO indicator 1, percentage of pregnant women from target population with risk evaluation completed before the 15th week of pregnancy, was revised downward from 50 to 40 percent. The target for PDO 3, percentage of individuals from target population diagnosed with hypertension under treatment according to national protocols, was also revised downward from 50 to 35 percent. The baseline data for PDO indicators 1-3 (as well as intermediate indicators 2-8) was not obtained until 2011 due to delays in the development of key IS applications. See Section 2.3 for more information. 1.4 Main Beneficiaries 12. The original target population was approximately 825,000 individuals from households categorized as poor living in participating regions8 VI and VIII.9 These included both the population covered or enrolled in SENASA’s SR and those lacking health insurance coverage. Of the total target population, approximately 271,000 were enrolled in the SR and 554,000 were not (Table 1). Table 1. Individuals to benefit from the introduction of results-adjusted capitations at the first level of care at Appraisal No. of individuals Region VI Region VIII Total from poor households Total No. of Poor 459,295 365,669 824,964 Enrolled in SR 148,525 122,525 271,050 Not enrolled 310,770 243,144 553,914 13. By Project closing, 366,236 individuals had been affiliated. These were direct Project beneficiaries. Following a Project restructuring in June 2014, Region VII was added to the Project target population. The poor population in the three regions ultimately participating in the Project (Region VI, VII and VIII) benefited from improved health sector institutional capacity and service delivery for a total coverage of 1.05 million individuals from poor households. 8 There are nine public regional health service providers in the DR connected by a network of complexity levels with the capacity to provide at least minimum, cost-efficient care as stipulated in the Basic Health Plan. 9 The categorization of “poor” in the DR is established through an In formation System and Beneficiaries Registry of the Cabinet of Socio-Policy Coordination (SIUBEN) proxy means test. 5 1.5 Original Components 14. The Project consisted of the following four components:  Component 1: Introducing results-based financing mechanism for the first level of care in Regional Health Services (US$21.49 million – Bank financing US$8.25 million). Using results-adjusted capitations, this component was expected to contribute to finance the delivery of ambulatory primary health care services to poor individuals in selected RHS (Regions VI and VII).10  Component 2: Strengthening the capacity of the MOH to improve and monitor health system responsiveness while fostering transparency and accountability (US$10.00 million – Bank financing US$6.10 million). This component was expected to finance goods, technical assistance, non-consulting services, training and operational costs to strengthen MOH capacity to deliver public health goods and services while fostering transparency and accountability, and to strengthen existing IS for the adequate functioning of results-adjusted capitations.  Component 3: Improving the quality of public spending on health care goods and services (US$12.57 million – Bank financing US$15.05 million). This component was expected to finance goods, technical assistance, non-consulting services, training and operational costs to strengthen coordination and institutional capacity of public sector organizations for more sustainable financing, planning and purchasing of health goods and services with an emphasis on primary health care.  Component 4: Support for Response to Public Health Emergencies (US$0.10 million – Bank financing US$0.08 million). This component would finance consulting and non-consulting services, technical assistance and goods including medicines, laboratory and protective equipment, laboratory reagents and other medical supplies, development and dissemination of materials for information campaigns, training for emergency preparedness to support the response to public health emergencies of local or international nature. . The component included a fast disbursing mechanism which could be accessed by a formal request from the Minister of Health to the World Bank following the declaration of a public health emergency 11 together with a non-objection request and procurement plan for activities to be carried out. 1.6 Revised Components N/A 10 These regions were selected using several criteria including progress in the preparation of performance agreements with the MOH and decentralization process, institutional capacity in managerial and technical aspects of delivering health care, size of the population categorized as poor by SIUBEN and size of the population covered by the SR. 11 The operational definition of a public heal th emergency would be consistent with the DR’s Risk management Law (147-02). 6 1.7 Other significant changes 15. The Project had three restructurings. The first restructuring was approved in June 2014. It had the purpose to (i) align indicator targets to take into account the expansion of the Project from Regions VI and VIII to include Region VII, update target population estimates based on then recent surveys and studies; (ii) use Project funds to cover the remaining financing needs of Component 1; (iii) reallocate funds to key activities that would further contribute to the attainment of the PDOs; and (iv) adjust disbursement estimates to reflect the expenditure projections based on the updated implementation plan. This restructuring: (a) revised the Results and Monitoring Framework to modify the baseline data for one indicator, and targets for nine indicators; (b) adjusted the cost of two components and eliminated the counterpart funding requirement for Component 1; (c) reallocated (US$3.6 million) from Component 1 (Introducing RBF Mechanism for the First level of Care in Regional Health Services) to successfully complete the planned activities under Component 3 (Improving the Quality of Public Spending on Health Care Goods and Services) of the Project;12 and (d) updated disbursement estimates. 16. The second restructuring involved an extension of the closing date in order to address insufficient counterpart funding. The second restructuring was approved on October 29, 2015 and extended the Project closing date by four months, from October 30, 2015 to February 29, 2016. The restructuring provided Project stakeholders with additional time to address the funding gap that had developed due to lacking counterpart funding and allowed for the completion of: (a) results-based transfers to participating regions, (b) the external technical audit related to results-based transfers; and (c) the implementation of technical assistance to the MOH. 17. The third restructuring was approved in December 2015 to alleviate funding constraints faced by the Government due to the country’s fiscal situation. The restructuring retroactively reduced counterpart funding from 31 to 12 percent of total Project costs. It: (a) modified the Project’s financing plan to increase the share of Bank financing retroactively; (b) reallocated funds between disbursement categories 13 ; (c) updated disbursement estimates; and (d) adjusted the cost of components. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 12 The reallocation was mainly proposed because of a reduction in the number of individuals to be covered in the participating regions under Component 1 due to the National Health Insurance/SENASA's active campaign in 2012 to cover more poor uninsured individuals under the Subsidized Health Insurance Regime. 13 The Government of the Dominican Republic requested a reallocation of unutilized US$2.3 million from Category 1 (Capitation Transfers Output based Disbursement) under Component 1 (Results-based financing mechanism for the first level of care) to complete the activities and finance the counterpart funds to Category 3 under Component 3 (Improving the Quality of Public Spending on Health Care Goods and Services) of the Project. 7 18. Given the Government’s ambitious, long-term vision for sector reform and the breadth of the existing legal framework for the health reform, the choice of an APL as a financing instrument was appropriate. The three phases had a logical approach, which built on the preceding activities. The selection of the PDOs correctly identified the key issues relevant to the sector and country and addressed the Government’s main health sector challenges. 19. The rationale for World Bank support was based on the long experience and cooperation with the DR’s health sector and lessons learned prior to Project preparation.14 APL2 took into account the lessons learned from APL1 implementation, including: (i) use of existing institutions and organizational arrangements to reduce implementation delays and ensure sustainability; (ii) coordination with other projects and development partners to exploit synergies with multi-sector interventions that focus on human development; and (iii) inclusion of a component with a rapid disbursing mechanism to support the Government’s emergency response capacity in the face of epidemics, natural disasters and/or other events that pose a threat to the public health. 20. The decision to move forward with PARSS2 was based on the progress achieved in PARSS1. Progress in the health reform agenda, in particular regarding two of the key bottlenecks - health insurance coverage and decentralization of services - supported the decision to move forward with the second phase15. Ongoing sector reform along with the improved definition of roles and coordination among key stakeholders and the development of a joint MOH-SENASA strategy based on results-based financing for the first level of care lay the groundwork for the preparation of PARSS2. 21. The Project responded to the GODR’s policy and priorities of supporting a more efficient allocation of resources, while maintaining poverty reduction goals. The Ministries of Health, Finance, Economy, Planning and Development and the SENASA decided to introduce a financing mechanism for the first level of care that generated incentives for an improvement in the quality of service provision with an emphasis on the poor. Results-adjusted capitations was the financing mechanism identified as the best option to improve the quality of spending through the separation of financing from the delivery of care and the introduction of strategic purchasing tools. With the support of the World Bank, the SENASA signed contracts with two RHSs (regions VI and VIII) during Project preparation to facilitate the introduction of the mechanism. The Project was consistent with the CPS FY2010 -13 (Report No. 49620-DO) discussed by the Executive 14 At the time of Project preparation the World Bank had more than 10 years’ experience in the Dominican health sector. Since 1998 the World Bank’s Provincial Health Project (Ln. 4272 -DO) had supported the provision of primary care services to the poorest population, with emphasis on maternal and child care in three regions (III, IV and VII), one province of Region VIII (Sánchez Ramírez), and in one health area of the national district (Santo Domingo Centro). In 2002, the first phase of the Program (PARSS1, LN 71850-DO) supported the GODR health sector priorities as set out in the Health Sector Reform and Modernization Program. 15 PARSS 1 successfully supported a wide range of relevant activities supporting progress of the health sector reform. However, it should be noted that the Project’s overall outcome was considered unsatisfactory based on the poor achievement of almost all of the outcome indicator targets. 8 Director on August 12, 2009, which had social cohesion, quality of public expenditures and institutional development among its pillars. Box No. 1. Capitations and results-based financing in health sector operations Capitations constitute a special payment mechanism that Bank financed projects have used in the design of various health sector operations in the region. Resources are allocated for the delivery of care to a target population on a per capita basis adjusted by the achievement of targets of priority services. Regional Health Services receive capitation payments in two installments to provide incentives for providers to enroll beneficiaries and to deliver services and meet agreed targets. The first installment (50 percent in this case) is calculated using the total certified list of beneficiaries and the average unit cost. The second installment (up to 50 percent) is calculated with data of each health providers’ achievement of results (targets for heath outputs/intermediary outcomes). In the case of the Project capitation payments, these are made on the basis of (i) a roster of eligible individuals; and (ii) the achievement by the RHSs of targets of a set of ten indicators (both verified by an external concurrent audits) (See Table 2 and Annex 2). Not every regional level provider receives 100 percent of the second installment; some may receive a smaller percentage if not all the targets are met. Targets and rules for transfers to the regional health providers are part of performance agreements established between the national and sub-national levels. Performance agreements are negotiated annually and include specific targets for health outputs/intermediary outcomes directly linked to delivery of the health services package. Capitation, used along with rigorous monitoring, and technical (medical) and financial auditing have supported the improvement of expenditure efficiency in the public health sector. Capitation mechanisms have been used as a strategic purchasing tool within the framework of policy reforms aimed at separating health system functions (financing, risk pooling, and delivery of services). Capitations can contribute to align providers’ incentives for improving coverage and quality of primary health care services. 22. Some of the key Project design strengths are the following:  Strategic relevance and appropriateness of Development Objectives: The PDOs selected adequately identified the DR’s relevant health sector challenges. The Project was firmly rooted in the GODR’s national health legal framework and supported the health sector transformation process started in 2001. It was consistent with the World Bank’s CPS and responded to the GODR and the World Bank policy and priorities of promoting a more efficient allocation of resources while maintaining poverty reduction goals.  Intensive policy dialogue supported the establishment of a well-planned Project design. Ongoing policy dialogue with the MOEPD, MOH and key health sector authorities initiated before PARSS1 implementation resulted in the inclusion of relevant health sector activities and a well thought out design, which resulted in a successful Project execution. 9  Quality of institutional framework: The Project was structured to work within the GODR institutional framework. The Project design was in line with the Government’s implementation capacity, as had been demonstrated during the implementation of APL1. Close coordination with the MOH and SENASA ensured that objectives could be achieved within existing institutional units and structures. The Project took advantage of the maturing coordination of processes between the MOH, the SENASA and the regional health care networks. It used existing resource allocation mechanisms for the SR and introduced some important refinements to improve providers’ incentives to deliver better quality of care.  Extent of integration and financial management and procurement quality: Adequate measures were in place to support early start-up of procurement activities, with full integration of the team’s procurement and financial management (FM) specialists during the design phase. The Project FM risk assessment was thorough, and took into account the Bank's 2005 Country Fiduciary Assessment Report. In May 2009 the World Bank carried out an assessment of the capacity of the two implementing agencies to implement Project procurement activities.  Strong collaboration with other development partners. Project design built upon experience gained from other projects and development partners to exploit multi-sector synergies. The Project was designed in close coordination with the social protection Conditional Cash Transfer (CCT)-Programa Solidaridad) project managed by the Inter-American Development Bank (IADB), ensuring complementarities. The Project used the CCT project’s clinical management system (CMS) for reporting targets and accountability processes for the RBF mechanism under Component 1. The World Bank’s Social Protection Investment Project (IBRD-7481) and its additional financing (IBRD-7798) provided critical support regarding the targeting of Component 1 affiliates, the documentation of poor citizens and social accountability mechanisms of the CCT program.  Inclusion of a specific component (Component 4) with a fast disbursing mechanism to respond to emergency situations resulting from epidemics, natural disasters or other public health threats. This was a result of lessons learned from the handling of public health emergencies during the implementation of other country World Bank health projects. 23. Implementation readiness was adequate for successful implementation of Project activities. Project technical discussions and preparatory activities began during PARSS1 implementation. By 2008, the MOH and SENASA had established coordination mechanisms including: (a) signed performance agreements and SENASA contracts with the RHSs; (b) introduced counterpart funding in the 2010 draft budget; (c) agreement on Monitoring Results Indicators (MRI) 16 between the MOH and VI and VIII RHSs; (d) DDEI’s (Directorate of Institutional Strategic Development -Dirección de Desarrollo Estratégico Institucional -DDEI) initiation of the process of selection and contracting of a procurement specialist with experience in Bank’s operations; and, (e) development of the terms of reference for the external concurrent audit. 16 The MRI indicators measured service quality according to national protocols and also monitoring Project progress. 10 24. Risk Analysis. Although the PAD identified a number of country, governance, institutional and operational risks, it did not sufficiently consider the challenges resulting from multiple agency implementation and counterpart funding. The established arrangement raised unrealistic expectations among the implementing units regarding their autonomy in developing separate operational and procurement plans and fiduciary responsibility over project implementation. As a result, once the Project became effective, differences over how to plan and manage activities emerged and hindered Project implementation. The lack of sufficient counterpart funding caused implementation delays and was likely not identified due to the fact that PARSS1 did not encounter counterpart funding problems during implementation.17 2.2 Implementation 25. Existing control mechanisms and management structures facilitated the implementation of key Project activities. The Project used several existing resource allocation mechanisms for the SR. For example, the clinical management system, developed for the CCT Solidaridad Program for verifying compliance with health co- responsibilities by households covered by the Program, was the main data source for a number of results framework indicators, especially for monitoring Component 1 and served as a health service quality-monitoring tool for the MOH. Use of the clinical management system was mandatory for health staff in order to promote its use and generate more data. The MOH also encourage the system’s use through incentives, training, and logistical and technical support. The Project also used an existing beneficiary identification instrument to target the poor. 26. The establishment of inter-institutional arrangements allowed for the coordination between a number of the participating government and sector institutions. An Inter-institutional Oversight Committee (Comité Directivo del Proyecto) set the MRI targets18 and made priority decisions.19 In addition, a Results Based-Financing Committee (RBFC)20 oversaw the day-to-day RBF mechanism operations in the selected regions. These inter-institutional arrangements guaranteed some level of coordination, particularly given the establishment of two different implementing units (see paragraph 29). 27. The Government acted proactively to overcome enrollment and information systems challenges. For example, to face the challenge that affected the limited enrolment of Project affiliates (disbursement was less than 1 percent of the initial 18-month Annual Operating Plan budget), the Government modified the tool used by the RHS to identify the eligible population (Information System and Beneficiaries Registry of the Cabinet of Socio-Policy Coordination, SIUBEN). It also rolled out the clinical management system 17 The ICR Report for PARSS1 makes reference to the Government satisfactory allocation of funds to finance Project activities despite the global economic crisis and its negative effects on the country. 18 MRI targets refer to the annual established results for each one of the monitoring result indicators (MRI). 19 This Committee comprised representatives from the MOH, MOEPD, SENASA, PROMESE/CAL and CERSS. 20 This Committee comprised representatives from the MOH, MOEPD, SENASA and the PCU. 11 (CMS), key for monitoring health indicators at primary care level, which was seldom used by the health system staff. As a result, the Project updated the Operating Manual to modify the identification system to cover individuals who were not registered in the contributory and subsidized health insurance regimes of the Dominican Social Security System prioritizing the poor. This adjustment displayed positive results as the number of enrolled individuals started to increase at a higher rate. 28. Project management commitment, intensive follow up on Project activities and Regional Health Services (RHS) ownership of the process contributed to the early achievement of results. The Mid-term Review (MTR) mission in November 2012, the technical support mission from January 28-February 1, 2013, and subsequent follow up discussions with the Project team reported important progress achievement in addressing various implementation bottlenecks, contributing to results improvements. Most of the clinical management system (CMS) issues, a central tool for Component 1 indicator measurement, were resolved as a result of support provided by the Social Cabinet and the PCU in collaboration with RHS authorities. The Social Cabinet provided IT trouble shooting support in the use of the CMS (there was a technical support hotline) while the PCU/CERSS provided regular technical and implementation support to the RHS authorities, while also being proactive in identifying solutions to address implementation bottlenecks. The Project also supported the development of the RHS’s MRI analysis capacity. Close support and monitoring was given to region VII once it was included in the Project. 29. Though the RBF mechanism implied a steep learning curve and encountered a number of obstacles, implementation was successful and resulted in a change in the incentive framework and culture. The achievement of the MRI framework targets and the expansion of the model to other regions reflect a successful component design and implementation. The structural changes supported through Project policies in the health- financing model transformed the incentive framework and as a result supported a cultural shift from the use of historical budgets to more systematic reporting and verification budget system, and raised the level of empowerment and motivation of health staff in the participating regions. 30. Project was implemented through two different units. The Project created a Project Implementation Unit (PIU) within the MOH’s 21 Directorate of Institutional Strategic Development (DDEI) that assumed responsibility for executing Components 2 and 4. The Executive Commission for Health Sector Reform (CERSS) remained the Project coordinating unit and executed Components 1 and 3. Both institutions are key institutions in the DR health sector. The MOH is the steward of the sector and responsible for public health (Salud Colectiva) services while CERSS has an official mandate of 21 Within the MOH, an Interdepartmental Technical Committee was created for Project implementation. The following units participated in the Project and were part of the Committee: Directorate of Strategic Institutional Development responsible for project management, Department of Quality Assurance, Vice Ministry of Public Health, Deputy Minister of Planning and Development, Directorate for the Strengthening of Regional Services. A Procurement Committee was also established with the participation of the MOH procurement and financial areas. 12 monitoring progress in health sector reform and coordinating the formulation and implementation of common strategies based on evidence for health sector institutions such as MOH and SENASA. Though at the time of Project preparation it was envisioned that by the end of PARSS2 the DDEI would assume responsibility for the execution of future projects, the CERSS remained a functioning and important entity within the health sector. Both entities managed to closely coordinate through institutional agreements (Project Oversight Committee and through the RBF component within the Results Based Financing Committee), supporting the successful implementation of the Project, but in particular of Component 1. Nevertheless, strategic and financial planning coordination could have been better; for example, costs could have been further streamlined by CERSS as noted in the MTR and also post-MTR SPN reviews. 31. Despite changes in MOH administration and institutional reorganization during Project life, the different executing units successfully supported Project implementation and PDO achievement. There were three ministers of health and two PIU coordinators during Project implementation, resulting in delays in implementation of the MOH lead components. In addition, Project implementation was also affected by changes in some key CERSS PCU staff including the Technical Coordinator, FM Unit Manager and Results-based Financing Component Coordinator. These changes led to some delays and affected the pace of Project implementation. Nevertheless, new staff were able to rapidly familiarize themselves and commit to the Project, supporting the achievement of all project objectives. 32. Despite delays in restructuring that slowed Project implementation, the Bank and Project teams were able to overcome bottlenecks and achieve the expected results. The need to address the shortage of funds caused by insufficient counterpart funds available for the Project was flagged during the December 2013 supervision mission22 and followed up through several implementation support missions. With the Project completion date fast approaching, The World Bank approved a Project restructuring to adjust the percentages of financing as requested by Government authorities and extend the closing date by four months to ensure Project completion and full disbursement of loan proceeds. 33. Concurrent technical and financial audits conducted by independent firms verified the eligibility of beneficiaries, achievement of targets and actual delivery of services. The audits were essential elements of the RBF mechanism. The concurrent external financial audit (Auditoría Concurrente Externa, ACE) reviewed individual files and conducted random reviews to verify that services were provided to eligible individuals, services delivered following Project guidelines and the accuracy of target achievement. The ACE23 verified that funds were disbursed in a timely fashion, used for Project purposes and according to Operation Manual stipulations. The provision of an independent external 22 The rating for counterpart funding was downgraded from Satisfactory to Moderately Satisfactory due to delays in the availability of Government counterpart funds after the December 2013 mission. By the end of May 2014, the counterpart gap amounted to DR$ 106,072,307 or approximately US$ 2,466,797. 23 Separate from the overall Project audit. 13 opinion led to an increase in accountability, the quality improvement of processes, and institutional capacity building, among other things. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 34. M&E Design. The monitoring results framework was simple, relevant, comprehensive, and generally measured the PDO well. The M&E framework included five PDO and 11 intermediate indicators. See Table 2 in Annex 2. Some indicators were utilized for both the Project evaluation and the results-based financing mechanism. The baseline data for the package of health service indicators was not established until 2011 due to delays in the development of key IS applications. M&E design included both internal oversight and external concurrent audits (conducted by an external firm). 35. A number of the baseline figures were quite low because they were generated from the clinical management system (CMS), which was rolled out in 2011. The use of the CMS required a behavioral change on the part of doctors who needed to follow health protocols and enter patient health information electronically during consultations. The clinical management system serves as a MOH health service quality monitoring tool and was the main source of data for a number of indicators, in particular for monitoring Component 1. These indicators not only measured coverage but also quality (adherence to protocols). In order to promote its use (and in turn generate more data) the MOH made use of the system mandatory for health staff while implementing incentives and supporting training, logistical and technical support for its use. 36. The results framework was revised in 2011, due to the expansion of the RBF mechanism to a third region (region VII) and need to align the targets of some indicators based on the target population expansion. The revision adjusted targets of the indicators measuring health services because for these indicators the numerator reflected the number of cases that were considered positive according to Project norms (for example number of pregnant women with risk evaluation before first week of pregnancy) while the denominator measured the universe of possible cases (all pregnant women between 1-15 weeks of pregnancy). The incorporation of a new region and its population increased the denominator rapidly requiring the revision of the targets for PDO indicators 1 and 3 and intermediate indicators 3 through 8. 37. The RBF mechanism monitoring results indicators, and their targets, were well selected, targeting a broad segment of the population and focusing on the country’s basic package of health services. Aside from the traditional indicators established by the MOH and usually used in health RBF mechanisms for the first level of care, the PARSS2 mechanism incorporated indicators of the new health agenda covering NCDs. Therefore, the MRI framework evaluated a broader segment of the population, targeting children, pregnant women and the older and/or more at risk population for NCDs. 14 38. M&E Implementation. The RBF mechanism utilized ten indicators against which the variable capitation amount was paid. The amount was calculated based on the achievement of annual targets set by the MOH for monitoring results of the prioritized interventions (Table 2). The indicators used in the RBF mechanism (MRI indicators) were also used as PDO and intermediate indicators. The MRIs measured service quality according to national protocols. The MOH set annual targets for each of the ten MRIs, however to ensure progressive target achievement, the MOH measured intermediate indicators 1, 2, 6, 7 and 9 semi-annually. Intermediate indicators #3, 4, 5, 8 and 10 were measured during the last semester of the year and had annual targets. Table 2: Monitoring Results Indicators No. Indicator 1 PDO 1 Percentage of pregnant women with risk evaluation completed before the 15th week of pregnancy 2 PDO 2 Percentage of children under 15 months with vaccination scheme completed according to national protocol 3 PDO 3 Percentage of individuals diagnosed with hypertension under treatment 4 II24 2 Percentage of post-partum women referred from hospital to their corresponding first level of care units with proper documentation included in their clinical files of health center. 5 II 3 Percentage of children over 14 months and under 24 months with vaccination scheme completed according to national protocol. 6 II 4 Percentage of children under 24 months with growth and development monitor controls according to national protocol 7 II 5 Percentage of individuals from target population screened for hypertension under treatment 8 II 6 Percentage of individuals from target population screened for diabetes according to national protocol 9 II 7 Percentage of individuals diagnosed with diabetes II under treatment 10 II 8 Percentage of individuals symptomatic of TB with results of sputum smears in clinical file corresponding health center 39. Utilization. All stakeholders, but in particular the RHSs, successfully utilized the monitoring and evaluation framework, particularly the MRI indicators which were the backbone of the RBF mechanism. The establishment of a monitoring system at the first level of attention in the targeted regions promoted a culture of evaluationM&E. The RBF mechanism incentivized the utilization of the M&E framework due to the regular indicator data reporting required and encouraged the buy in by the RHSs. The regions largely achieved most of the MRI targets as well as all the PDO and intermediate indicators. 24 II: Intermediate Indicator. 15 40. The framework provided a solid first experience with the RBF mechanism. The success of the mechanism resulted in the expansion of the model, including the MRI framework and operating manual, to other regions. As the mechanism is institutionalized and health sector stakeholders continue to gain related operational capacity, indicators and targets can be modified and made more complex. However, the country’s IS has to develop further if the results framework is to be made more flexible and challenging. 41. The M&E framework established for the Project, and in particular the MRI framework, was successfully utilized during implementation and subsequently duplicated for the IADB Project. The regular reporting (every four months to a year) on a number of indicators by the RHS bolstered utilization of the mechanism and built knowledge capacity. SENASA verified the completeness of the information, calculated the amounts to be transferred and sent information to the PCU for the transfer. 2.4 Safeguard and Fiduciary Compliance 42. Safeguards. The Project was classified as a Category C for Environmental Assessment, given that Project activities would not cause an adverse environmental impact and therefore did not trigger the environmental safeguard policy (OP/BP 4.01). Nevertheless, an environmental assessment was conducted due to the Bank’s proposed work with the counterpart to develop a strategic approach to address health sector biomedical waste management, as a measure of quality enhancement. The project was well-placed to provide technical assistance to support improved waste practices. 43. The implementation of environmental safeguard activities was satisfactory. The Project supported technical assistance and training activities to handle the differentiated management of biomedical and infectious waste generated in Santo Domingo health centers. This work was used to extend the proper waste management to the national level in accordance with Regulation 126-09 (regulation on health center generated waste). 44. Fiduciary. The Project had a moderately satisfactory procurement rating for most part of the Project’s life, suggesting that Project activities were carried out in accordance with the World Bank’s procurement guidelines. Procurement was rated satisfactory during the Project’s initial stage due to the intensive focus the PCU at CERSS and PIU at MOH placed in this area. Procurement plans were submitted following Project effectiveness, procurement specialists with experience in Bank operations were hired in both units and in general the Project showed good procurement execution during the first months of Project implementation. The satisfactory rating was maintained until the Project began experiencing procurement delays due to problems encountered mainly with terms of reference, and technical specifications and some shortcomings in certain evaluation processes. At that time a moderately satisfactory rating was assigned given that the last ex- post Project review confirmed that MOH processes were transparent and followed Bank’s policies with minor deficiencies. Nevertheless, moderate deficiencies were found in CERSS hindering the timely and efficient achievement of important products and delivery of services. No procurement assessment under supervision was completed. 16 45. Financial Management (FM). The Project had a moderately satisfactory FM rating in the last ISR despite the fact that key FM areas were considered either Satisfactory or Moderately Satisfactory. The areas of internal controls and flow of funds were rated MU due to insufficient allocation of Project counterpart financing. Both the PCU at CERSS and PIU at MOH/DDEI processed commitments corresponding to RHSs, other providers/consultants for consulting services and operational costs using Bank financing from the designated accounts, with the expectation of replacing these funds once counterpart funds were received. All these transactions were posted in the Project records as accounts payable to the Bank. The lack of sufficient counterpart contributions caused implementation delays in Components 2, 3 and 4 and became critical as the implementing agencies continued facing funding issues until June 2015, i.e. shortly before Project closing date (at the time November 2015). This situation resulted in the Bank undertaking a detailed review of project expenditures and commitments. Based on the results of the review, the Bank increased, through a Project restructuring its share of Project financing and correspondingly lowered the share of counterpart financing at the GODR’s request. In addition, after client consultation, the Bank reserved the right to prior-approve Statement of Expenses (SOEs) corresponding to the costs to be financed from both the PCU and PIU when processing disbursement with the new co-financing levels and the terms of reference of the final Project audit were adjusted to incorporate a 100 percent review of the SOEs submitted during the audited period. Audit reports were submitted by the Borrower on time and were in general unqualified. 46. FM was a key function for the success of the Project, in particular for Component 1 since the RBF mechanism represented not only a new way of financial accountability in the sector but required very intensive and complex inter- institutional coordination as well as rigorous supervision and training. A financial concurrent audit25 verified that Component 1 funds were disbursed in a timely fashion, used for Project purposes and according to Operation Manual stipulations. These mechanisms helped prevent fraud, hold fund recipients accountable, and ensure that funds are used according to program’s guidelines. 2.5 Post-completion Operation/Next Phase 47. A third and final phase of the APL may be prepared as a follow up operation to consolidate PARSS1 and 2 achievements. The GODR expanded the experience of PARSS2 to two new projects financed by the Inter-American Development Bank (IADB) which are the RBF model. 25 Separate from the overall Project’s audit. 17 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 48. The relevance of objectives is High. The objectives were consistent with: (i) the country’s current development priorities; and (ii) current World Bank, country and sector assistance strategies and corporate goals. The overall strategic goal of the country partnership strategy (CPS FY15-18 Report #89551-DO) is to support the Government’s efforts to sustain growth and make it more inclusive, including (i) promoting equitable, efficient, transparent and sustainable management of public resources, and (ii) strengthening social service delivery, among other objectives. The DR’s National Development Strategy or Vision 2030 includes a pillar focusing on social development (striving to guarantee health to the whole population and promote poverty reduction and social and territorial inequality). Likewise, the Government’s 2013-2016 plan for the public sector focuses on institutional development with ambitious medium-term goals in the areas of social development and productive development (expanding access to public health insurance and upgrading the quality of health services). 49. The relevance of Project design and implementation is high as planned activities were consistent with the Project objectives. Project activities supported the achievement of the stated objectives, while the expansion of the RBF mechanism to other regions during Project implementation (initially to region VII during implementation and to Regions I-V through an IADB project) further confirmed design relevance.26 The IADB project is using the Bank’s approach and operating manual. SENASA is now examining the possibility of expanding the mechanism nationally. 50. While health insurance coverage was expanded with the rollout of contributory and subsidized regimes, further efforts are needed to attain universal coverage. In order to achieve the country’s goal of universal health coverage, there is a need to further strengthen the capacity of SENASA to maintain and expand health insurance coverage especially to the poor and the non-formal sector. Nevertheless, the gradual capture and documentation of the population through the expansion of the SR supported by the Project has a number of benefits in addition to those related to health coverage. 51. The Project supported the health sector reform agenda and strengthened the MOH’s stewardship capacity. Project activities reinforced the SENASA’s institutional capacity, transforming its business model and strengthening its role as strategic purchaser of health services. The Project improved the PROMESE/CAL supply chain management model. The Project provided training, logistics (vehicles to transport/distribute medicines) and IS support which contributed to the availability and quality of medicines and supplies in the participating RHSs. 26 Implementation in region 0 (Santo Domingo) is still on hold. 18 3.2 Achievement of Project Development Objectives 52. Four out of the five PDO indicators were achieved at the time of the first Project restructuring and all of the PDO indicators at Project completion (Table 3). Rating the PDO indicators in two phases, with the cut off according to the June 2014 restructuring, generates a rating of moderately satisfactory (Phase 1) and highly satisfactory (Phase 2) for PDO 1 and highly satisfactory for both phases for PDO 2. Ten of the intermediate indicators achieved or surpassed their target at the time of the restructuring and all were achieved or surpassed at Project completion. The PDO 1 - improve the capacity of RHS to deliver, in a timely fashion, quality services known to improve the health of mothers, children and people with chronic conditions by public providers at the first level of care, is measured by PDO indicators 1, 2 and 3 and eight intermediate indicators. The PDO 2 - improve health system responsiveness, defined here as the institutional capacity of public sector health organizations to conduct strategic purchasing of health care services and goods, and to respond to public health emergencies is measured through PDO indicators 4 and 5 and intermediate indicators 9, 10 and 11. Table 3. PDO and Intermediate Indicator Achievement 53. PDO 1 contributed to the expansion of ambulatory primary health care service delivery to poor individuals in regions VI and VIII initially and then also region VII. 19 The PDO and intermediate indicators evaluating PDO 1 were related to a set of basic ambulatory care services covering a significant portion of the population in these areas. While demand side interventions are important to further increase prenatal consultations and institutional births, available evidence suggest that poor practice standards largely explain why weak outcomes arise in spite of a very high ratio of births attended by medical professionals.27 54. The three PDO indicators (1, 2 and 3) supporting PDO 1 demonstrated significant improvement in the capacity of the RHSs to deliver quality services and in capturing the health information of the targeted population. PDO indicator 1, percentage of pregnant women from target population with risk evaluation completed before the 15th week of pregnancy was achieved and surpassed (50 percent from a base line of 0.43 percent) with the original and revised target. Greater control at the onset of pregnancy can have a positive impact on maternal mortality. The indicator increased from a baseline of .43 percent, likely low due to lack of data. PDO 2, percentage of children from target population under 15 months with vaccination scheme completed according to national protocols, was surpassed (68 percent from a baseline of 0). PDO 3 percentage of individuals from target population diagnosed with hypertension under treatment according to national protocols was partially achieved (38 percent from a base line of 3.5 percent) with the original target and surpassed at 110 percent with the revised target. The fact that these indicators were largely achieved with the original targets despite an increase in the target population of over 180,000 from region VII reflects the success of the Project. Service access was also strengthened through the implementation of an information, education and communication campaign regarding the rights of the affiliated. 55. The achievement of the PDO indicators not only supported the PDO 1 but also expanded health insurance coverage and revealed the successful establishment and implementation of the RBF mechanism in the three regions targeted by the Project. Project affiliation of individuals lacking health insurance (and then enrollment in SENASA) provided a more institutional channel for the RHS’s to deliver quality health services. 28 The Project affiliated 366,236 individuals throughout the Project cycle.29 Project efforts to identify poor persons with no insurance coverage assisted SENASA in expanding its SR coverage with persons initially covered by the Project being eventually covered by the SENASA, saving SENASA’s time and funds to find the uninsured persons by looking at the Project’s list of affiliates. Ninety-one percent of those requesting services in the RHSs were either SENAS’s affiliates or those with no insurance coverage, and most were Dominicans with limited or no education. 27 Dominican Republic CPS 2015-2018 28 Formal assignment differs from SENASA enrollment. For the Project, the RHSs registered individual data in the family health file along with the certification of poverty status in order to formally assign an individual. 29 Of the participating regions, Region VIII had 179,487 participants, region VI 116, 475 and the region that joined last, region VII, had 70,274 participants. 20 56. In addition to improving quality services for those in the regions, the mechanism improved the financial efficiency of the MOH budget and boosted resource flows earmarked to SENASA’s subsidized regime. Allocating funds based on performance and not historic (with some allowance for inflation) budgetary trends supported the improvement of both supply- and demand- side health system performance. This was ground-breaking, given that most resource flows are still governed by a historic budget, via inputs (including human resources), transferred by the MOH to RHSs. Furthermore, performance improved budgeting and the additional funds generated from Project affiliation boosted service provision at the first level of care.30 57. The financial incentives imbedded in the RBF mechanism promoted an improvement in data collection capacity and delivery of services. The improvement in data collection supported the regions ability to obtain information regarding health service users in their catchment areas. More knowledge regarding users improved the capacity of RHSs to deliver, in a timely fashion, quality services known to improve the health of mothers, children and people with chronic conditions by public providers at the first level of care. Data collection was boosted by the proper information documenting in clinical files (Intermediate Indicator 1) and the use of the Clinical Management System. 58. Improvement of health system responsiveness, defined here as the institutional capacity of public sector health organizations to conduct strategic purchasing of health care services and goods, and to respond to public health emergencies (PDO 2) was largely met through PDO indicators 4 and 5 and intermediate indicators 9, 10 and 11. The Project strengthened the MOH, SENASA and the RHS’s health system responsiveness through data collection and analysis regarding risk factors and burden of disease, and improved the MOH’s response to public health emergencies.31 59. The Project successfully strengthened the MOH’s stewardship capacity. The Project improved transparency and accountability in the delivery of public health services and goods, while strengthening sector related IS and developing new M&I instruments. The improvements were largely due to the establishment and implementation of the RBF mechanism as well as other key activities. Among these were the updating of norms for the operation and accreditation of national health system facilities and services; preparation of a draft law for the separation of functions and decentralization and establishment of the managing body of the RHS; discussion and dissemination of the Special Sanitary Career Law (Carrera Sanitaria Especial); development of primary health care strategy model of care; technical support for the development of health planning and the development of the methodology for depicting of the country’s Health Map; development and update of an inventory of the sector institutional regulatory legacy. 30 The Project succeeded in narrowing the financing gap of the regional health providers. The regional health providers have to provide services to all of the individuals that show up at the health centers whether they have health insurance or not. However, they only receive a per capita transfer from SENASA for the individuals that are affiliated with the subsidized health insurance. 31 Finally, the Project has supported the country's capacity to respond to the Ebola threat by financing key equipment and goods and training to facilitate identification and management of possible Ebola cases. 21 60. The Project contributed to improving the MOH's institutional capacity through studies, assessments, and by strengthening the functioning and inter-operability of health information modules (PDO indicator 4). The financing and preparation of a number of studies and surveys provide a wealth of information that if properly utilized can assist with decision and policymaking. Both PDO indicator 4 and intermediate indicator 10 measured information collection and dissemination. Despite the implementation and dissemination of the studies, the adequate utilization may prove more of a weakness in the DR. Finally, the Project also implemented an FM information system at the participating regional health facilities, thus strengthening their managerial capacity, contributing to improving service delivery. 61. Implementation of the RBF mechanism strengthened the RHS’s ability to collect and over time analyze health information. Improvements in the RHS’s data collection capacity are likely the result of better registration or an improvement in the following of individuals in the regions. The introduction of the mechanism and the regional health centers appreciation of the benefits of the framework provided the basis to expand RBF nationally. In order to strengthen the IS, intermediate indicator 9, supported the establishment of an office of national M&E systems and improvement of the existing instrument of the MOH’s HMMIS, which specifically focused on the IS, measured the successful adoption of an automated/electronic module for M&I with all the information needed for monitoring results on a quarterly basis at the first level of care with data available for the RHSs, the MOH central level and the SENASA. 62. The Project strengthened the institutional capacity of the sector and the health insurance system. The fortification of the IS for the management of medicines and medical inputs at the first level of attention improved the health system’s responsiveness to conduct strategic purchasing of health care services and goods (PDO 2). PDO Indicator 5 was achieved with regions VI and VIII producing monthly reports on the number of poor individuals who were prescribed a medicine at the first level of care, including a dashboard with information extracted from a database generated by SIMI. The Project succeeded in strengthening the institutional capacity of SENASA with the objective of transforming its business model and strengthening its role as a purchaser of strategic health services. The investment in software, equipment technical assistance and non-consulting services and training for better planning and more efficient distribution of medicines and other inputs purchased by the PROMESE/CAL supported this process. PROMESE/CAL enhanced the management capacity of the sector to improve the availability and affordability of medicines as reflected in the achievement of intermediate indicator 11, the percentage of accredited health centers from participating RHSs that don't have a PROMESE/CAL pharmacy and receive on a regular basis medicines procured and distributed by PROMESE/CAL. The Government has expressed interest in expanding the IS for Medicines and Medical Inputs supported by the Project in two additional regions, if funding becomes available. 22 63. SENASA’s technical and operational capabilities were significantly strengthened through Project activities supporting PDO 2 (improve health system responsiveness). The SENASA’s institutional capacity was boosted by Project financed technical assistance, consulting services, and training. A number of the activities implemented built up the SENASA’s actuarial analysis and risk management capacities, which will likely be further strengthened by the implementation of the consultancy recommendations. The SENASA’s operational capabilities were also boosted through the hiring of three consultants to support information processing regarding the RBF mechanism and technological support for the monitoring of indicators for the financing system. The component also supported the build-up of human resources capacity regarding the purchase of health services via specialized courses in terms of quality, medical auditing and international experiences. 3.3 Efficiency 64. The rating for Efficiency is Substantial based on the economic analysis. The Project contributed to improving the results-orientation of spending in primary health care interventions, which are cost-effective in nature such as pregnancy risk management, immunizations, and monitoring growth and development of young children-- all of which tend to focus on the first 1,000 days of life, which are critical for human capital development. 65. The focus on preventive measures and the shift away from more expensive health care treatment services improved the sector’s cost efficiency. In addition, taking into account the double burden of disease in the country and the increasing threat of NCDs, the Project contributed to shifting the agenda and strengthening the coverage and quality of interventions toward selected NCDs. Under the Project, an inter-institutional committee comprised of the MOH, CERSS, SENASA, Social Cabinet and with participation of regional health authorities was established to maximize synergies and ensure the coordination of activities under the RBF component and that lessons learned were shared, especially among participating regions. Project identification of those lacking insurance coverage assisted SENASA in expanding its subsidized health insurance coverage by providing the necessary information to enroll these individuals. 66. The cost benefit analysis prepared for the PAD revealed a positive net present value from the assumed reduction in maternal and infant mortality and disability reduction. Though the rates may overestimate reductions since any changes are multi- causal and therefore difficult to attribute to the Project, the Project has contributed to an improvement in health care and health care efficiency. 67. A comparative quantitative analysis completed during Project implementation provided some useful observations between the regions with and without Project implementation. The comparison indicated a greater number of consultations in the regions of Project implementation, particularly in the case of pregnant mothers, but less so in NCD screening. 23 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 68. The overall outcome rating is Satisfactory. Relevance of objectives, design and implementation is High. Efficacy is rated Substantial with the achievement of PDO 1 rated Substantial and PDO 2 rated High, while efficiency is rated Substantial. The Project accompanied the country’s ongoing health sector reform and had a positive and notable impact on health sector functioning. The Project successfully established the RBF mechanism in the health sector improving service provision and financial efficiency in the regions of Project implementation (Table 4). Table 4. Overall Outcome Rating Phase 1 Phase 2 Total Relevance High High Objective High High Design High High Implementation Substantial Substantial Efficacy Substantial Substantial (1)PDO 1 Substantial High (2)PDO 2 High High Efficiency Substantial Substantial Overall Outcome Satisfactory Satisfactory Rating Value 4 4 % of Loan Disbursed 60% 40% Total 2.4 1.6 4 = S 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 69. The Project had a positive social impact in the country by increasing health care access for the poor and improving the mechanisms to target and deliver health care. The Project focused on inequality reduction in health care service access and in narrowing health status disparities amongst population groups and between urban and rural regions. By targeting those in the poorer regions where infant and maternal mortality are higher, the national mortality rates should overtime exhibit a decreasing trend. 70. Component 1 succeeded in expanding the local health staff’s knowledge regarding their catchment areas. In an attempt to improve the capacity of RHS’ to deliver quality services known to improve the health of mothers, children and people with chronic conditions by public providers at the first level of care (PDO 1) and affiliate these with the SR, health service staff learned about the individuals living in their regions and their health conditions. 24 (b) Institutional Change/ Strengthening 71. The Project supported the build-up of the health sector’s institutional capacity, in particular the MOH, SENASA and the RHS’s. More information regarding the institutional capacity strengthening is provided in other sections of the ICR. However, the institutional strengthening of SENASA, a recently established institution, was very relevant and particularly impacted by the Project’s RBF mechanism. Though the mechanism was implemented and managed by the CERSS implementing unit, the SENASA built significant institutional capacity regarding the framework. As a result of its exposure to the mechanism and a consultancy regarding the RBF mechanism (Jaramillo et. al) financed by the Project, SENASA is now planning on institutionalizing the RBF mechanism nationally. (c) Other Unintended Outcomes and Impacts (positive or negative) 72. The RHS’ expanded fiscal envelope resulting from the capitation payments for the new affiliates encouraged regions to be more proactive in identifying and registering individuals. This supported innovative measures by the regional centers to capture possible affiliates and track the population targeted for the different MRIs (for example tracking pregnant women in order to complete a risk evaluation before the 15th week of pregnancy). 73. The Project financed the purchase and installation of solar panels for 10 regional pharmacies. The electricity generated has provided lighting for a number of the neighbors, including school age children, which lacked these services. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 74. The 2013 satisfaction survey of health service providers contracted by SENASA found a 75.5 percent satisfaction index with SENASA service components (medical services authorizations, audits, fees, payment and treatment.) The dimensions that impacted positively on overall satisfaction were the hiring processes and services provided through the electronic platform. While those dimensions that were negatively rated and therefore offer opportunities for improvement were the processes related to authorizations, billing and payment. When segmenting the results by type of providers, the survey showed a high level of satisfaction among health clinics (60.5 percent) high cost centers (79.2 percent) diagnostic centers (71.4 percent), clinical laboratories (71.4 percent), independent dentists (73.3 percent) and pharmacies (67.5 percent). However, the opportunity for improvement was most notable among independent doctors (48.0 percent). 25 4. Assessment of Risk to Development Outcome Rating: Modest 75. The RBF mechanism was successfully adopted, incorporated as a health sector tool and expanded as a pilot project to other regions. The benefits of the financing mechanism were recognized by the health sector and as a result replicated as a pilot. As discussed previously an IADB Project modeled after PARSS2 is currently being implemented as pilot in Regions I-V, while the expansion of the RBF mechanism to region 0 (Santo Domingo) is still waiting for Government approval. 76. SENASA is planning on managing and expanding the mechanism nationally (as a national program and not as an external project), which would be beneficial since currently the Bank PARSS2 regions are not included in the IADB project and therefore could begin to revert the progress made until PARSS3 approval. Nevertheless, the risk of implementation is significant since currently there is no firm timeframe for nationally rolling out the mechanism. Furthermore, though the Project regions adopted an indicator dashboard that includes many of the Project indicators as well as others, they are not receiving financing for the “results” making the current situation largely unsustainable. A national expansion managed by SENASA would institutionalize the system, while standardizing the different platforms and tools used between the Bank and IADB project. 77. Project policies supporting the health system reform process are likely to be sustained. The knowledge gained from Project activities, which supported financial transparency and efficiency, strengthened the health IS tools, improved budgeting and health service capacity while strengthening health infrastructure, should remain in place. 78. Resource constraints and inefficient spending could negatively impact the sustainability of some of the actions supported by this Project. The DR spends less than other countries on health, on average 2.6 percent of Gross Domestic Product 2009-2012.32 The RBF mechanism was specifically implemented to improve the quality of spending and expand social protection in health. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 79. Quality at Entry is rated as Satisfactory. Project design was in line with the GODR and World Bank strategies and introduced innovative approaches. The selection of PDOs correctly identified some of the health sector challenges. Project design was framed to work with the existing units and structures and took advantage of the maturing 32 World Bank Databank - http://data.worldbank.org 26 coordination processes between the MOH, SENASA and the regional health care networks. Given the ambitious, long-term vision for sectoral changes, the choice of an APL as a financing instrument was appropriate. The selection of the results framework indicators was suitable and effectively measured the achievement of Project objectives. 80. Project design had minor shortcomings related to the organizational arrangements, in particular the assignment of Project execution responsibilities to two fiduciary units. Nevertheless, the arrangement was examined carefully and justified during the PCN stage. In addition, the risk regarding counterpart funding issues, considered at the PCN stage was removed from the PAD, likely due to the positive counterpart funding experience during PARSS1. The Project was prepared and approved in a six-month period,33 as agreed upon with the Government. (b) Quality of Supervision Rating: Satisfactory 81. Quality of Supervision is rated as Satisfactory. The Project counted on a steady, experienced, technically diversified and committed Bank team, which generated a good dialogue with all the participating government units. In collaboration with counterparts, the World Bank task team tried to address bottlenecks, either technical or operational, throughout implementation. The team worked very closely and intensively with Project counterparts, engaged in numerous technical, operational support and supervision missions, video and audio interactions and prepared informative Aide Memoires and reports. The World Bank team also supervised and provided assistance in the implementation of technical and financial audits and commissioned an external review of Project staffing and other operational aspects including internal controls and costs with the goal of providing a detailed report with concrete recommendations to improve quality and efficiency of project management and coordination, as well as project management expenditures. Local fiduciary Bank staff in collaboration with counterparts made strong efforts in the resolution of implementation obstacles. The Bank proactively addressed the lack of counterpart funding bottlenecks since December 2013 and engaged in an intensive dialogue with the GODR to overcome the situation.34 The close partnership with counterparts contributed to the achievement of important results as described earlier. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 82. For the reasons indicated above, the Overall Bank Performance is rated Satisfactory. 33 The Project Concept Note Review was on March 3, 2009, appraisal on June 16-19 June, 2009, negotiation on July 31, 2009 and approval on September 17, 2009, while the Loan Agreement signed on October 30, 2009 and Project approval ratified by Congress on December 15, 2009. 34 In retrospect one restructuring addressing counterpart financing and extending the closing date would have been more efficient. 27 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 83. The GODR embarked on an ambitious and comprehensive reform process, which aimed to improve the health status and quality of life of the Dominican population. Strong Project commitment was reflected in the Government’s endorsement of all Project activities. The Government took the RBF strategy as the basis for transforming and strengthening primary care. The Government program and MOH strategic agenda confirmed the country's commitment to achieving the Millennium Development Goals (MDGs), in particular prioritizing the reduction of maternal mortality and improving maternal and child health. The Government sought synergies among the poverty reduction policies of various agencies fostering coordination between public institutions and other stakeholders. 84. The Government’s performance is rated Moderately Satisfactory due to implementation bottlenecks and delays caused by insufficient counterpart funding. Due to a deteriorating fiscal situation the government reduced counterpart funds from 30 to 17 percent for Category 2 and from 30 to 15 for Category 3. As noted in section 1.7, a third Level II restructuring, finalized on February 3, 2016 allowed the Bank to increase its financing share of selected expenditure categories to offset the lacking counterpart funds. Delays in addressing the counterpart funds issue and officially seeking an increase in the WB’s share of financing contributed to implementation delays. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 85. There were two implementing units responsible for Project implementation: the PCU at CERSS (in charge of overall Project coordination) and the PIU at MOH’s DDEI. 86. Project management commitment and ownership as well as intensive follow up regarding project activities contributed to Project success. Very effective coordination between the MOH, SENASA, CERSS and the RHAs was a key element for the success of the Project. During implementation, CERSS increased by 300,000 affiliates due to the existing framework and the proactivity of the PCU in providing RHS support. The PCU and PIU carried out frequent site visits to the RHS and organized regular review and implementation workshops with them. These workshops promoted exchanges of information and lessons learned among participating regions. The agenda of studies and surveys prepared and managed by both the PCU and the PIU was ambitious and revealed strong commitment to the achievement of project objectives as well as solid technical capacity. In general, the Project demonstrated adequate implementation, supporting the achievement of the project component activities. 28 87. There were also minor shortcomings in both Project management units. For example, there were some delays in technical and audit reports delivery under the management of CERSS and delivery of some consultancies managed by DDEI, which were resolved by extending the Project’s closing date. With regard to FM, the Mid-term Review found that while the PIU had been conservative in managing operating costs, the PCU could have taken stronger measures to contain operating costs, particularly in terms of the number of staff and vehicles used.35 There were a number of measures introduced to reduce these costs after the Bank’s detailed expenditure review.36 88. As a result of the counterpart fund delays, both the PCU and the PIU processed commitments temporarily using Bank financing from the designated accounts, with the expectation of replacing these funds as soon as they received the promised counterpart funds. All these transactions were posted in the Project records as accounts payable to the Bank. However, both units made an effort to follow up with financial authorities to try to close the gap regarding insufficient counterpart funding. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 89. For the reasons outlined above, the overall Borrower Performance is rated Moderately Satisfactory 6. Lessons Learned 1. Adaptable Program Loans: a) Can be powerful instruments for the consolidation of complex reform processes. The institutional capacity developed at the national level and in the regions through PARSS1 as well as the consensus building fostered among MOH, SENASA, CERSS, MOEPD, PROMESE/CAL were critical for the implementation of the RBF mechanism. The ability to adjust Project objectives and implementation activities in between APLs can serve to institutionalize capacity, strengthen public spending quality and improve health care accountability. 2. Result based mechanisms (RBF): b) Constitute powerful tools for improving service quality and in turn the health of people. By establishing clearly defined expected results, collecting information to assess progress on a regular basis, and taking timely corrective action, health 35 The Bank’s expenditure review found that expenditures in the amount of US$ 1.1 million were not eligible for Project financing. 36 The Bank expenditure review undertaken in June and July 2015 indicated that while the PCU undertook measures to reduce costs after the MTR, it could have further reduced its staffing costs especially during the last year of implementation. After the 2015 expenditure review, the PCU took steps to optimize its number of staff. It also relocated to share a building with the National Health Service in order to reduce office rental costs. 29 staff can improve the management of their projects or investments and, in turn, achieve a sustained improvement in the lives of the target population. c) Can enhance management capacity at the regional levels contributing to decentralization. The implementation of the RBF financing scheme in the DR supported the building up of skills of regional staff. Furthermore, the financial flows to the project RHS promoted financial monitoring and accountability. d) Could have a greater impact if staff involved were also provided with an incentive. Projects can institute internal incentives for staff involved in the RBF mechanism implementation. Incentives do not have to be necessarily economic; they could include scholarships, trips to see similar experiences, the provision of a PC or laptop, etc. e) Can generate initial resistance, which is only resolved once benefits are perceived. The project demonstrated that there was an initial opposition to RBF because it implied increased staff responsibility. The resistance diminished as achievements incentivized the process and the mechanism was incorporated. f) Generate local creativity in finding solutions to problems faced in the process. A notable dynamism arose with the RHS’s as they sought to increase the uptake of project resources. Regional centers generated creative and timely solutions to some problems that arose during project implementation. An example of this was the development and implementation of a computer application for the affiliation process called User Base developed by SRS VIII. g) Information systems play an essential role in RBF mechanisms. Often, local ISs need to be strengthened in order to support an RBF framework. Because results are verified via a defined set of indicators, the existence of clinical and management ISs, particularly databases reliable and updated, constitutes a central aspect of RBF. 3. When possible, projects should be implemented via one implementing agency but if the country context justifies the use of more than one agency then more attention would need to be given to improving coordination and containing operational costs. Though the selection of two implementing units made sense during Project preparation, the structure resulted in the duplication of staff and resources. Furthermore, shared Project implementation, while desirable to develop capacities in various institutions, does not always guarantee the proper articulation of the various components. If possible, effective coordination mechanisms should be promoted to ensure that the integration is achieved. Nonetheless, if the country context justifies the use of one implementing agency, then more attention should be given to ensuring (a) close coordination between implementing agencies and (b) that staffing and operational costs are sufficient to support Project activities while also being carefully contained so they could be sustained by domestic resources after the Project ends. 4. When implementing a RBF framework, indicators and targets should conform to the system's capabilities. In choosing indicators and targets the clinical and administrative conditions of the health system must be taken into account. As there are usually initial weaknesses in the systems, goals should be set conservatively so that stakeholders obtain progressive improvements in order to incentivize those 30 involved. Once the system has been established and the mechanism internalized the targets and the framework can be made more complex and dynamic. 5. When possible preference should be given to independent institutions with local context knowledge for undertaking external technical audits. While international firms are perceived to be likely more independent, they tend to be less knowledgeable of local policies and significantly more expensive due to travel and per diem expenses. 6. Enhancing and sustaining local capacity contributed to Project success. The Project focused on the development of local capacity through intense training. During Project implementation, MOH authorities tried not to replace staff at the central and regional level. These two aspects greatly contributed to the Project success. 7. National counterpart contributions within a Project should be carefully evaluated. This Project demonstrated that counterpart contribution requirements should be significantly assessed in external financing operations, especially when a country’s economic circumstances result in uncertain budget management. 8. Prioritization of the use of existing institutions and organizational arrangements to avoid implementation delays and ensure project sustainability. To a large extent, the project made use of established processes and systems followed by the MOH, SENASA, and RHS. It also supported the adoption of the Clinical Management System as the main source of information for Component 1 when it was rolled out by the Social Cabinet and adopted by the MOH to monitor the quality of care in primary level facilities. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 90. The report properly states the development of Project activities and achievements. It makes clear that the experiences and progress made through PARSS2 constitute a significant step towards transforming the delivery of health services at primary care level in quality services, particularly through the RBF strategy (Component 1); the strengthening of institutions involved in the process (Component III); and reinforcing the capacity of the MOH as the steward of the health system (Component II). From the above, the desirability of continuing the third phase of the program is clear. This will ensure that the achievements in the three regions can be sustained and consolidated and that these effects can be transferred to other areas of social policy. Despite the fact that there was a substantial rotation of staff responsible for implementing the Project as well as irregularity in the flow of counterpart funds, the Project showed continuity and overall satisfactory performance achieving the proposed objectives. In this context, it is worth noting the work and close collaboration between the Government and the World Bank during critical periods of execution. 31 (b) Cofinanciers N/A (c) Other partners and stakeholders N/A 32 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Actual/Latest Percentage of Components Estimate (USD Estimate (USD Appraisal millions) millions) Component 1:Introducing results-based financing mechanism for the first level 21.49 8.25 38.38% of care in Regional Health Services Component 2: Strengthening the capacity of the MOH to improve and monitor health system responsiveness 10.00 7.09 70.9% while fostering transparency and accountability Component 3: Improving the quality of public spending on health care goods 12.57 18.17 144.55% and services Component 4: Support for Response to 00.10 0.07 70.00% Public Health Emergencies Total Baseline Cost 44.16 33.58 76.04% Physical Contingencies 0 0 Price Contingencies 0 0 Total Project Costs 44.16 33.58 76.04% Front-end fee IBRD .08 .08 100 Total Financing Required 44.24 33.66 76.08 (b) Financing Appraisal Actual/Late Type of Estimate st Estimate Percentage Source of Funds Cofinancing (USD (USD of Appraisal millions) millions) Borrower 13.74 4.10 29.83% International Bank for 30.5 29.56 96.91% Reconstruction and Development Annex 2. Outputs by Component 1. The project had four components. The detailed results for each component are included below and the Results Monitoring Framework at the end of the Annex. Component 1 - Introducing results-based financing mechanism for the first level of care in Regional Health Services (US$21.49 of which US$14.52 bank financing). 2. The objective of the component was to introduce a RBF mechanism to the first level of health care in the Dominican Republic. The establishment of the first component in particular served to support PDO 1 (to improve the capacity of RHS to deliver, in a 33 timely fashion, quality services known to improve the health of mothers, children and people with chronic conditions by public providers at the first level of care) for the regions of Project implementation. The component was evaluated via 3 PDO indicators (percentage of pregnant women from target population with risk evaluation completed before the 15th week of pregnancy, percentage of children from target population under 15 months with vaccination scheme completed according to national protocols and percentage of individuals from target population diagnosed with hypertension under treatment according to national protocol) and 8 intermediate indicators. By Project completion all the indicators were successfully achieved. 3. The Project supported the affiliation of individuals lacking health insurance. The Project affiliated and assigned a Family Heath File (Ficha Familiar de Salud - FHF) to around 366,236 individuals throughout the life of the Project. 37 (Table 1). Formal assignment was achieved through the RHS’s collection of data via personal interviews and by completing the FHF, an instrument used by the MOH for health related data collection. Project affiliation (not SENASA enrollment) provided additional funds for the RHS’s, as individuals affiliated were those demanding services from the health centers but without a per capita transfer from the SENASA. The RHS were able to use the extra funds to improve service provision. In 2014 the SENASA suffered a financial crisis, which implicated reduced payments for the RS, which increased the regions interest in the Project. The information compiled in the FHF then served to facilitate enrollment in SENASA, institutionalizing these individuals health insurance status. Table 1. Population covered by Component 1 Month/Year Region VI VIII VII Total Dec 2011 19,762 32,181 51,853 Dec 2012 22,941 30.098 53,039 Dec 2013 45,024 61,697 23,966 130,687 Jul 2014 40,642 55,047 29,102 124.794 Source: CERSS 4. The Project successfully established and implemented the RBF mechanism in the three regions targeted by the Project (VI, VII and VIII). In addition to improving quality services for those in the regions, the mechanism improved the financial efficiency of the MOH budget and the flows earmarked to SENASA’s SR. The financial incentives imbedded in the RBF mechanism also promoted an improvement in data collection capacity. This is demonstrated by intermediate indicator 1, the percentage of clinical files with complete information for external audit to verify RHS reported MRI result, which surpassed the target and had reached 90 percent by Project completion. The MRI linked to the RBF mechanism were also increasingly achieved as experience with the mechanism expanded. (See graph 1). 37 Of the participating regions, Region VIII had 179,487 participants, region VI 116, 475 and the region which joined last, region VII, had 70,274 participants. 34 5. The use of the Clinical Management System CMS (Sistema de Gestión Clínica) expanded in the targeted regions improving the country’s ability to track users and generate health information. The Project provided technical and logistical support to the regions to ensure that the doctors systematically registered the required information in order to facilitate indictor monitoring. In 2014, 74 percent of Clinical Management Consultations took place in the Project areas. The proportion of consultations as a percent of the total consultations fell as the IADB Project began implementation in 2014. In 2015 there were 2,079,227 registered consultations in all of the RHS centers of which 956,182 (46 percent) were in PARSS2 regions. Ninety-one percent of those requesting services in the SRSP were either SENASA affiliates or those with no insurance coverage, and most were Dominicans with limited or no education. 6. Capitation payments were made available to the RHS in two installments. The first installment was 50 percent of the total amount of the capitation transfer, paid on a monthly basis upon certification and validation by SENASA of a registry of eligible beneficiaries. The second installment for up to the remaining 50 percent was paid every four months calculated on the basis of how many of the ten MRI targets were achieved by the RHSs and upon verification by SENASA of the performance against the agreed upon MRI targets. Each indicator achieved was worth five percent of the overall capitation. 7. The achievement of the Component 1 PDO and intermediate indicators indicate that the RBF mechanism successfully expanded service provision and quality in the Project regions. The indicators used for the RBF mechanism (see Table 2 in the main section of the document), or MRI, covered a broad range of the population in the Project regions. The arrangements and services prioritized were based on those already existing for the financing and delivery of health care to the poorest groups of the population through the SR. Therefore, the incentives for the RHS and the SENASA were aligned to improve the quality of attention in the first level of care to the poor. Component 2: Strengthening the capacity of the MOH (Ministry of Health) to improve and monitor health system responsiveness while fostering transparency and accountability 35 8. The Component successfully strengthened the MOH’s stewardship capacity. The Project improved transparency and accountability in the delivery of public health services and goods, while strengthening the information systems and developing new Monitoring and Information (M&I) instruments. The improvements were largely the result of the establishment and implementation of the RBF mechanism and a significant number of technical assistance activities 38 aimed at developing and/or updating of standards, manuals, guides, regulations of different laws related to health and MOH technical documents; developing and strengthening the network of health services (Red Unica de Servicios de Salud) and annual operational planning and development of rules for the operation and accreditation of facilities and services of the National Health System and improving primary health care service. The Component was divided into two subcomponents and evaluated via two PDO indicators. The subcomponents aimed to: (a) 2.1 strengthen the strategic planning and evaluation capacity of the MOH to improve its stewardship to foster transparency and accountability with an emphasis on improving health system responsiveness and, (b) 2.2. Strengthen the monitoring, information and evaluation systems for results-based financing. The Component was evaluated via two PDO indicators (studies on risk factors and burden of disease has been conducted, disseminated and used by corresponding MOH units (Salud colectiva) to develop annual operational plans for health promotion and prevention and at least one region producing monthly reports on the number of poor individuals who were prescribed a medicine at the first level of care and actually received medication within 48 hours at the point of consultation service or in a facility of PROMESE/CA) and two intermediate indicators. 9. A strengthening of the information system was critical for the establishment of the RBF mechanism and for the development of the system. Subcomponent 2.1 supported the strengthening of mechanisms for information collection, which enhanced results and accountability at the primary care level. Intermediate indicator 9, which specifically focused on the IS, measured the successful adoption of an automated/electronic module for M&I with all the information needed for monitoring results on a quarterly basis at the first level of care with data available for the RHSs, the MOH central level and SENASA. In terms of the ISs, as part of the separation of functions, the clinical management system is in transition toward the National Health Service (Servicio Nacional de Salud, SNS). This shift is being accompanied by a consultancy financed by the IADB. 38 Through the component the MOH carried out approximately 17 consultancies, 26 workshops and 21 training courses which focused on a number of the issues listed in the PAD. These contributed to the strengthening of the stewardship role of the MOH, separation of the purchase and provider functions and the new MOH institutional structure. This component also financed purchase of equipment and materials for the strengthening of the development of information infrastructure and connectivity of the MOH 36 10. Through the implementation of the RBF mechanism the Component supported the RHS capacity to collect and over time analyze health information. Improvements in the RHS’s data collection capacity are likely the result of better registration and/or improved population tracking. Since the ISs needed strengthening and adaptation Subcomponent 2.2 supported a newly created office of national M&E systems and improved the existing instrument of the Health Management Monitoring and Information System (HMMIS) of the MOH. 11. The financing and preparation of a number of studies and surveys provide a wealth of information that if properly utilized can assist with decision and policymaking. Both PDO indicator 4 and intermediate indicator 10 measured information collection and dissemination. Studies on risk factors and burden of disease were conducted, disseminated and used by the corresponding MOH units (Salud Colectiva) to develop annual operations plans for health promotion and prevention (PDO indicator 4). Likewise, indicator 10, was achieved as a number of risk factor and burden of disease surveys were designed, implemented and the results disseminated. Despite the implementation and dissemination of the studies, the adequate utilization may prove more of a weakness in the DR. 12. The Component also supported the improvement of the MOH’s response to public health emergencies (Component 4). The Component trained staff in response to a possible Ebola outbreak (sent them to Colombia and Cuba), provided equipment, supplies and a surveillance vehicle. See Component 4. Component 3: Improving the quality of public spending on health care goods and services (US$12.57 million – Bank financing US$8.8 million) 13. The component supported an improvement in the institutional capacity of the sector and the health insurance system, with an emphasis on the RBF mechanism, improving public spending efficiency. This component had three subcomponents and was evaluated via one intermediate indicator, which was successfully achieved. The component improved the IS for the management of medicines and medical inputs in the first level of attention. There is now an established method for medical prescriptions in region VI and VIII. However, logistical issues related to the lack of electricity in some areas caused problems in the implementation of the dispensation model supported by the Project. In regions VI and VIII the Project was only able to install the dispensation model in 4 places, therefore the Project expanded the implementation of the prescription and dispensation model in region VII. The system, SIMI (Sistema de Información Médica Intrahospitalaria) generates prescription and dispensation reports. Subcomponent 3.1 Strengthening public sector planning, procurement and distribution of medicines and medical inputs. 14. The subcomponent financed investment in software, equipment technical assistance and non-consulting services and training for better planning and more efficient distribution of medicines and other inputs purchased by PROMESE/CAL. The principal 37 products generated include: (a) equipment for the PROMESE/CAL northern region warehouse, (b) conceptual design of the “Information System of Medicines and Inputs in the National Health System Framework of the Republic”, (c) implementation of IS and management model for a chain of medicine and input provision in the country, including an automatic electronic prescription system in regions VI, VII and VIII, and (d) implementation of SIMI, making the strategic purchase of medicines and inputs for the public sector more efficient. Subcomponent 3.2 Strengthening institutional capacity to expand and enhance social protection in health for the poor. 15. This subcomponent financed technical assistance, consulting services, and training in order to strengthen the institutional capacity of the SENASA and other government ministries interested in RBF mechanism. The Component also supported the expansion of the SR and generated the following outputs: (a) development of a tariff system and mechanism for the hiring of service providers of SENASA, (b) conceptual design and development of SENASA information risk management, (c) three consultancies for the complementary strategies of IEC to affiliates in order to improve access and healthy habits; (d) the preparation and printing of SR affiliate cards in the regions covered by the Project; (e) support documentation for the strengthening of the National Health Service IS; (f) three consultancies tied to the actualization of costs for the SR, which provided information regarding the financial sustainability of the SR. 16. SENASA’s technical and operational capabilities were significantly strengthened through Project activities supporting PDO 2 (improve health system responsiveness). A number of the activities implemented through this component built up the actuarial analysis and risk management capacities of SENASA, which will likely be further strengthened by the implementation of the consultancy recommendations. The operational capabilities of SENASA were also boosted through the hiring of three consultants to support information processing regarding the RBF mechanism and technological support for the monitoring of indicators for the financing system. The component also supported the build-up of human resources capacity regarding the purchase of health services via specialized courses in terms of quality, medical auditing and international experiences. 38 Component 4: Support for Response to Public Health Emergencies 17. The Component was introduced in order to provide a quick disbursing financial mechanism in response to a possible public health emergency. In 2014 the MSP requested the disbursement of health emergency earmarked funds following the global alert issued by the World Health Organization regarding the Ebola epidemic. The funds were used to strengthen the capacity of the MSP to face the threat of the Ebola virus in the DR. This included improvements in the detection, investigation and treatment of possible cases. The component also financed the purchase of a medical equipment, supplies, a vehicle and Ebola related training in Cuba and Colombia. Component activities revised early warning system procedures and took focused on sanitary issues. 39 Table 2. Results Monitoring Framework Revised Target PDO Indicators Baseline Original Target (following Actual 2016 restructuring 1) PDO I I. Percentage of pregnant women from target population with risk 0.43 50'/o 40% 50% evaluation completed before the I 5t h week of pregnancy % Met 100% 125% 2.Percentage of children from target population under 15 months 0 60% 68% with vaccination scheme completed according to national protocols 113 % 3. Percentage of individuals from target population diagnosed 3.54% 50% 35% 38% with hypertension under treatment according to national protocols %Met 74% 110% PDO2 4.Studies on risk factors and burden of disease has been Study results used to Studies conducted, conducted, disseminated and used by corresponding MOH units 0 develop annual operation plans disseminated and used (salud colectiva) to develop annual operational plans for health promotion and prevention % Met Achieved At least one region Regions VI and VIII are 5.At least one region is producing monthly reports on the number of producing annual producing reports, including a poor individuals who were prescribed a medicine at the first level of reports dashboard with Information care and actually received medication wi t h i n 48 hours at the point extracted from the database 0 genera ted by SIMI of consultation service or in a facility of PROM ESE/CAL % Met Achieved Intermediate indicators Component I I .Percentage of Clinical files with complete information for external 0 85% 90% audit 10 % Met 106% verify RHS reported MRI results 2.Percentage of post-partum women referred from hospital to their corresponding first level of care units with proper documentation 0.44% 50'/o 63% included in their clinical files of health center. % Met 126% 3.Percentage of children over 14 months and under 24 months 0 60% 45% 63% with vaccination scheme completed according 10 national protocol % Met /05% 140% 4.Percemagc of children under 24 months with growth and 0.27% 60% 50% 62% development monitor controls according 10 national protocol % Met 103% 124% S. Percentage of individuals from target population screened for 0.89% 60/'o 40% 64% hypertension according to national protocol % Met 107% 161% 6.Percentage of individuals from target population screened for 0.07% 40% 35% 43% diabetes according to national protocol % Met 108% 123% 0.79% 40% 30% 34% 7.Percentage of individuals diagnosed with diabetes type I I under treatment % Met 85% 114% 8.Percentage of individuals symptomatic of TB with results from 0 40% 35% 45% sputum smears in clinical file of corresponding health center % Met 113% 129% Component 2 9.The Health Management System of the MOH includes an Results-adjusted Model developed and used by automated/electronic module for M&l with all the information No module Capitation module used regions VI and RHS's needed for monitoring results on a quarterly basis at the first level of VIII. care with data available for RHSs, the MOH central level and the NHI. % Met Achieved Studies' results completed and Study realized and results I 0. Risk factors and burden of disease surveys designed implemented No Studies disseminated and disseminated used for planning and results disscmina1cd. public health interventions with an emphasis on health promotion and prevention. % Met Achieved Component 3 11 .Percentage of accredited health centers from participating RHSs that don't have a PROMESE/CAL pharmacy and receive on a regular 0 80% 1 00% basis medicines procured and distributed by PROMESE/CAL. % Met 125% 40 Annex 3. Economic and Financial Analysis 1. This annex presents the results of the economic and financial analysis for the Dominican Republic's Health Sector Reform APLII (PARSS2). The analysis is based upon a cost/benefit analysis conducted for the PAD and an experimental evaluation comparing the regions with and without Project implementation. The results of the cost- benefit analysis and the experimental approach indicate that, under reasonable parameters and expectations, the net present value of the project was positive. 2. The economic and financial analysis conducted during Project preparation for PARSS2 consists of: a) the computation of the net present value (NPV) of a chronological stream of quantified benefits and costs derived from the implementation of PARSS2, and b) sensitivity analysis of the project’s NPV using varied parameters, including discount rate and reduced rates of mortality and disability. 3. There are two major categories of quantifiable benefits from all the components of PARSS2 that were evaluated in the cost benefit analysis. The first category is the benefit derived from reduced deaths among pregnant women, puerperal mothers and infants. The second category is the decrease in deaths and disabilities from hypertension and diabetes. There is also the less quantifiable, however in this case perhaps more reliable benefit of increased access to health services evaluated in the experimental approach. 4. The cost/benefit analysis is based upon the expected future stream of income from labor during the lifetime of each individual directly affected by the project in order to quantify the benefit derived from avoiding a death or a life with disability. These calculations were performed using the current distribution of labor income for the age and gender groups that make up the targeted population (Poor People, Levels I and II in the Regions VI and VIII, according to the nomenclature of the DR's MOH). 5. The dataset used in the computations is the Demographic Survey of Health of the Dominican Republic (2007). For the sake of accuracy, all relevant variables and parameters were restricted to the subsample that corresponds with the targeted population, except where the size of that subsample implied very large sample errors for certain indicators or parameters. In those cases, the national sample for levels I and II of poverty will be used to impute values for the targeted population. 6. All monetary values were expressed in US dollars, and the exchange rate for converting present value Dominican pesos to US dollars was 36.00 Dominican pesos per US dollar. 7. The future flows of each category of quantified benefits were discounted at an appropriate rate to compute its Present Value. In this regard, the literature on the evaluation of health projects recommends a discount rate that is comparable to the social investment opportunity cost in the country where the project will be implemented. The yield of Dominican sovereign bonds is around 8 percent. 41 8. The Present Value of the Future Benefits of the reduction in infant mortality, maternal mortality and disabilities from chronic diseases are positive although lower than originally estimated in the PAD. Assumptions may have been on the high end while reductions are multi causal and therefore difficult to attribute to the Project. The ranges of reduction in the rates of infant mortality, maternal mortality, and disabilities from chronic diseases due to the implementation of the project assumed were between 10 percent and 25 percent), between 40 percent and 50 percent and between 40 percent and 50 percent, respectively. The actual rates when comparing data from the 2007 and 2013 ENDESA survey were 15.62 percent for infant mortality indicating that the reduction is on the lower end of the estimation used.39 The Maternal mortality ratio, according to the World Health Organization increased between 2005 and 2015 from 64 to 92 (there was no data in the 2013 ENDESA survey). Therefore the assumptions made would not be within the range of the actual changes in the MMR. However, this is likely due to issues related to institutional births, since 99 percent of women give birth in hospitals and therefore beyond the scope of PARSS2 which deals with primary level of care. The estimates regarding the reduction from chronic diseases due to Project implementation were in line with actual/current estimates based on available data. (Table 1). Table 1. Estimated Present Value of the Future Benefits (US$) at a discount rate of 8 percent when the rates of reduction of infant mortality, maternal mortality, and disabilities from chronic diseases are at the middle of their respective ranges (from Project PAD). Present Value of Present Value of Present Value of Year Benefit from Maternal Benefit from Infant Benefit from Deaths Avoided Deaths Avoided Disabilities Reduced 1 24,257 33,839 2,976 2 1,134,226 1,582,273 139,148 3 1,909,280 2,663,492 234,233 4 2,380,707 3,321,145 292,068 5 755,442 1,051,855 87,680 9. The same exercise was performed upon the stream of the costs of the implementation of the project to compute the Present Value of the costs . The difference between the present value of the stream of quantified benefits and the present value of the stream of expenditures yields the net present value of the project. 10. A comparative approach completed during Project implementation provides some useful observations between the regions with and without Project implementation. The results are generally favorable. Since the IADB project did not begin implementation until 2014 we can compare the difference from mid-2011 to 2014 39 However the data only reaches 2013. 42 via the clinical management system. The analysis examined differences between: (a) pregnancy and post-pregnancy, (b) control of healthy child growth, (c) control and following of NCDS, and (d) general pathologies. In each of the cases the number of consultations per person were generally greater in the regions where the Project was implemented. We can largely say that the financial incentives improved the data collection capacity of the RHS’s. This could be due to better registration or an improvement in the following of individuals in the regions. At least we can assume that the Project promoted a change in the management and control for services at the first level of attention. 11. The Project regions measured a higher average number of consultations per pregnancy than the non-Project regions. The three regions where the Project was implemented counted on an average of 2.27 consultations per pregnancy versus 1.8 in the non-Project regions (an average 35 percent more). Three other indicators also favored the Project areas: a) regions with only one consultation during pregnancy, b) pregnancies with more than three consultations, c) woman that have a consultation before the 12th week of pregnancy (Table 2). Table 2. Pregnancies Region # App. # Preg Avg % of % of % % more % w app. App. App. preg only than 3 before 12 one app. week app. No Proj. 17,380 9,655 1.8 21% 26.4% 63.9% 10.7% 23.8% Project 65,414 26,894 3.43 79% 73.6% 46.1% 23% 38.3% Total 82,794 36,549 2.27 100% 100% 50.8% 19.8% 34.5% Source: Bank draft - Programa de Apoyo a la Reforma del sector Salud 2(PARSS2). Componente 1: Análisis de algunas evidencias 12. The Project regions also registered more comprehensive services reflecting better quality of service. As revealed in Table 3, all except for one of the parameters measured were better in the Project areas. Seven of the parameters, showed significantly improved differences (58 percent better). Adolescent pregnancies (under 19 years of age) also reveal improved treatment in the Project regions. Table 3. Service quality during first consultation (Percentage) Region FPP Hemoglobin Previous VDRL ITS Anti- Gest. Uterine Fetal Blood Iron Folic acid (%) (%) Weight (%) (%) tetanus Age height Heart pressure Supplement supplement (%) (%) (%) (%) rate (%) (%) (%) (%) No 98.3 9.9 82.0 68.3 69.2 82.3 29.2 28.2 26.2 31.4 4.9 4.6 Project Project 99.7 23.6 83.4 69.6 71.1 80.3 46.5 40.9 36.4 49.0 8.1 7.6 TOTAL 99.3 20.0 83.0 69.2 70.6 80.8 42.0 37.5 33.7 44.4 7.2 6.8 Proj/No 1.01 2.38 1.02 1.02 1.03 0.98 1.59 1.45 1.39 1.56 1.65 1.65 Project Source: Bank draft - Programa de Apoyo a la Reforma del sector Salud 2(PARSS2). Componente 1: Análisis de algunas evidencias 43 13. Postpartum consultation also revealed the benefits of Project activities. The number of women with consultations in the first level of care following giving birth is nearly four times higher in the Project areas (Table 4). Table 4. Postpartum Postpartum % of with postpartum as Births appointments a % of births No Project 10,847 890 8.21% Project 25,297 8846 34.97% Total 36,144 9736 4.26 Source: Bank draft - Programa de Apoyo a la Reforma del sector Salud 2(PARSS2). Componente 1: Análisis de algunas evidencias 14. A great number of consultations implies better service quality since the exposure of the individuals to a health care consultation assumes a better situation for the individual. The differences between Project and non-Project areas are less clear when examining healthy children consultations and NCD screening. While Project areas reveal more healthy-child consultations, non-Project areas reveal more comprehensive consultations. Likewise, there were no clear differences when examining comprehensive screening for a number of NCDs. 15. The Project spent less than originally earmarked for Component 1. This was partly due to the enrollment of individuals under Component 1 by SENASA. This was a positive in terms of the institutionalization of the health insurance but had financial implementation for the different stakeholders (SENASA, the component and the RHSs) that must be analyzed. 44 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility / Specialty Lending Abreu Rojas, Zoila Catherine Procurement Specialist GGO04 Altimari Montiel, Fabiola Senior Counsel LEGLE Bablumian, Isabella Consultant GHND R Guedes, Andrea C. Senior Operations Officer GED03 Mitrovic, Snezana B. Adviser GGOPI Montenegro Torres, Fernando Senior Economist GHN04 Moreno Zevallos, Maria Gabriela Program Assistant GHN04 Novinskey, Christina Health Specialist GHND R Perazzo, Alfredo C Consultant GHN04 Perez Gil, Julian Consultant Rodriguez De Pichardo, Maritza Senior Financial GGO22 Management Specialist Ruiz, Sonia E Schmunis, Rocio Operations Officer GHN05 Valladares, Elio Enrique Williams, Judith Marcano Supervision/ICR Abreu Rojas, Zoila Catherine Procurement Specialist GGO04 Afanador Machuca, Santiago Jose IT Officer ITSGP Avila Rodriguez, Jose Mauricio Baquero, Patricia H. Consultant GWAD R Beneitez, Geraldine C. Program Assistant GHND R Camporeale, Vanina Senior Operations Officer GHN04 Carpio, Carmen Senior Operations Officer GHN04 Cavero Uriona, Jorge Alberto Chinea de Leon, Oscar Emilio Escandon, Ana Cecilia Consultant GHN04 Gonzalez, Viviana A. Program Assistant GHN04 Gordillo-Tobar, Amparo Elena Senior Economist GHN04 Hernandez Martinez, Nelson Consultant GHN04 Alberto Hullin, Carol Consultant Jahnsen, Javier Consultant GHN04 45 Macias, Claudia Senior Operations Officer GHN04 Marte Romero, Adriano Medina Yaguaran, Reinaldo J. Senior IT Assistant ITSCW Mi, Yuan Miranda Vico, Marcos Vicente Consultant Montenegro Torres, Fernando Senior Economist GHN04 Moreno Zevallos, Maria Gabriela Program Assistant GHN04 Nunez, Ramiro Consultant ITSCR Pena, Christine Lao Senior Human GHN04 Development Economist Perazzo, Alfredo C Consultant GHN04 Platais, Gunars H. Rodriguez De Pichardo, Maritza A Senior Financial GGO22 Management Specialist Santiago Bench, Natalia Consultant Schmunis, Rocio Operations Officer GHN05 Streveler, Dennis Consultant GHN05 Sviedrys, Nikolai Consultant OPSPF Veizaga, Mary Lou M Wu, Yingwei Senior Procurement GGO04 Specialist Williams, Judith Marcano Xu, Jing Zamecnik, Natasha Consultant GSU10 (b) Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) No. of staff weeks USD Thousands (including travel and consultant costs) Lending FY09 6.67 75.05 FY10 6.91 22.91 Total: 13.58 97.96 Supervision/ICR FY10 14.21 42.57 FY11 25.66 141.15 FY12 30.87 162.70 FY13 22.24 166.02 FY14 17.39 134.30 FY15 13.69 109.05 FY16 8.62 77.76 Total: 132.68 833.55 46 Annex 5. Beneficiary Survey Results N/A 47 Annex 6. Stakeholder Workshop Report and Results N/A 47 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR FINAL EXECUTION REPORT Health Sector Reform Second Phase APL (PARSS 2) THE WORLD BANK GROUP 48 1 INTRODUCTION This document contains the Final Report of the Project to Support the Implementation of the Health Sector Reform in its second phase (PARSS2). The aim of the report is to succinctly present the final results of the project’s implementation, both in physical and financial terms; to highlight key achievements in regard to its indicators and lessons learned; and to provide an overall analysis of the Project’s contribution according to the development goals set out in the context of the Health Sector Reform in the DR. The report’s preparation and format reflects the requirements contained in the Loan Agreement BM-7777-DO, developed by the Project Coordinating Unit of the (CERSS), with the support of the Department for Strategic Institutional Development in the Public Health Ministry (MOH). This report will be complemented by an Implementation Completion and Results Report (ICR) to be conducted by external consultants hired by the World Bank (WB), the Project’s financing entity. The document presents a brief description of the Project’s background, an analysis of its results indicators and targets, as well as a description of the most important achievements and the final results of the financial implementation. Finally, a section on lessons learned and prospects for process sustainability is included. 2 BACKGROUND In mid-2001 the DR legally consolidated a process of reforming its health system through two fundamental laws, the law 42-01 (General Health Law) and the Law 87-01 (Law creating the Dominican System of Social Security -SDSS-). The model is based on the separation and decentralization of the four basic functions of health systems: stewardship, financing, insurance and provision of services. The health sector reform originates with two fundamental purposes:  To improve the health status and quality of life of the Dominican population, with emphasis on the most vulnerable groups and the protection of women.  To make structural and functional changes in the main health sector institutions for the organization of a National Health System. In this regard, in June 2003 the Government of the DR and the World Bank (WB) signed an agreement for financial support and technical assistance that contributed to the implementation of the Health Sector Reform. This financial support was designed as an APL. The series of three APLs began with the first project to support the Health Sector Reform (PARSS1), whose implementation effectively began in 2005 and ended in 2009. Considering the negative effects of the global economic crisis and the urgent need for progress in key reform processes, that also have synergies with the new planning framework and management of the national budget results and other social protection programs in the country, the Dominican government decided to initiate the quick 49 preparation of the second phase of the APL (PARSS2). This was done in order to provide resources to strengthen the primary care level, improving citizens’ access to quality services, which constitute an important part of the costs of illness the populations’ poorest households. The second phase of PARSS (PARSS2) began operations in January 2010, with a more targeted range of activities that were focused on aligning financial incentives and improving management, in order to foster improvements in the delivery of services in the primary care level. The PARSS2 is designed to maximize the use of the newly established structures for the separation of functions in the health sector and the decentralization process, managing for results and accountability in the delivery of public sector services. 3 OBJECTIVES OF PARSS2 The Project's development objectives are to: (i) Improve the capacity of RHS to deliver, in a timely fashion, quality services known to improve the health of mothers, children and people with chronic conditions by public providers at the first level of care; (ii) improve health system responsiveness, defined here as the institutional capacity of public sector health organizations to conduct strategic purchasing of health care services and goods, and to respond to public health emergencies. The proposed indicators to measure progress of the Project objectives include: (a) Indicators for Component 140: (i) Percentage of pregnant women from target population with risk evaluation completed before the 15th week of pregnancy. (ii) Percentage of children from target population under 15 months with vaccination scheme completed according to national protocols. (iii) Percentage of individuals from target population diagnosed with hypertension under treatment according to national protocols. (b) Indicators for Components 2 and 3: (i) Baseline studies on risk factors and burden of disease have been conducted, disseminated and used by corresponding MOH units (collective health) to develop operational plans for health promotion and prevention. (ii) At least 1 region is producing reports every 4 months on no. of poor individuals who were prescribed a medicine (from essential medicines list) at the first level of care or in a PROMESE/CAL establishment and received medication in 48 hours. 40 These indicators were selected from the catalogue of basic SFS services to improve key health outcomes for the poorest population, whose wellbeing greatly concerns the Dominican Government. 50 4 BRIEF DESCRIPTION OF THE PROJECT’S COMPONENTS The project comprises four components: Component 1: Establishment of a Results-based financing (RBF) mechanism for the first level of care in regional public health networks. The objective of this component is to introduce an advanced financial mechanism within the two main sources of public funding for the first level of health care: the MSP budget for RHS and the SR funds, transferred by the SENASA for RHS as part of the formal contract. RBF mechanisms would promote accountability and seek to align the incentives between these two sources of subsidies for the improvement of service quality in health care centers. This component used the results-adjusted capitations for the allocation of resources for the first level care for the poor in RHS regions VI, VII41 and VIII. The project expanded its impact on the poor in the participating regions through an agreement with SENASA to introduce the same mechanism of capitation for individuals registered in the SR. To ensure the sustainability of RBF mechanisms introduced by this Project, agreements and priority services were based on those already existing for the financing and delivery of care to the poor population groups through the SR. Capitation adjusted results are transferred to the RHS in two parts. One is a fixed amount and represents 50% of the capital and the other is a variable amount based on the results obtained and could reach a maximum of 50% of the capital, according to information on the achievements and progress on the ten annual indicators defined for monitoring results (IMR). For this purpose, a consulting firm was hired to carry out concurrent external technical audits (CEA). Component 2: Strengthening the capacity of the Ministry of Health to improve the health system’s response, promoting transparency and accountability. The objectives of this component are: (i) to strengthen the administrative capacity of the MOH to improve transparency and accountability in the delivery of public health services and goods with emphasis on primary care services; (ii) to support the MOH in strengthening the existing IS and to develop new M&I instruments, needed for the proper operation of the capitation based on results, that provide timely information on the quality of service delivery; iii) to contribute to the financing of interventions MOH and to coordinate health system activities to provide a faster and better response to public health emergencies. This component has two sub-components: 41 From September 2013, this included RHS-VII in the RBF mechanism. 51  Sub-component 2.1. Strengthening the MOH’s assessment and strategic planning capacities to improve its management, promoting transparency and accountability with emphasis on improving the health system response; its main objective is to strengthen the institutional capacity of the MOH to carry out strategic analysis, planning and implementation of sectoral interventions in order to exercise its supervision over the delivery of health services and public health interventions at national and regional level.  Sub-component 2.2. Strengthening the Monitoring, Information and Evaluation systems for performance-based funding; focused on supporting the Office of the National M&I System to improve the existing instruments in the MOH’s Management of Health Information System. Component 3: Improving the quality of public spending on health sector goods and services. The objective of this component is to strengthen the institutional capacity of public sector organizations for sustainable financing, planning and procurement of health services and other key inputs needed to improve the quality of health care, with an emphasis on primary care units. The sub-component has the following sub-components:  Sub-component 3.1. Strengthen the planning, procurement and distribution of medicines and medical supplies in the public sector; aimed at improving the planning, procurement and more efficient distribution of essential drugs and other supplies purchased by PROMESE/CAL.  Sub-component 3.2. Strengthen the institutional capacity to expand and improve social health protection for the poor; directing interventions to strengthen the institutional capacity of SENASA and other key entities to consolidate and expand the SR. Specifically, by focusing resources on improving the RHS’ provider payment system, by institutionalizing results-based financial mechanisms, and by strengthening accountability and transparency in overall health funding. Component 4: Support in the response to public health emergencies The fourth component was included as part of the lessons learned within the framework of the global economic crisis, its impact on fiscal resources, and the new threat to public health in the DR posed by the influenza AH1N1 virus. This type of component provides quick access to Bank funds, in order to allow a more effective response to health emergencies. Experience shows that, in emergencies, the need for amendments and fund reallocations can delay access to funds rapidly. 52 5 SUMMARY OF THE PROJECT’S OUTCOME INDICATORS PROJECT DEVELOPMENT INDICATOR ACHIEVED TARGETS AT PROJECT CLOSING PROPOSED TARGET OBJECTIVE 1. Percentage of pregnant women from target Improve the capacity of population with risk evaluation completed 50% 40% Regional Health Services before the 15th week of pregnancy. (RHS) to deliver, in a timely fashion, quality services 2. Percentage of children from target known to improve the population under 15 months with vaccination health of mothers, children 68% 60% scheme completed according to national and people with chronic protocols. conditions by public 3. Percentage of individuals from target providers at the first level population diagnosed with hypertension of care. 38% 35% under treatment according to national protocols. 4. Baseline studies on risk factors and burden of disease have been conducted, Study results have The target has been achieved. The studies disseminated and used by corresponding MOH been used to were disseminated between technical MoH units (collective health) to develop develop annual staff in the VI, VII and VIII RHS. operational plans for health promotion and operation plans. prevention. Reports are being developed for two regions Improve health system participating in PARSS (VI and VIII), which responsiveness, defined include a dashboard composed of information here as the institutional generated in the SIMI: capacity of public sector 1.- MEDICAL PRESCRIPTION health organizations to - list of protocol treatments conduct strategic purchasing - categorization by pathology 5. At least 1 region is producing reports every of health care services and - list of protocol treatments categorized by 4 months on no. of poor individuals who were At least one region goods, and to respond to pathology prescribed a medicine (from essential is producing public health emergencies. - list of prescribed active components medicines list) at the 1st level of care & quarterly reports. - list of prescribed medicines or medical received medication in 48 hrs. supplies - list of CPN treatments in each province/region 2.- DISPENSATION - list of dispensed active components - list of dispensed medicines - list of dispensations by CPN 6. Percentage of clinical files with complete information for external audit to verify RHS 99% 85% reported MRI results. 7. Percentage of post-partum women referred from hospital to their corresponding first level of care units with proper documentation 63% 50% included in their clinical files at the health center. 8. Percentage of children over 15 months and under 24 months with vaccination scheme 63% 45% completed according to national protocol. 9. Percentage of children under 24 months with growth and development monitor 62% 50% Component 1 controls according to national protocol. 10. Percentage of individuals older than 18 years screened for hypertension based on 64% 60% national protocols. 11. Percentage of individuals older than 18 years screened for diabetes according to 43% 35% national protocol. 12. Percentage of individuals diagnosed with diabetes type II under treatment according to 34% 30% national protocol. 13. Percentage of individuals symptomatic of TB with results of sputum smears in clinical 45% 35% file of corresponding primary level health center. 53 PROJECT DEVELOPMENT INDICATOR ACHIEVED TARGETS AT PROJECT CLOSING PROPOSED TARGET OBJECTIVE 14. The Health Management System of the MOH includes an automated/electronic Results-adjusted The Ficha Familia and SGC have been module for M&I with all the information Capitation launched. The results-adjusted capitation needed Module used in mode is used in eight regions. for monitoring results on a quarterly basis at at least one region. the first level of care. Studies' results Component 2 completed and disseminated and used for 15. Risk factors and burden of disease surveys The surveys were disseminated between planning public designed, implemented and their results technical MoH staff in the VI and VIII RHS. health disseminated. interventions with an emphasis on health. 16. Percentage of accredited health centers from participating RHS that do not have a Component 3 PROMESE/CAL pharmacy and receive on a 100% 80% regular basis medicines procured and distributed by PROMESE/CAL. 6 SUMMARY OF PARSS2’ MAIN ACHIEVEMENTS Implementation of PARSS2 resulted in significant achievements and externalities, deepening the sector’s reform process and improving the quality of health services to the population. A summary of the improvements is included below:  Introduction of RBF mechanism at the primary care level in three Regional Health Services, expanding to all the affiliated RS SENASA population and poor population (366.236 people under the regime), which have been incorporated into the SDSS, predominantly as members of SENASA.  Implementation of the RBF mechanism involved financial transfers of approximately RD $ 400 million to the three health regions selected to implement the pilot promoting the culture of accountability and monitoring. The RHS selected were: Region VI (San Juan de la Maguana, Azua and Elías Piña), Region VII (Mao Valverde, Santiago Rodriguez, Montecristi and Dajabón), and Region VIII (La Vega, Sanchez Ramirez and Monseñor Noel).  Development of a computerized registry at the primary care level through the System for Clinical Management (QMS) in the Regional Health Services VI, VII and VIII. This increased to 80% the total registrations in the country during the first years of PARSS2 and before the expansion of the RBF mechanism to the other five RHS, carried out with IDB financing.  The RBF mechanism established systematization capacities and managerial discipline, which enabled the fulfillment of the proposed indicators by more than 90% on average, according to the last five concurrent quarterly external audits.  Once the effectiveness of the RBF mechanism was proven, its application was gradually replicated at the first level of care in the rest of the country’s health regions. 54  Support the strengthening of the Health Information System (ISMS) as part of the component strengthening key health sector institutions; as well as the development and implementation of information tools and basic procedures for the establishment and strengthening of the Ministry’s stewardship role within the framework of separation of functions.  Support SENASA’s institutional strengthening with the aim of transforming its business model and strengthening its role as strategic buyer of health services.  Support the establishment of a supervision and monitoring methodology for primary care management in prioritized areas, which serves as the basis for the adoption of an M&E culture in these areas.  Design and implementation of an IS and management model for the medicines and health inputs supply chain (Electronic Prescription), operating in regions VI, VII and VIII. Through this initiative, investments were made in software, equipment, technical assistance and training for better planning, procurement, distribution, prescription and dispensing of essential drugs. As a result, the conditions were established to ensure the availability, quality and good state of medicines and supplies in the participant RHS’s. Additionally, as well as providing information on the diagnosis and treatment versus the epidemiological profile of the population, transparency was strengthened regarding patient medicine prescription and dispensation in the health system by medical staff, health staff, funding source or condition of affiliation to SDSS and health regions. External Factors affecting PARSS2 The Dominican government has recently launched a series of structural changes and policy institutions in the health sector, in line with the project objectives and its contribution to the reform. The main changes are summarized below:  At the end of 2014, according to Decree 379-14, the administrative, functional and territorial decentralization of the Vice-ministry of Attention to the People and the Regional Health Services was made effective. Its aim was to begin the separation of functions regarding the provision of health and stewardship of community health services, in compliance with the General Health Law and the Law on Social Security. Thus, the General Directorate of Public Health Care Services (DGCSS, by its Spanish acronym) was created as the entity which would be responsible temporarily for coordinating the RHS with their institutions and the self-managed health care centers.  In July 2015, the President signed law 123-15, which created the National Health Service and that defined its mandates and the timeframes for the integration and adaptation of the single network of public health services under the NHS and for its operation.  The MOH has proposed a redefinition of the care model, whose conceptual proposal is a comprehensive approach that defines and describes the optimal way to organize health activities in order to meet the populations’ requirements and 55 demands. The implementation of a project to design the operational model has already began.  Through Decree 608-12, the Essential Program/Central Logistics Support (PROMESE/CAL) was created. This program will stay under the MOH to serve as the only central supplier of medicines, medical supplies and laboratory chemicals, to meets the demand of the National Health System.  The Government has made the RBF strategy the basis for the transformation and strengthening of primary care, expanding the experience of PARSS2 with two new projects financed by the Inter-American Development Bank (IDB). It has also adopted the mechanism for the management of resources transferred to the health regions by SENASA.  The Government Program, the Strategic Agenda of the MOH and the country's commitment to the Millennium Development Goals have prioritized the goal of reducing maternal mortality and improving maternal and child health.  As part of the MOH’s strategic objectives, as defined in its Multiannual Strategic Plan, are strengthening the leadership capacity within the Ministry, through the effective management of the NHS; improving the Essential Public Health Functions’ performance; promoting the development and strengthening of skills required to increase the quality of life of the population, ensuring equal access to safe and effective drugs through proper implementation of the national drug policy; and ensuring health services to the population and the financial sustainability of the National Health System. 7 PROJECT AND FIDUCIARY MANAGEMENT Two entities shared technical and fiduciary responsibilities for the Project’s implementation: the MOH, through PIU-DDEI, was in charge of components 2 and 4; while in the CERSS, the Project Coordination Unit (PCU) assumed responsibility for components 1 and 3. At the same time, the CPU was responsible for coordinating, collecting and channeling the necessary information for the execution the PARSS2, according to the provisions in the Operations Manual, the Loan Agreement and other WB standards and guidelines applicable to the Project. Although there were substantial changes in the teams’ composition, Project continuity and implementation was strong with an overall satisfactory Project performance and objective achievement. Close collaboration with the World Bank team, both in Washington and with the DR representatives, was essential to the Project’s execution, especially during critical periods. During implementation, two requests for fund reallocation were processed and approved. These requests resulted from the failure to disburse counterpart contributions as set forth in the Loan Agreement and from excess resources in the expenditure forecasts of Component I. Additionally, a request for extending the Project’s closing date was processed and approved. Therefore, instead of closing on October 30, 2015, the Project 56 closed on February 29, 2016. Regarding the execution of the loan proceeds, RD $ 28.9 million were applied, equivalent to 94.8% of the total amount approved. PARSS2 Original Final Effectivity date 19-Jan-10 NA Closing date 30-Oct-15 29-Feb-16 Total amount of project US$44.2 million US$34.6 million Loan amount US$30.5 million US$30.5 million Counterpart amount US$13.7 million US$4.1 million Financial Management Approved Executed % Total US$30.5 million US$28.9 million 94.8 Component I US$8.67 million US$8.25 million 95.2 Component II US$7.00 million US$6.10 million 87.1 Component III US$14.65 million US$14.39 million 98.2 Component IV US$0.10 million US$0.08 million 78.6 Front-end Fee US$0.08 million US$0.08 million 100.0 8 LESSONS LEARNED Significant lessons emerged during project implementation, which can be used for the continuity and sustainability of the program, beyond PARSS2 completion. The lessons are as followed:  Results-based financing has great potential to generate changes and modifications in health systems, promoting transparency, monitoring, accountability, and improving the quality of public expenditure in the health sector. It can have a positive influence both in the clinical management area and in the technical and administrative functions.  Financing mechanism results can be used to achieve specific objectives of the health system. Project experience shows that health systems can become more dynamic and oriented towards the system’s priority targets. In this regard, clearly defined indicators are required to focus efforts towards the sector’s strategies and policies, taking into account the health system’s profile.  Financing by results generates positive externalities. The central actions of this financing model can spread to other activities and processes, extending the positive impact of the Project beyond its main initiatives.  Financing by results can help improve the coverage and quality of services provided. The resources provided act as an incentive to improve coverage and access; while the variable portion, derived from IMR compliance, incentivizes the quantity and quality of services offered.  New initiatives and RBF projects generate initial resistance that weakens once benefits are perceived.  ISs play an essential role in the RBF modalities. They can reduce the time it takes to input data and generate reports. A unique registry should be the preferred 57 mechanism to further reduce work by effectively capturing clinical and management data, with a clear awareness of the importance of implementing a rigorous and continuous monitoring strategy, and updating and maintaining the IT systems to preserve their reliability, particularly database and operability.  Performance incentive procedures and resource delivery times must be agile and rapid in order for the beneficiary to consider them a real motivating factor to induce change and improvement.  It is necessary to deepen interventions for the transformation of the SENASA’s business model by strengthening the institution as a strategic buyer of health services and as a funding entity of the public sector in the context of the Dominican Social Security System.  Interventions under the project greatly contributed to creating the basis for a risk- management and operational control system, which should be developed along with the RBS model and an efficient strategy to improve SENASA risk-management.  For future operations, it is important to accompany interventions in PROMESE/CAL, SENASA, MOH and other beneficiaries with a strong component of political and institutional capacity improvement, and with better integration of institutional plans and external financing’s development objectives.  The implementation of SIMI established the basis for a cross-sector health IS. This allows National Health System managers to effectively use and spend on drugs and medical supplies provided by national budgets. This transformed the management systems for: Optimizing the use of resources, ensuring the availability of medicines and supplies, improving the quality of clinical actions and assisting planning and decision-making tasks.  Having achieved considerable success in SIMI implementation results in RHS VI, VII and VIII, it is necessary to extend the model of electronic medical prescription for the first level of care to other regions of the country.  ICT infrastructure deficiencies were identified in SIMI project execution, which must be addressed to maximize system use and disseminate the experience in the first level of attention of the public health service network.  The experience with the national counterpart’s contributions suggests projects could be carried out without this entity in future, especially when economic circumstances impose a budget deficit. 9 PROJECT SUSTAINABILITY Experiences and progress made through PARSS2 have constituted a significant advance towards transforming the delivery of quality health services at the primary care level. This provides a solid base from which the third phase of the program (PARSS3) seeks to: further consolidate and guarantee the sustainability of the reform process for the provision of quality health services in the context of a comprehensive health system; and to contribute to build a financially sustainable model of attention. Moreover, it is important to note that, in order to ensure the sustainability of reforms undertaken under PARSS2, firm steps need to be taken for strengthening institutional capacities, namely: 58  Training of RHS for managing the QMS, registration services, and the analysis of their results as a requirement for the preparation of lists of services provided during each period;  Strengthening the technological infrastructure through the implementation of tools and systems that facilitate management and decision-making;  Promoting frequent knowledge exchange meetings between the RHS with the objective of sharing best practices;  Supervision, monitoring and ongoing training to ensure timely detection of risks and appropriate mitigation measures;  Strengthening the role of planning and scheduling to ensure an increase in the quality of the investment of economic resources and the creation of portfolios of investment projects with an emphasis on the health strategy;  Strengthening the financial and administrative structures of the RHS, improving technical staff profiles and incorporating new routines for the programming and usage of funds;  Support improvements in the provision of medicines and medical supplies;  Improvement of SENASA’s institutional capacities for the strengthening of its financial sustainability. These institutional capacities, together with the MOH's decision to continue and extend the use of the RBF mechanism in others RHS’s, reinforces the favorable prospects for the continuity and sustainability of the progress achieved through the PARSS2. 59 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A 60 Annex 9. List of Supporting Documents FISS-MSP (1997): Usos Alternativos de Financiamiento en Salud. Estudio sobre Puntos Críticos para su Reasignación en Base a1 Criterio de Coste Efectividad. Segunda Parte. Intervenciones Costes Efectivas para las Condiciones Médicas con Mayor Carga de Morbilidad (AVISA).Uruguay. Gertler P.; Giovagnoli P.; & Martinez S. (2014). “Rewarding Provider Performance to Enable a Healthy Start to Life: Evidence from Argentina’s Plan Nacer”. World Bank Policy Research Working Paper 6884. Inter-American Development Bank (IADB). (2010). Apoyo al Programa de Protección Social - Segunda Fase. CCT-Programa Solidaridad. Washington, DC. Ministry of Health (Secretaría de Estado de Salud Pública y Asistencia Social). (2001). Ley General de Salud. Ley No. 42-01. Dominican Republic. National Social Security Council (Consejo Nacional de Seguridad Social). (2001). El Sistema Dominicano de Seguridad Social. Ley No. 87-01. Dominican Republic. World Bank Indicators World Bank. (2005). Uruguay Health Sector review. An Analysis of the Health Sector: Groundwork for an Evidence-Based Reform; Report No. 33710-UY, Washington DC. World Bank. (2005). Country Assistance Strategy for the Oriental Republic of Uruguay for the period FY05-FY10; Report No. 31804-UY; Washington DC. World Bank. (2007). Healthy Development: the World Bank strategy for health, nutrition, & population results. Washington, DC: World Bank. World Bank. (2009). Country Partnership Strategy for the Dominican Republic for the period FY10 – FY13; Report No. 49620-DO; Washington, DC World Bank. (2009). Project Appraisal Document. Health Sector Reform APL 2 (PARSS2) Project; Report No. 49731-DO; Washington, DC. World Bank (2009). Loan Agreement. Health Sector Reform APL 2 (PARSS2) Project; Washington, DC World Bank (2009 – 2015). Supervision Aide Memoires and Implementation Status Reports; Washington, DC World Bank. (2010). Implementation Completion and Results Report. Health Reform Support (APL1); Report No. ICR00001420; Washington, DC World Bank (2014). Restructuring Paper. Health Sector Reform APL2 (PARSS2) Project; Report No. RES11756; Washington, DC World Bank (2015). Amendment to the Loan Agreement. Health Sector Reform APL2 (PARSS2) Project; Washington, DC 61 MAP 62