Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004484 IMPLEMENTATION COMPLETION AND RESULTS REPORT (P110599) ON A LOAN IN THE AMOUNT OF US$461 MILLION TO THE ARGENTINE REPUBLIC FOR THE ESSENTIAL PUBLIC HEALTH FUNCTIONS AND PROGRAMS II PROJECT December 13, 2018 Health, Nutrition & Population Global Practice Latin America And Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective December 1, 2018) Currency Unit = Argentina Peso (Arg$) Arg$ 37.50 = US$1 FISCAL YEAR July 1 - June 30 Regional Vice President: Jorge Familiar Country Director: Jesko Hentschel Senior Global Practice Director: Timothy Evans Practice Manager: Daniel Dulitzky Task Team Leader(s): Luis Orlando Perez ICR Main Contributor: Cecilia Zanetta ABBREVIATIONS AND ACRONYMS AIDS Acquired Immune Deficiency Syndrome ANMAT National Food, Drug, and Health Technology Administration (Administración Nacional de Medicamentos, Alimentos y Tecnología Médica) CPS Country Partnership Strategy COFESA Federal Health Council (Consejo Federal de Salud) CU Coordinating Unit EPHFs Essential Public Health Functions FESP I Essential Public Health Functions Project I FESP II Essential Public Health Functions Project II FHP Federal Health Plan FM Financial Management FY Fiscal Year GAAP Governance and Accountability Action Plan GDP Gross Domestic Product GoA Government of Argentina HIV Human Immunodeficiency Virus ICR Implementation Completion and Results Report IPPF Indigenous Peoples Planning Framework KOI Key Outcome Indicator LEGISALUD Sanitary Legislation Information System M&E Monitoring and Evaluation MSN National Health Ministry MSP Provincial Health Ministry NCDs Non-Communicable Diseases NOMIVAC Federal Register of Nominalized Vaccination OFERHUS Federal Observatory of Human Resources in Health OM Operational Manual OP/BP Operational Policy/Bank Procedures ORAF Operational Risk Assessment Framework PACBI Diseases of High Cost and Low Incidence PAD Project Appraisal Document PAHO Pan-American Health Organization PHA Public Health Activities (Actividades de Salud Público) PIU Project Implementation Unit PDO Project Development Objective PPHPs Priority Public Health Programs PROCYS Procurement Management Information System PROFE Federal Health Program PAP Procurement Action Plan REDARETS Public Network of Evaluation of Health-Related Technologies REDOs Register of Blood Donors REFES Federal Registry of Health Establishments REFEPS Federal Registry of Health Care Professionals RBF Results-Based Financing RENIS National Registry of Health Research RHUS Federal Network of Human Resources in Health SEPA Procurement Plan Execution System SINTRA National System for Transplants SISA Health Services Information System SMIS Integrated Monitoring System of Medical Supplies SNARES Registry of Medical Residences STD Sexually Transmitted Diseases SHIV HIV Patient Management System TA Technical Assistance TB Tuberculosis UFIS International Financing Unit for Health – National Ministry of Health YPLL Years of Potential Life Lost WHO World Health Organization TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .......................................9 II. OUTCOME .................................................................................................................... 10 A. RELEVANCE OF PDOs ............................................................................................................10 B. ACHIEVEMENT OF PDOs (EFFICACY) ......................................................................................11 C. EFFICIENCY ...........................................................................................................................19 D. JUSTIFICATION OF OVERALL OUTCOME RATING ....................................................................20 E. OTHER OUTCOMES AND IMPACTS (IF ANY)............................................................................20 N.A. .........................................................................................................................................21 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION ...................................................................................21 B. KEY FACTORS DURING IMPLEMENTATION .............................................................................22 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 22 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................22 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE .....................................................24 C. BANK PERFORMANCE ...........................................................................................................27 D. RISK TO DEVELOPMENT OUTCOME .......................................................................................28 V. LESSONS AND RECOMMENDATIONS ............................................................................. 29 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 56 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 58 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 59 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 61 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 74 The World Bank Essential Public Health Functions Programs II Project (P110599) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P110599 Essential Public Health Functions Programs II Project Country Financing Instrument Argentina Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Argentine Republic Ministerio de Salud de la Nacion Project Development Objective (PDO) Original PDO The PDOs are to: (i) improve the stewardship role of the federal public health system, through the strengthening of Essential Public Health Functions; and (ii) increase the coverage and clinical governance of Priority Public Health Programs Page 1 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 461,000,000 461,000,000 445,785,893 IBRD-79930 Total 461,000,000 461,000,000 445,785,893 Non-World Bank Financing Borrower 0 0 0 Total 0 0 0 Total Project Cost 461,000,000 461,000,000 445,785,893 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 20-Dec-2010 03-Aug-2011 01-Dec-2014 30-Jun-2016 30-Apr-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 13-Jun-2016 403.02 Change in Loan Closing Date(s) Reallocation between Disbursement Categories 24-Apr-2018 445.57 Change in Disbursements Arrangements KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 20-Feb-2011 Satisfactory Satisfactory 0 Page 2 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 02 23-Jul-2011 Satisfactory Satisfactory 0 03 22-Jan-2012 Satisfactory Satisfactory 139.30 04 09-Nov-2012 Satisfactory Satisfactory 159.24 05 10-Jul-2013 Satisfactory Satisfactory 232.01 06 19-Jan-2014 Satisfactory Satisfactory 328.03 07 12-Jul-2014 Satisfactory Satisfactory 328.03 08 06-Jan-2015 Satisfactory Satisfactory 369.17 09 30-Jun-2015 Satisfactory Satisfactory 385.45 10 29-Dec-2015 Moderately Satisfactory Satisfactory 403.02 11 23-Jun-2016 Moderately Satisfactory Satisfactory 403.02 12 21-Dec-2016 Moderately Satisfactory Moderately Satisfactory 418.66 13 20-Jun-2017 Moderately Satisfactory Moderately Satisfactory 430.84 14 28-Dec-2017 Satisfactory Moderately Satisfactory 445.57 15 30-Apr-2018 Moderately Satisfactory Moderately Satisfactory 445.57 SECTORS AND THEMES Sectors Major Sector/Sector (%) Health 100 Public Administration - Health 70 Health 30 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Human Development and Gender 90 Disease Control 40 HIV/AIDS 6 Tuberculosis 4 Non-communicable diseases 30 Health Systems and Policies 50 Health System Strengthening 50 ADM STAFF Role At Approval At ICR Regional Vice President: Pamela Cox Jorge Familiar Calderon Country Director: Penelope J. Brook Jesko S. Hentschel Senior Global Practice Director: Keith E. Hansen Timothy Grant Evans Practice Manager: Joana Godinho Daniel Dulitzky Task Team Leader(s): Fernando Lavadenz Luis Orlando Perez ICR Contributing Author: Maria Cecilia Zanetta Page 4 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Except during the 2008-2009 period, Argentina's economy had exhibited strong growth since its collapse in December 2001. A dynamic expansion of private investment, internal consumption and exports fueled the country's recovery in the aftermath of the crisis, with real GDP growth averaging 8.8 percent per year between 2003 and 2008. The strong economic growth, in turn, helped reduced poverty from 58 to 23.4 percent, and unemployment from 21.5 to 8.3 percent in 2003 and 2007, respectively. Economic activity slowed down in 2008 due to the impact of the global financial crisis, with real GDP growth declining from 9 to 4.1 percent between 2007 and 2008. Economic growth continued to sharply decline during 2009, when it contracted by 5.9 percent, as the impact of the negative global context was further exacerbated by non-economic factors affecting the country's economic performance, including a severe drought and dengue and A/H1N1 outbreaks. The economy rebounded once again in 2010, fueled by the fast growth that resulted from an expansion of public spending, external demand and rising commodity prices, with real GDP growth reaching 10 percent. 2. Argentina continued to face a silent epidemic of chronic diseases due to the ageing of the population and increasing numbers of people practicing unhealthy behaviors. Between 1990 and 2008, Argentina improved the health status of mothers and children, reducing infant mortality rates (from 25.6 to 12.5 per 1,000 live births) and the prevalence of infectious diseases. However, there was little progress to reduce the burden of disease due to chronic diseases due to a little focus on health promotion and disease prevention. In 2009, more than 81 percent of Argentina’s burden of disease was due to non-communicable diseases (NCDs). 3. Since 2004, the Federal Health Plan (FHP) had provided the framework for the country's health sector strategy, balancing medium- and long-term objectives with the short-term priorities that had emerged from the 2001-2002 economic crisis. Specifically, the FHP was an integrated package of complementary policy reforms and actions intended to increase the effectiveness of public proceeds to improve the health status of the poor through: (i) increased access for the poorest mothers and children to basic services; (ii) enabling the National Ministry of Health (MSN) to resume its stewardship and regulatory functions in core areas of public health (i.e., essential public health functions - EPHFs); (iii) consolidate regulatory reforms in the social health insurance system to avoid negative spillovers in the sector in the provision of services for the poor and uninsured; and (iv) trigger significant changes in the relationship between national and provincial governments, as well as between provinces and health service providers. 4. The Bank's partnership with Argentina’s health sector was formulated in support of the FHP. In this context, the Bank provided support through four health operations. Two of these operations supported insurance reform through the expansion of an explicit package of health services, mainly maternal and child services, for those without social security coverage: (i) Plan Nacer I (P071025, US$135 million, approved in 2004); and (ii) Plan Nacer II (P095515, US$300 million; approved in 2006). The other two operations complemented insurance reform and supported strengthening the public health system: (i) Essential Public Health Functions Project I - FESP I (P090993, US$219 million, approved in 2006); and (ii) Prevention and Management of Influenza Type Illness and Strengthening of Argentina's Epidemiological System Project (P117377, US$141 million, approved in 2010). Page 5 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 5. The FESP I operation, in particular, served as one of the backbones of public health service reform in Argentina. It strengthened the organizational structure and institutional capacity of the MSN and provincial health ministries (MSPs) with a focus on prevention of chronic NCDs and Vector-Borne Diseases. It also introduced, for the first time in the region, an innovative output-based financing mechanism for public health transfers resources from the MSN to MSPs based on their achievement of agreed results and targets linked to the delivery of specific public health institutional outcomes and activities (so-called Actividades en Salud Pública - PHAs). 6. This operation, the Essential Public Health Functions Project II, was conceived as a follow-on to FESP I. As such, it sought to further strengthen the management and epidemiological surveillance of key programs related to chronic diseases using a result-based approach. Like its predecessor, this operation provided support for the FHP, which, on its second phase (2010-2016), continued to focus on reinforcing the MSN’s stewardship role, introducing strategies to promote better life-styles and preventive measures for selected chronic diseases, improving the management of health coverage for disadvantaged populations, and ensuring coordination at the federal and provincial levels. Theory of Change (Results Chain) Page 6 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Project Development Objectives (PDOs) 7. Project Development objectives were to: i) improve the stewardship role of the federal public health system, through the strengthening of the EPHFs; ii) increase the coverage of Priority Public Health Programs (PPHPs); and iii) increase the clinical governance of PPHPs. 1 8. EPHFs refer to core elements of public health policy that apply across all activities, independent of specific diseases or levels of interventions. These functions include: Policy Regulation; Surveillance; Monitoring and Evaluation; Health Promotion (includes Healthy Municipalities and Healthy Communities); Social Participation; Planning and Policy-making; Equity Promotion; Human Resource Development; Quality Assurance; Public Health Research; and Emergencies and Disasters. PPHPs refer to specific activities aimed at specific diseases or levels of intervention. These PPHPs include: NCDs; PROFE; Vaccine-preventable diseases; Vector-borne diseases; HIV/AIDS; Safe Blood; and Tuberculosis. Key Expected Outcomes and Key Outcome Indicators (KOIs) PDO 1 • KOI 1 - Increase in the percentage of certified departments or local territories with satisfactory or highly satisfactory epidemiological surveillance nodes from 0 in 2010 to 80 percent in 2015. • KOI 2 - Certification of at least 200 out of the 700 participating municipalities as “Health Responsible Municipalities”. PDO 2 • KOI 3 - Increase in the percentage of children under one year of age vaccinated with pentavalent vaccine from 93.6 percent in 2010 to 95.00 percent in 2015. • KOI 4 - Reduction of the prevalence of tobacco consumption in adults 18-64 years from 30 to 20 percent in 2010 to 27 percent in 2015. PDO 3 • KOI 5 - Increase in kilograms of blood-plasma produced by MSN labs from 33,000 kgs in 2010 to 40,000 kgs in 2015. Components Component 1. Strengthening of MSN’s and Provincial Ministries’ Stewardship Capacities and Improve the Public Health Infrastructure of the Priority Public Health Programs (actual US$87,8 million versus estimated US$123.2 million, equivalent to 27 percent of total loan amount). This component aimed to strengthen the national and provincial stewardship capacities focusing on the EPHF framework and selected Priority Public Health Programs (PPHPs). This component also aimed to strengthen the public health infrastructure, including human resources, for public health labs, blood banks, cold chain and supply monitoring systems. It included the following two subcomponents: 1 Note that, in the Loan Agreement, the PDOs are described as "a) improve the stewardship role of the federal public health system, through the strengthening of the EPHFs; and b) increase the coverage and clinical governance of Priority Public Health Programs (PPHPs). For the purpose of this ICR, the PDOs have been further "unpacked" in accordance to ICR guidelines (OPS5.03- GUID.140). Page 7 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 1.1. Strengthening stewardship in public health (estimated US$25.2 million, equivalent to 6 percent of total loan amount): This subcomponent focused on: a) reengineering the MSN to improve management of NCDs and PPHPs; and b) increasing the capacity of the MSN and MSPs on EPHF and PPHP management through technical assistance; specific research studies; social communication and health promotion plans; development of management and information systems and provision of technical equipment; and training; and c) promoting innovations in health promotion to reduce risk factors through: two pilots in Misiones and Tucuman provinces; and various small subprojects related to health promotion and healthy lifestyles, prevention and research to be implemented by selected NGOs, academic institutions and/or municipalities. 1.2. Modernizing the infrastructure of the public health system (estimated US$98 million, equivalent to 21 percent of total loan amount): This subcomponent focused on: a) the renovation and expansion of health infrastructure, including: expansion of the national public health laboratory network for epidemiologic surveillance; restructuring of existing blood bank network; rehabilitation of cold chain chambers under the Immunization Program; rehabilitation of the food laboratory of the National Food, Drug, and Health Technology Administration (ANMAT); and b) the modernization of supply monitoring systems, including provision of strategic health supplies for priority programs and installation of supply monitoring systems at the national, provincial and health facility level for five supply chains (i.e., blood, HIV/AIDS, STDs, tuberculosis, and vaccines). Component 2. Improve Results at the Provincial Level in Priority Public Health Programs (actual US$334,9 million versus estimated US$322 million, equivalent to 70 percent of total loan amount): This component aimed to contribute to the achievement of results in seven selected programs at the provincial and municipal levels. It included the following two subcomponents: 2.1. Provision of PHA-Eligible Operational Costs to support the implementation of PHAs (estimated US$42 million, equivalent to 9 percent of total loan amount): This subcomponent supported the following PHAs: public health surveillance activities, and other epidemiological technical systems; disease control activities; monitoring of compliance with national and provincial norms and standards including training and supervisory activities to improve public health quality; health promotion, communication, education and social participation activities with an emphasis on healthy lifestyles; local distribution of key medical supplies for public health programs; and regulatory activities carried out by the MSPs. 2.2 Improving the efficiency and performance of the Federal Health Program - PROFE (estimated US$280 million, equivalent to 61 percent of total loan amount): This subcomponent aimed to transform PROFE into a system similar to a public health insurance system with managed care that provides verified benefits to the most vulnerable populations for high-cost/low-incidence conditions and disabilities through the use of modern administrative, management and financing tools. Component 3. Administration, Monitoring and Evaluation (actual US$22 million estimated US$14.5 million, equivalent to 3 percent of total loan amount): This component provided financing for activities related to the management of the Project. It included the following two subcomponents: Page 8 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 3.1. Financing of incremental operating costs (estimated US$9.5 million, equivalent to 2 percent of total loan amount): This subcomponent financed incremental operating costs of the Coordination Unit, the MSN's International Financing Unit for Health (UFIS) and Participating Provinces, including individual consultants. 3.2 Financing of the external technical verifications and audits; and Project monitoring and evaluation systems (estimated US$5.0 million, equivalent to 1 percent of total loan amount): This subcomponent provided financing for the external technical verifications and audits (independent external technical audit) of Component 2, and the development and implementation of Project monitoring and evaluation systems for the Project. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets N.A. Revised PDO Indicators 9. During the Mid-Term, the operational definition of KOI 2 was modified, as the indicator did not have enough sensitivity to capture actual changes. Specifically, rather than including the entire universe of surveillance nodes, the revised operational definition focused only on individual nodes representing 25 percent of all the nodes and 75 percent of all reporting. Baseline and end-project targets remained unchanged. Revised Components N.A. Other Changes 10. A Level 2 Project restructuring was approved on June 13, 2016 to: i) provide a 22-month extension of the Closing Date, from June 30, 2016 to April 30, 2018 to provide sufficient time to complete the civil works; and ii) reallocation of funds between disbursement categories to realign the disbursement categories with the actual implementation of the Project. 11. A Level 2 Project restructuring was approved on April 24, 2018 to change disbursement arrangements. Specifically, the Bank financing percentage of eligible expenditures for Category 1, "Civil Works" of the Project was increased from 80 percent to 100 percent for expenses incurred starting January 1, 2018 to accelerate disbursements and make use of savings generated as a result of the devaluation of the Argentinian peso in the last nine months. In addition, the Loan’s grace period was extended by two months to allow disbursements from the Loan for withdrawal applications received by the Bank by close of business on October 31, 2018, for eligible expenditures made before the Closing Date. Rationale for Changes and Their Implication on the Original Theory of Change N.A. Page 9 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) II. OUTCOME A. RELEVANCE OF PDOs 12. The relevance of the operation's Project Development Objectives (PDOs) is deemed High. The various aspects that contribute to their relevance can be summarized as follows: • Full consistency with the Argentina's Federal Health plan (FHP) - The PDOs were fully consistent with the FHP (2004-2007; 2010-2016). Specifically, the FHP was formulated by representatives of the MNS and PMSs in the framework of the Federal Health Council (COFESA) and, as such, reflected the consensus among the key actors in the health sector at the national and provincial levels. The FHP identified: i) the main weaknesses of Argentina's health sector, including, among others, large inequalities in health services across provinces, week stewardship capacity, uneven quality of care, and scarce resources allocated to prevention; ii) priority areas of intervention in terms of institutional development, including, the systematic development of human resources, quality standards, and information systems; and iii) health priorities, including, among others, immunizations, risk factors of NDCs, safe blood, HIV/AIDs, TB, vector-borne diseases. The FHP's diagnosis of the country's health sector, its strategic vision, and the specific priorities largely informed the conceptualization of the EPHF operations. 2 • Full consistency with international best practices - The focus on EPHFs reflected the conceptual framework and international best practices identified by the World Health Organization (WHO) and Pan-American Health Organization (PAHO). 3 • Full consistency with the Bank's strategy in Argentina - The PDOs were closely aligned with the Bank's Country Partnership Strategy - CPS for FY2010-2012 (Report No. 48476-AR) discussed by the Executive Directors on June 9, 2009. Specifically, the Project's PDOs were closely aligned with two of the three pillars of the strategy: social inclusion (including the specific outcomes of consolidating improvements in health indicators); and improved governance (including the specific outcomes of strengthening public sector management and enhancing service delivery outcomes). Likewise, the operation's PDOs are also fully aligned to the current CPS FY2015–2018 (Report 81361-AR), discussed by the Executive Directors on September 9, 2014. Specifically, the Project's PDOs are consistent with the strategy's overall focus of promoting shared prosperity and reducing poverty, as well as firmly inserted within one of its three broader themes: increasing the availability of assets for people and households (including the specific result of increasing the proportion of eligible people benefiting from effective healthcare). • Adequate response to Argentina's federal structure - The emphasis of the operation's design in promoting the articulation between the national and provincial levels (and even the municipal level in the context on the Healthy Municipalities and Communities Program) provided an adequate response to the country's federal structure. Specifically, Argentina's federalism is characterized by a high degree of provincial autonomy and few mechanisms to effectively coordinate the provinces’ actions or to reduce inter‐provincial inequities in health spending. Since national spending in health health 2 The "Plan Nacer" I and II addressed other critical priorities identified in the FHP, including the maternal and child care as a priority, and the need to address the persistent inequalities persist in access and quality of health care through the development of social health insurance mechanisms. 3 http://www1.paho.org/hq/dmdocuments/2010/EPHF_Strategy_to_Strengthen_Performance.pdf Page 10 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) represented on average 50 percent of total public health spending between 2004 and 2015, 4 it is then critical to promote consensus building and introduce incentives to reduce fragmentation and increase coordination in the country’s health system. • Adequate focus on NCDs - The operations' focus on the NCDs is highly relevant given Argentina's demographic and epidemiological characteristics. As the country's population continues to age and is increasingly exposed to health risk factors, NCDs have become the main causes of death and disability. It was estimated that, in 2016, NCDs were responsible for 78 percent of all deaths in the country. 5 B. ACHIEVEMENT OF PDOs (EFFICACY) PDO I - To improve the stewardship role of the federal public health system, through the strengthening of the EPHFs - Substantial 13. Overall, the achievement of PDO 1 is rated Substantial. The Project provided support (i.e., technical assistance, training, studies and surveys, equipment, and civil works) for the implementation of key activities aimed at enhancing EPHFs (e.g., the development of norms and regulations, quality assurance mechanisms, epidemiological surveillance, health promotion through healthy municipalities development and tobacco prevalence reduction, the development and implementation of information systems to improve efficiency and transparency in patient care and supply management, and human resources management and development, among others) that are at the intrinsic to the stewardship role of the national and provincial health ministries. Although the majority of civil works financed under the operation were still under construction at closing, the operation can also be expected to contribute to the improvement of the physical infrastructure of the public health system, including the consolidation of the National Safe Blood Network through the refurbishing or construction of four regional blood banks, the expansion of capacity for vaccine and joint storage centers, and the rehabilitation of the National Biotery Lab. Finally, although intangible, one of the operation's most important contributions to PDO 1 was the implementation of the PHAs, as their implementation forced vertical programs to work in an integrated manner at both national and provincial levels. Implementing PHAs, based on national program protocols, helped unify and homogenize the implementation of service delivery. 14. The operation’s PRF reflects the substantial attainment of PDO 1, with the two KOIs measuring the attainment of this PDO being achieved or largely achieved, as well as three out of five Intermediate Result Indicators (IRIs) being either achieved or surpassing targets. Specifically: • KOI 1 - Largely achieved: The percentage of certified departments or local territories with satisfactory or highly satisfactory epidemiological surveillance nodes increased from 0 to 75 percent between 2010 and 2017 (94 percent level of achievement with respect to target of 80 percent). Although the achievement of the epidemiological surveillance network was slightly less than anticipated, the increase in this composite index represents improvements in terms of timeliness (i.e., opportunity), frequency, decentralization, and coverage of surveillance reporting by individual nodes representing 25 percent of all the nodes and 75 percent of all reporting. Moreover, in addition to the improvements to the current surveillance system, the operation provided support to the 4 Subsecretaría de Programación Económica. "Gasto Público Consolidado," November 2016. 5 World Health Organization. "Argentina: Non-communicable Diseases (NCDs)," Country Profiles, 2018. Page 11 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) development of its follow-on version (SNV.02), which is being integrated into the Health Services Information System (SISA) and is about to be rolled out. • KOI 2 - Achieved: 200 participating municipalities have been certified as Health Responsible Municipalities (100 percent level of achievement with respect to target of 200 municipalities). This represents a significant achievement in terms of promoting healthy environments that are conducive to reducing risk factors associated to NDCs. Moreover, it is the result of the effective vertical articulation of health care policies between the national, provincial as well as municipal levels. • IRI 1.1 - Surpassed: The number of intensive health care units evaluated and properly registered in the system increased from 0 to 633 between 2010 and 2017 (158 percent level of achievement with respect to target of 400 units). This achievement has important implications, not just in terms of ensuring consistent quality standards throughout the country, but also provides a valuable tool for referrals, and planning in the case of emergency and catastrophic situations. • IRI 1.3 - Not Achieved (Expected to Be Achieved in 2019): The number of national and provincial public health labs and blood banks constructed and/or rehabilitated was one as opposed to 12 as originally anticipated (8 percent level of achievement with respect to target of 12). As a result of significant delays due to weak capacity in the procurement of civil works and lack of adequate budgetary allocations, progress toward improving the public health infrastructure was drastically below expectations at the time of the operation's closing date. Specifically, only one laboratory was completed by the Project's closing date, and 12 physical works still remained under implementation, exhibiting various levels of progress ranging from 32 to 90 percent and a financial average execution of 63 percent as of October 31, 2018. It is expected that all civil works will be completed in the first half of 2019. The National Government has expressed a strong commitment toward completing these works and has formally accepted the financial responsibility for the completion of these works. • IRI 1.4 - Surpassed: A total of 1,665 health personnel from the MSN and provinces received training in public health labs, blood banks, and vaccination for more than 40 hours (208 percent level of achievement with respect to target of 800 health personnel). • IRI 1.6 - Achieved: Increased participation of provinces in training of health personnel in public health between 2010 and 2017, from 0 to 24 provinces (100 percent level of achievement with respect to target of 24 provinces). • IRI 1.9 - Not Achieved: There were no pilot studies with nutritional sprinkles component to reduce iron deficiency anemia as initially anticipated (0 percent level of achievement with respect to target of two studies). This, however, is considered a minor shortcoming. As discussed below (see Table 1), there were far more reaching achievements in the area of public health research that go beyond the impact of two individual studies. 15. The operation's outcomes under PDO 1, however, go well beyond those reflected in the PRF, as summarized in Table 1 (see Annex 1.B for a detailed description of the operation's outputs, intermediate results, and outcomes). Page 12 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Table 1. Strengthening of EPHFs under FESP II Regulation + Quality Assurance • All public and private health care facilities in the country were incorporated into the Federal Registry of Health Establishments (REFES) utilizing a single identification code (i.e., over 25,000 health care facilities, 13,000 pharmacies and 1,100 drug companies). • Health care professionals in the country were assessed and incorporated into the newly established Federal Registry of Health Care Professionals (REFEPS) utilizing a single identification code (i.e., over 850,000 health care professionals, 105,000 medical licenses, 220 health care professions, 154,000 health care specialists, as well as 1,502 teaching institutions). • Enhanced quality assurance mechanisms for health care and patient safety were implemented, including the accreditation of intensive care units. • The Sanitary Legislation Information System (LEGISALUD), which provides access to health-related legislation at the international, national, provincial and municipal level, has been implemented and is now available online. Surveillance • The performance of the National Surveillance System was significantly improved in terms of timeliness (i.e., opportunity), frequency, decentralization, and coverage of surveillance reporting by individual nodes representing 25 percent of all the nodes and 75 percent of all reporting. • Epidemiological surveillance of specific events has improved significantly, providing the opportunity for timely clinical interventions. For example, the reporting of suspected arbovirosis events (e.g., dengue and Zika) is now done automatically, at an individual level, and in an average of two days after testing, compared to the consolidated information that was manually submitted by provinces on a weekly basis in 2009. Likewise, the notification of the results of syphilis testing among expecting mothers increased from 48 percent to 81 percent of live births between 2008 and 2016. Monitoring and Evaluation • The Health Services Information System (SISA) was further developed and expanded, including the incorporation of several registers that were developed or expanded under FESP II, including: Federal Registry of Health Care Professionals (REFEPS) Federal Registry of Health Establishments (REFES) Federal Register of Nominalized Vaccination (NOMIVAC) Management system of HIV/AIDs patients (SVIH); Integrated Monitoring System of Medical Supplies (SMIS) implemented for vaccinations, blood, TB, and HIV/AIDS; Registry of Medical Residences (SNARES); and the new National Health Surveillance System (SNVS 2.0). • Other registers that were created or expanded under the FESP II include: Register of Blood Donors (REDOS); National Registry of Hemotherapy Providers; National Registry of Institutional Friends of Voluntary Blood Donors; Register of Plasma Providers; National Registry of Health Research (RENIS); and National System for Transplants (SINTRA). Health Promotion, including Communications and the Healthy Municipalities and Communities Program • Two hundred out of the 700 participating municipalities have been certified as Health Responsible Municipalities. • A total of 1,011 bakeries have joined the "Less Salt, More Life" initiative, and 81 percent of them have been evaluated and certified by the Healthy Argentina Plan. • A total of 36 new or improved health promotion agreements, and a total of 37 new regulations have been issued designed to promote healthier products, behaviors and environment. Planning and Policy Making • The systematic and standardized registration of health care professionals has made it possible for the first time to adequately determine the number, qualifications, geographical location of the country's "health care workforce," a critical input for need assessment and planning of health care delivery and workforce development. Page 13 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) • The systematic and standardized registration of private and public health care facilities has allowed to determine the number, level of service, geographical location, and other characteristics of health care facilities throughout the country. This constitutes a critical input for need assessment and planning of health care delivery. For example, the registration of Intensive Care Units for adults, children and premature babies provides a valuable tool for referrals, emergency and catastrophic situations. Human Resources Development • Enhanced access to information for decision-making as a result of the creation of the Federal Network of Human Resources in Health (RHUS) and the Federal Observatory of Human Resources in Health (OFERHUS). OFERHUS collects, produces and disseminates information on human resources within the health sector at the provincial and national levels. • Establishment and implementation of the National System of Accreditation of Medical Residences, including developing standards for 12 areas of specialization and establishing the foundations for a Single Residency Examination. • Enhanced human resources within the MSN and MSPs as a result of training. Public Health Research • Substantial strengthening of the Provincial Health Research Systems. Specifically, the level of development of Argentina's National Health Research System was considered High during the period 2011-2015, compared to Medium for the period 2005-2010 (Ibero-American Ministerial Network of Learning and Health Research - RIMAIS, 2015). PDO 2 - To increase the coverage of PPHPs 16. Achievement of PDO 2 is rated Substantial. There were robust achievements in terms of expanded coverage (i.e., prevention, diagnosis, and treatment) in six out of the seven PPHPs supported under the operation, except for the reduction in the coverage of pentavalent vaccine that was experienced during the operation's lifetime. 17. One out of two KOIs and five out of five IRIs measuring the attainment of this PDO were either achieved or surpassing targets. Specifically: • KOI 3 - Not Achieved: The percentage of children under one year of age vaccinated with the pentavalent dropped from 93.6 to 88 percent between 2010 and 2017 (0 percent level of achievement with respect to target of 95 percent). 6 Although coverage had experienced an increase between 2010 and 2015 (from 93.6 to 93.8 percent, respectively), it exhibited a drastic drop in 2016 and 2017 (see Table 2). This was the result of both "growing pains" or added burdens to the system’s capacity as part of the program enhancement efforts (i.e., the introduction of new vaccination registration practices under NOMIVAC and an increase in the number of compulsory vaccines in recent years), as well as administrative weaknesses (i.e., delays in the purchase of vaccines as a result of the authorities' decision to change providers and the procurement mechanism, as well as the lack of availability of timely budgetary resources). Although some of these factors were outside the orbit of the Project, the poor performance of this indicator points to weaknesses in the clinical governance and service delivery of the immune-preventable diseases program, which translated into what is expected to be a temporary decrease in coverage. 6 Note that the last ISR (April 30, 2018) the coverage of the pentavalent vaccination was reported as 91.9 percent for 2017, which reflected a preliminary estimation based on the number of vaccines applied regardless of the age of the children (i.e., in other words, it also included "late" vaccinations). Page 14 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Table 2. Coverage of Pentavalent Vaccine among Children under Age 1 - 2010 to 2017 Source: Directorate of Vaccination and Immune Preventable Diseases - DICEI, MSN 2018. • KOI 4 - Surpassed: The prevalence of tobacco consumption in adults 18-64 decreased from 30 to 25.1 percent between 2010 and 2017, exceeding the 27 percent end-target (163 percent level of achievement with respect to target of 27 percent). This indicator is subject to the influence of a myriad of external factors; thus, it cannot be fully attributed to this operation. However, the actions aimed at reducing tobacco consumption conducted as part of NCDs prevention activities, public health communication campaigns, and the Healthy Municipalities and Communities Program are likely to have contributed to its decline. • IRI 1.7 - Surpassed: The percentage of women between 50 and 69 years old in the two pilot provinces benefiting from at least one HPV and/or cytology test provided by the health system increased from 10 percent to 76.6 percent between 2010 and 2017 (132 percent level of achievement with respect to target of 60 percent). • IRI 1.8 - Surpassed: The percentage of women between 50 and 69 years old in the two pilot provinces benefiting from at least one mammography provided by the health system increased from 10 percent to 90 percent between 2010 and 2017 (160 percent level of achievement with respect to target of 60 percent). • IRI 2.3 - Surpassed: The percentage of PROFE renal chronic dialysis patients evaluated and included in SINTRA increased from 0 to 95.1 percent (119 percent level of achievement with respect to target of 80 percent). • IRI 2.4 - Surpassed: The percentage of PROFE newly diagnosed patients with renal chronic dialysis evaluated within six months of beginning dialysis increased from 0 to 100 percent between 2010 and 2016 (125 percent level of achievement with respect to target of 80 percent). • IRI 2.5 - Surpassed: The percentage of PROFE beneficiaries with low-incidence, high-cost diseases included in the PROFE health care chain increased from 0 to 83.3 percent (167 percent level of achievement with respect to target of 50 percent). 18. The operation's outcome under PDO 2, however, go well beyond those reflected in the PRF. As summarized in Table 3, coverage increased in all PPHPs with the exception of the immune-preventable diseases as previously discussed (see Annex 1.B for a detailed description of the operation's outputs, intermediate results, and outcomes). Table 3. Changes in coverage of PPHPs 1 - Non-Communicable Diseases • The prevalence of tobacco consumption in adults 18-64 decreased from 30 to 25.1 percent between 2010 and 2017. • Daily sodium intake / inhabitant decreased by 2.02 grams between 2009 and 2013, from 11.2 to 9.2 grams/ per capita. • Substantial increases in the percentage of women between 50 and 69 years old benefiting from cancer- prevention tests.in the two pilot provinces 2 - PROFE • The percentage of PROFE newly diagnosed patients with renal chronic dialysis evaluated within six months Page 15 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) of beginning dialysis increased from 0 to 100 percent between 2010 and 2016. • The percentage of PROFE beneficiaries with low-incidence, high-cost diseases included in the PROFE health care chain increased from 0 to 83.3 percent. 3 - Immune preventable diseases • This is the only PPHP that exhibited a decrease in coverage, as there was a significant decrease in the coverage of the pentavalent vaccine with respect to the baseline during the operation's lifetime, from 93.5 to 88 percent between 2010 and 2017. The drop of the registered vaccination is also reflected in the decrease in the percentage of applied doses relative to the total number of doses distributed, from 71 to 48.4 percent between 2016 and 2017. 4 - Vector borne diseases • Strengthening in early diagnosis and timely treatment of Chagas disease, with an increase of 38.8 percent in treatments between 2010 and 2016. • Significant improvement in prevention of vector transmission of Chagas disease, with at least 95 percent of dwellings in the endemic area subjected to entomological surveys and spraying in 52 out of 162 departments in 2017 (equivalent to 32 percent). 5 - HIV / AIDS • Argentina has the largest coverage of antiretroviral therapy in Latin America, with 81 percent of Argentines living with HIV (approximately 69,200 people) receiving antiretroviral therapy, compared to a global average coverage of 41 percent. 6 - Safe Blood • Safer blood (i.e., less risk of transmission of infections through blood transfusions), as shown by a decrease in reactive serology at the national level (i.e., positive results detecting the presence of infections) from 9% to 4.7% between 2009 and 2016. • Increased access to safe blood, with the number of the transfusion centers increasing from 244 to 861 between 2011 and 2016. 7. Tuberculosis • Significant expansion of coverage of TB treatment, with the percentage of patients tested positive for pulmonary TB patients receiving treatment increasing from 54 to 98.6 percent between 2007 and 2017. • Universal TB treatment guaranteed, with 9,000 drug treatments for sensitive and resistant TB provided in 2016. PDO 3 - To increase the clinical governance of PPHPs 19. Achievement of PDO 3 is rated Substantial, as there were robust achievements in terms of improvements in clinical governance (i.e., recognizably high standards of care, transparency and accountability for those standards, and constant improvement) 7 in seven PPHPs supported under the operation. 20. One out of one KOI and four out of six IRIs measuring the attainment of this PDO were achieved or surpassing the target. Specifically: • KOI 3 - Surpassed: Blood plasma produced by MSN and provincial labs increased from 33,000 to 43,557 kilos between 2010 and 2017 (151 percent level of achievement with respect to target of 40,000 kg). Achievements within the Safe Blood Program are notably robust, as a result of clear strategic vision and strong ownership within the program. • IRI 1.2 - Not Achieved: The number of PROFE population from Tucuman and Misiones enrolled in a 7 PAD, footnote 17, page 7. Page 16 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) clinical governance program for kidney disease prevention and control increased from 0 to 2,562 from 2010 to 2016 (64 percent level of achievement with respect to target of 4,000 PROFE beneficiaries). • IRI 1.5 - Surpassed: A total of 3 technical audits on clinical effectiveness of the diseases of low incidence and higher cost, renal disease and hemophilia were conducted between 2010 and 2016 (150 percent level of achievement with respect to target of two technical audits). • IRI 2.1 - Achieved: The number of tobacco-free municipalities increased from 0 to 142 between 2010 and 2017 (101 percent level of achievement with respect to target of 140 municipalities). • IRI 2.2 - Not Achieved: The number of PROFE women with more than 7 children having signed their letter of rights increased from 0 to 8,477 between 2010 and 2016 (21 percent level of achievement with respect to target of 40,000 PROFE female beneficiaries). The logistics behind this activity proved to be overly complex, as a large proportion of this population live in remote areas of the country. The fact that it was not achieved is considered only a minor shortcoming. • IRI 2.6 - Achieved: The percentage of operational HIV/AIDS Counseling and Testing Centers increased from 0 to 70.6 percent (101 percent level of achievement with respect to target of 70 percent). • IRI 2.7 - Surpassed: Blood donations per 1000 inhabitants over 18 years old increased from 25.6 to 30.9 percent between 2010 and 2017 (106 percent level of achievement with respect to target of 30.6 percent). 21. The operation's outcomes under PDO3, however, go well beyond those reflected in the PRF. As summarized in Table 4, there were significant improvements in the clinical governance in the seven PPHPs supported under FESP II (see Annex 1.B for a detailed description of the operation's outputs, intermediate results, and outcomes). Table 4. Improvements in clinical governance of PPHPs 1 - Non-Communicable Diseases • In 2012, Argentina enacted one of the first sodium reduction laws in the world. • The number of tobacco-free municipalities increased from 0 to 142 between 2010 and 2017. • 74 percent of the provinces have elaborated their quarterly report with the information of all the cases of injury by external causes and loaded it onto SISA. • A total of 5,831 Health Centers for Primary Care (CAPS) have been assessed and certified regarding their approach to NCDs. 2 - PROFE • The number of PROFE population from Tucuman and Misiones enrolled in a clinical governance program for kidney disease prevention and control increased from 0 to 4,000 from 2010 to 2016. • A total of 3 technical audits on clinical effectiveness of the diseases of low incidence and higher cost, renal disease and hemophilia were conducted between 2010 and 2016. • The number of PROFE women with more than 7 children having signed their letter of rights increased from 0 to 8,477 between 2010 and 2016. • The percentage of PROFE renal chronic dialysis patients evaluated and included in the National Transplants Information System (SINTRA) increased from 0 to 95.1 percent. 3 - Immune preventable diseases • Federal Register of Nominalized Vaccination (NOMIVAC) is now operational in 22 provinces, with over 40 percent of vaccination events registered in NOMIVAC in 2017. The implementation of this registry constitutes a significant advance, as it records vaccinations at the individual level (i.e., infant) including his/her physical address. • Enhanced M&E of vaccines due to the utilization of the SMIS. Page 17 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 4 - Vector borne diseases Public health outcomes • Significant progress in vector control of Chagas disease, with two provinces (i.e., San Luis and Tucumán) succeeding in interrupting vector transmission. Clinical governance • Strengthening in the reporting of Chagas events, with reporting coverage of pregnant women tested for Chagas in the SNVS averaging 60 percent in the last 3 years (2014-2017). • Epidemiological surveillance of mosquito-transmitted diseases has improved significantly, providing the opportunity for timely clinical interventions. Specifically, the reporting of suspected arbovirosis events (e.g., dengue and Zika) is now done automatically, at an individual level, on an average of two days after testing, compared to the consolidated information that was manually submitted by provinces on a weekly basis in 2009. • System of real-time notifications of routine and emergency actions of the personnel of the National Coordination of Vector Control (CNCV) was implemented under the Project and is operational. Particularly during outbreaks, it is a critical tool to strengthen emergency management, allowing for evidence-based decision making and rapid resource deployment. 5 - HIV / AIDS • The percentage of operational HIV/AIDS Counseling and Testing Centers increased from 0 to 70.6 percent between 2010 and 2017. • As a result of the implementation of the HIV Patient Management System (SVIH), the application and approval times of both antiretroviral treatments and viral load authorizations have been reduced substantially from 15 days to 48 hours, and from 30 days to less than a day, respectively, between 2010 and 2017. • Enhanced M&E of HIV/AIDs medications due to the utilization of the SMIS. 6 - Safe Blood • Blood plasma produced by MSN and provincial labs increased from 33,000 to 43,557 kilos between 2010 and 2017, exceeding the 40,000 kg end-target. • Blood donations per 1000 inhabitants over 18 years old increased from 25.6 to 30.9 percent between 2010 and 2017. • More effective management of blood intake, as shown by the decrease in the percentage of deferred blood donors from 21% to 17% between 2009 and 2017. • Enhanced efficiency in blood processing, as shown by: i) the increase in the percentage of transfusions of fractionated blood units from 84% to 95.3% between 2009 and 2016; and ii) the decrease in transfusions of whole or unprocessed blood from 8,406 units in 2009 (1.6% of transfusions) to 1,160 units (0.17% of transfusions) in 2016. • Restructuring of the blood network, including: i) reducing the number of hospital-based blood production centers from 238 to 147 between 2011 and 2016; and ii) increasing the number of centralized blood center from 27 to 38 during the same period. • Enhanced M&E of blood production supplies due to the utilization of the SMIS. 7. Tuberculosis • Enhanced M&E of TB medications due to the utilization of the SMIS. Justification of Overall Efficacy Rating 22. As a whole, the Project's achievement of objectives is deemed Substantial. The Project resulted in a robust set of outcomes in the three areas of intervention (i.e., strengthened stewardship of the federal public health system, expanded coverage of PPHPs, and enhanced clinical governance of PPHPs). As shown on Table 5, the overall level of achievement amounts to 90 percent when taking into Page 18 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) consideration the achievement of the indicators included in the PRF. However, as documented in the context of each PDO, the operation's outcomes well exceed those reflected in the PRF. Table 5. Justification of Overall Efficacy Rating C. EFFICIENCY Assessment of Efficiency and Rating 23. The operation's economic efficiency is deemed Substantial to reflect its High economic efficiency and Modest implementation efficiency. Specifically: 24. Economic efficiency: The operation's economic efficiency is deemed High. The economic analysis performed at Appraisal showed that the Project was expected to yield a net present value of benefits, after investment and recurrent costs, of about US$9.1 billion, and produce an internal rate of return above 50 percent over a ten-year period. In addition to project costs of US$461 million, the analysis included the costs of capital and recurrent expenditures related to the management of the seven selected programs, to be sustained once the main interventions of the Project were completed. Project interventions were anticipated to produce substantial savings over the medium to long-term. Savings were expected to accrue through reduction in morbidity and mortality rates associated with communicable and non-communicable diseases, reduction of the risk factors, and corresponding savings in hospital costs, outpatient costs, and treatment costs. 25. In the case of the non-accrued benefits from vaccination, there are several factors that indicate that they might not be considerable over a ten-year horizon. First, it can be expected that the actual drop in coverage is less than the one shown in the data due to misreporting as a result of the transition to the NOMIVAC register. Reporting is expected to improve once this transition is completed. Second, the actual coverage of pentavalent vaccination increases once late vaccination events included (i.e., while the coverage of on-time vaccination events was 88 percent in 2017, it increases to 89.5 percent when all infants one-year old or less are included, regardless of the timing of the vaccination as long as it took placed within the first 12 months after birth. Third, the drop exhibited in 2016 and 2017 is likely to be reversed in 2018, as the MSN now has experience with the purchases of vaccines from the new providers. Moreover, the forgone economic benefits anticipated as a result of increased coverage in pentavalent vaccinations are offset by the benefits from other Project outcomes that were not included in the original economic evaluation, including benefits from NCD-prevention activities, Chagas- Page 19 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) prevention activities, increased efficiency from the reorganization of the blood production network, and the implementation of SMIS. (See Annex 4 for a detailed analysis of the operation's economic efficiency). 26. Implementation efficiency: The operation's implementation efficiency is deemed Modest. As mentioned earlier, the closing date was extended 22 months to allow sufficient time for the implementation of civil works, which, in turn, had experienced considerable procurement delays. While one of the planned civil works was completed at operation's closing date, the remaining works were still under implementation and expected to be completed in their totality in early 2019. From an efficiency perspective, these delays have resulted in the deferral of the economic benefits associated with these civil works. As mentioned above, savings from the increased efficiency of the restructuring of the blood network are estimated at US$5 million per year once the civil works are completed. (See Annex 4 for a detailed analysis of the operation's implementation efficiency). D. JUSTIFICATION OF OVERALL OUTCOME RATING 27. The operation's overall outcome rating is deemed Satisfactory to reflect its High Relevance, and Substantial Efficacy and Efficiency. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 28. Although the operation did not seek a specific impact on gender, in the framework of the two provincial pilots, it put in place explicit warranties to entitle PROFE female beneficiaries to health prevention services (such as cervical cancer screening, reproductive health and STD/HIV/AIDS). Institutional Strengthening 29. The operation had a strong focus on strengthening stewardship and governance at both the national and provincial levels. It also had a significant impact on enhancing vertical and horizontal coordination within the country's health sector (i.e., national and provincial levels, and across provinces, respectively). Enhancing coordination is particularly important in the context of Argentina's federalism, as the MSN has narrow control over the MSPs, including limited mechanisms for strengthening and expanding health intelligence; establishing and enforcing standardized norms and protocols; and establishing information systems. In this regard, the operation helped expand the MSN's influence over the MSPs by fostering consensus building and buy-in on the part of the provinces with regard to specific actions (e.g., national and regional meetings and workshops for planning and coordination, training, field visits to provinces) as well as control over their implementation (i.e., PHAs). Likewise, the implementation of PHAs based on national program protocols, helped unify and homogenize the implementation of service delivery. At the local level, the implementation of PHAs forced programs to better organize their activities and to improve the reporting of service delivery outputs by systematizing the information and improving its accuracy. Finally, having a third-party evaluation of PHAs resulted in increased accountability of reporting. Page 20 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Mobilizing Private Sector Financing N.A. Poverty Reduction and Shared Prosperity 30. PROFE, one of the Priority Health programs supported under the operation, targets the most vulnerable groups, including women with seven children or more and their family, the disabled, and people over 70 years without social security income. In addition, the Project activities aimed at supporting improvements in NCD-related services and protecting against prevalent health risk factors, since the burden of disease associated with NCDs affects the vulnerable population disproportionately. Finally, the unification of norms and quality standards helped reduce inequalities among provinces, as did the targeted focus on specific PHAs through the PHA mechanism. Other Unintended Outcomes and Impacts N.A. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 31. The operation's design exhibited significant strengths, including: • As discussed earlier, the operation benefitted from having highly relevant PDOs and design. It represented the natural extension of its predecessor (FESP I) and was fully consistent with the long term IBRD support strategy over the previous decade as well as the overall consensus regarding the challenges facing Argentina's health sector and the strategic vision for the medium- term. • Interventions at the provincial level were structured around a performance-based element, in which shortfalls in achievements of agreed targets by provinces resulted in reduced allocations. While the financial incentives were not significant as a share of the provincial health budgets, they constituted a significant proportion of their discretionary spending. As a result, these incentives resulted in powerful mechanisms to facilitate the articulation of the national and provincial levels. • While the inclusion of all 24 provincial jurisdictions for each of the EPHFs and PPHPs posed significant implementation challenges, it served partly as a risk diversification mechanism (i.e., assuming uneven levels of commitment and technical capacity), it was well suited to Argentina's federal organization. • Except for the procurement, FM functions, and safeguards, the operation was implemented within the regular structure of the NMH and PMHs, thus promoting in-house capacity building and ownership. • The design incorporated lessons learned from other operations and international experience. However, the FEPS I was still under implementation during FESP preparation (in fact, the implementation of the two operations overlapped for 16 months). 8 This may have precluded from fully capitalizing on the lessons learned from FESP I. 8 The closing date of FESP I was December 31, 2012, while FESP II was declared effective on August 3, 2011. Page 21 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 32. Despite the operation's overall sound design, some weaknesses can be identified in hindsight, including: • The operation's scope was likely too ambitious. Specifically, the operation included a total of 18 vertical areas of intervention (i.e., 11 EPHFs and seven PPHPs), which presented a challenge for coordination and supervision at the provincial level, particularly in those provinces lacking adequate capacity. In turn, each vertical area of intervention required coordination and implementation with each of the 24 participating provinces, which presented a challenge for coordination and supervision at the national level. When combining the vertical and provincial dimensions, the operation involved 432 sub-sets of intervention activities, which posed a significant challenge for both the Coordinating Unit (CU) as well as the Bank Team. 9 B. KEY FACTORS DURING IMPLEMENTATION • Recurrent changes of authorities (ministers, secretaries, under-secretaries) and program coordinators at both the national and provincial levels, at the very least, slowed down implementation and often resulted in changes in the specific priorities assigned with the operation. In particular, there were two presidential elections during the operation's lifetime (i.e., 2011 and 2015), which resulted in a change in the national authorities in December 2015. In addition, provincial authorities changed at least once in 17 out of the 24 provincial jurisdictions. In terms of health authorities, the operation was implemented under three different ministries at the national level and at least 62 ministers at the provincial level. • Weak procurement capacity for civil works resulted in significant delays and constituted an important implementation bottleneck. In addition, there were insufficient budgetary allocations to the Project during 2017 and 2018 due to tight fiscal conditions. This, in turn, resulted in delayed payments for civil works, further exacerbating their implementation delays. Prior to the operation’s closing date, national health authorities requested and successfully obtained additional government resources to ensure the full completion of the civil works still under construction. • Results-based transfers to provincial governments provided an effective mechanism to improve performance. First, it provided incentives to provincial units for attaining the agreed targets. Second, the fact that a third party was responsible for verifying results through a technical audit enhanced transparency and helped ensure the consistent adoption of standards and protocols across provinces. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 33. The operation's M&E system was robust, as it built upon the one developed under FESP I. Specifically, the M&E system focused on: i) collecting routine national health statistics; ii) monitoring of public medical supplies through the web-based SSMI; vi) conducting national epidemiological surveillance; 9 In addition, there were two pilot projects in Misiones and Tucuman, as well as some activities with other individual government agencies. Page 22 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) and iv) undertaking external audits for measuring results at provincial level. The operation's PRF also exhibited some weaknesses, as follows: • PDO definitions - PDO 1 called for "improving the stewardship role of the federal public health system, through the strengthening of EPHFs." The reliance on a broad concept such as "stewardship" without including a specific definition introduces ambiguity and makes the evaluation more challenging. Likewise, it is not clear whether the federal public health system refers to both the national level and provincial levels. The evaluation of PDO1 under this ICR focuses on the strengthening of EPHFs at the national level and the public health care infrastructure, which, as a whole, are considered to result in enhanced stewardship. Similarly, PDO2 called for "increasing the coverage and clinical governance of PPHPs." In this case, however, a definition of clinical governance was included in a footnote, 10 and was implicitly operationalized in the operation's indicators. • Indicators - It would have been important to include an outcome indicator reflecting improvements in PROFE's clinical governance as a result of the operation. Such indicator would have been particularly important given than PROFE-related financing accounted for 61 percent of the total loan proceeds. In addition, the outcome indicator focusing on tobacco use, which was reportedly included mainly to monitor the trend in tobacco consumption in Argentina rather than to measure the impact of the FESP II, should not have been included in the PRF, as it depends on a myriad of factors outside the scope of the operation (such as the success of individual marketing campaigns by tobacco companies). Finally, operational definitions for both the KOIs and Intermediate Results Indicators (IRIs) were included in the PAD, which is considered to be a best practice to be replicated. 34. It was also envisioned that the operation would finance an impact evaluation, adopting a variety of strategies and use triangulation methods to assess the impact of the Project. The second National Risk Factor Survey 2009 provided the baseline data, and funding for a follow-up survey was included for an evaluation during the final year of implementation. The evaluation also contemplated the possibility of including a specific module to evaluate prevention activities and assess variations in coverage under the seven selected health programs. M&E Implementation 35. M&E capacity in the Coordinating Unit (CU) was robust, and Project activities were well monitored. The CU adequately fulfilled its responsibilities, ensuring a properly functioning M&E system and supervising participating provinces and national programs to ensure the accuracy, timely gathering, analysis, and dissemination of information. Overall, project implementation institutions (e.g., provinces, health providers and decentralized sector agencies) adequately performed their duties for the collection of indicators and results for activities under their responsibility and submitting progress reports and other project information to the CU. Progress reports were well prepared. Mid-term and end-of-project reports were thorough and insightful. Finally, third-party technical audits were utilized to monitor project outputs and verify payment-related information at the provincial level. 10 "Clinical governance has three dimensions: recognizably high standards of care, transparency and accountability for those standards, and constant improvement". (PAD, footnote 17, page 7). Page 23 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 36. The impact evaluation that was originally envisioned as a tool to assess the impact of the Project rendered very limited results due to methodological weaknesses. A third Risk Factor Survey was conducted in 2013 to assess changes in the prevalence of risk factors. While the changes could not be attributed to the activities supported by the operation, there were useful in providing a general indication of the overall trends. M&E Utilization 37. The Project's M&E system was essential throughout implementation, as it allowed monitoring progress toward the implementation of activities as well as PDO achievement. The Project's M&E system also served as the de facto M&E system for the MSN, as it was the only M&E tool to consolidate provincial data on activities being carried out by the national directorates and program units. In addition, the Project's M&E system supported the MSN in: (i) the compilation of traditional national statistics of the health sector, (ii) the monitoring of supplies for public health, (iii) the execution of epidemiological surveillance at the national level, and (iv) the performance of external audits for the measurement of results at the provincial level. Justification of Overall Rating of Quality of M&E 38. The overall quality of the M&E is deemed Substantial. M&E systems were robust, and reporting and monitoring during the life of the operation was adequate both at the national and provincial levels. The utilization of the M&E system proved to be valuable beyond the context of the operation, as it served to consolidate provincial data on activities being carried out by the national directorates and program units. Having a more robust impact evaluation as originally anticipated would have further strengthened the assessment of the operation's impact as well as the opportunity to generate more in-depth learning from its implementation experience. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Fiduciary Financial management implementation support. 39. The Project team had a constructive engagement with the Bank on financial management (FM) aspects throughout the entire implementation period. FM advice was provided in a timely manner. Moderate FM shortcomings were identified during supervision missions mainly focusing on delays and quality of Interim Financial Reports (IFRs) submission. Agreed actions were properly addressed by the Project team and, as a result of this effort, quality of IFRs was improved. Overall FM ISR rating of the Project has ranged from satisfactory to moderately satisfactory. 40. Disbursements. The Project disbursed through IFRs submitted to the Bank within 45 days of the quarter end. All IFRs were reviewed and found acceptable. The Ministry of Economy and Public Finance, through its Directorate of Projects Financed by Multilateral Organizations, controlled Designated Account B. Designated Account A was managed by the Ministry of Health. Disbursements reached to US$ 446,725,757.08 (97percent of the loan resources) which includes an advanced to the Designated Account A that will be returned to the Bank. The Project closed on April 30, 2018. Loan’s grace period was extended Page 24 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) for two months to allow documentation of eligible expenditures. Advances to the designated accounts have been fully documented except for the outstanding balance of US$939,864.02 which has been returned to the Bank. An amount of US$15,214,106.90 has been cancelled corresponding to non- executed funds. 41. Auditing arrangements. (i) Financial Statements Audits. The Project financial statements audits have been carried out by the Argentina Supreme Audit Institution - Auditoria General de la Nación (AGN) since the Project's inception. Audit reports were received by the Bank with some delay; they were reviewed and found acceptable; and no accountability issues arose throughout the entire Project's lifetime. The 2017 audited financial statements were due on June 30, 2018 and are still pending to be submitted to the Bank. Follow up on the overdue report is underway and 2017 audit report is expected to be submitted to the Bank not later than November 30, 2018. Besides, the last audit report, covering the period from January 1, 2018 to the closing date, including the grace period ended on October 31, 2018, is expected to be submitted to the Bank not later than April 30, 2019. (ii) Technical audit. In addition to financial audits, the project had external technical audits carried out by independent audit firms to validate delivery of selected sample of outputs (Public Health Activities, category #5); reliability of the unit cost of outputs; enrolled beneficiaries and capitation payments under PROFE (Component 2.2, category #6). Technical audit reports were reviewed on a sample basis whenever outputs were documented to the Bank accompanying the submission of IFR. Procurement 42. The procurement performance of the Project was highly influenced by various changes in the management of UFIS and high staff rotation. In this context, a challenge to Project implementation was the need to improve the quality of the procurement documents, which had a direct impact on implementation efficiency. To address these challenges, procurement supervision included standard prior review, an annual post review mission to identify areas of improvement and specific trainings to strengthen Project implementation and shorten the learning curve of new staff. 43. The last two years of implementation were primarily focused on executing small civil works contracts for which the Project Implementation Unit (PIU) made significant efforts to procure and implement on time. To ensure that these national procedures were carried out efficiently and in compliance with the Bank’s Guidelines, the Bank Team worked closely with the procurement staff at the PIU to develop a standard bidding document that was applied in all the civil works contracts. The PIU also prepared, at the request of the Bank, a plan to make sure civil works were delivered and supervised on time. Despite these actions, there were delays in the implementation of the procurement processes and deviations during contract execution. 44. Regarding the implementation of goods contracts, and as a result of a post review, the need to strengthen contract supervision was identified to ensure they are delivered on time to the identified places. To this end, UFIS put in place a detailed action plan which included a close supervision of these aspects. Though the results of such plan are not visible for this operation, the remaining portfolio implemented by this PIU with be highly benefited by these actions. Page 25 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Environmental 45. The creation in the MSN of a Technical Area of Environmental Safeguard (TAES), responsible for assisting the MSN and MSPs, was a key tool for the execution of the Project and for the sustainability of the results. 46. The performance rating of the safeguards for OP / BP 4.01 was (HS) Highly satisfactory. The capacity of the TAES was adequate and was strengthened with the incorporation of personnel and with the articulation with other health projects financed by the Bank. The Safeguards Performance Rating for OP / BP 4.09: (S) Satisfactory. No pesticides have been acquired by the Project, nor have they been reimbursed according to the provincial subprojects (the acquisition was not admissible). 47. In response to the warning about the increased risk of Dengue transmission in Argentina, the MSN generated a document on the "Guidelines for the prevention and control of Aedes aegypti". In this way the use of pesticides decreased significantly. 48. The TAES elaborated and adopted different tools for the waste management in health establishments, establishing a standard for the entire country, which led to its adoption and implementation at the subnational level. An adequate budget allowed the TAES to strengthen the articulation of the MSN with the subnational levels and between the technical areas of the MSPs. 49. The TAES provided the technical assistance to the MSPs to promote the creation of Health and Environment areas, and they have contributed to the sustainability of the technical training of the provinces, (face-to-face training workshops; Virtual Courses of Integral waste management in health establishments, and manuals for the management of works in health establishments). 50. There were several lessons learned in the area of environmental safeguards, including: • The institutional strengthening and sustainability of the Provincial Environmental Health units depend on being integrated into the structure of the PHM, and that they have an adequate budget from the start to guarantee the timely delivery of supplies and services, and continuously training of the staff (motivated by high turnover characteristic of this activity). • The realization of National Meetings, virtual training, and the availability of common information reservoirs on the MSN website are an opportunity and a tool to unify criteria and monitoring methodologies, indicators and indices, which allow monitoring of the waste management in health establishments. • The audit carried out by the TAES in the provinces contributed to the decision making by the key actors involved in of the political level. Social 51. The Project financed activities addressing Indigenous Peoples (OP 4.10) in a satisfactory manner. Building on the achievements and the experience of FESP I and working on an integrated approach with other WB and IDB Projects, institutional progress was made regarding the indigenous Page 26 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) health agenda. Still, efforts will be needed within other ongoing Health Projects to enhance the highly uneven provincial performance. 52. An Indigenous Peoples Planning Framework (IPPF) was prepared in consultation with the Indigenous Participation Council of the National Institute for Indigenous Affairs. The IPPF focused on institutional strengthening at provincial and municipal level, using specific PHAs to incentive MSPs’ commitment. 53. At the provincial level, the Project lead to the creation of 14 Provincial Indigenous Health Units. Nineteen out of the 20 provinces that activated the safeguard had a specific indigenous health area at the end of the project and it is a requirement for the three provinces that will be incorporated at the Supporting Effective Universal Health Coverage in Argentina Project (P163345) to have a specific indigenous health area. Under FESP II these areas were in charge of developing a social assessment on a biannual basis, and designing, consulting and implementing an annual Indigenous Peoples Plan (IPP). 54. Despite the fact that the Project’s national team worked on regular technical assistance activities and carried out regular results monitoring actions, performance of activities for indigenous peoples under the Project was highly uneven between provinces: IPPs execution results were on average 64 percent but ranged between 100 percent (in the case of 6 provinces) and 0 percent (in the case of 2 provinces). 55. Nevertheless, the Project achieved some positive results. At the local level, 495 intercultural workshops were executed on different indigenous heath issues, reaching 12,339 people including indigenous community members, health staff and health officials. These workshops were held in disperse rural indigenous communities and in health institutions with indigenous people in charge. During the first two years of the project, 23 communities were incorporated at Healthy Municipalities and Communities Program, but as this activity was difficult to implement, focus was then changed to health establishment’s intercultural adaptation activities that were developed in 10 provinces. IPPF originally included the provision of energy and communication equipment for disperse community’s health centers, but this activity was cancelled. During the last year of the Project four trucks were acquired instead to facilitate provincial unit’s access to disperse communities. The MSN has not yet received two of them in order to deliver them to the correspondent MSP. 56. On 2017, all safeguards teams working on WB financed projects were transferred to the UFIS, which facilitated an integrated approach and allows now to continue with actions regarding IP health within other projects, like Supporting EUHC in Argentina (P163345) and AR Chronic Conditions and Injuries (P133193). C. BANK PERFORMANCE Quality at Entry 57. Bank's performance in ensuring Quality at Entry is assessed as Satisfactory. Overall, the operation was highly relevant, as it was based on a thorough knowledge of Argentina's health sector and was in line with the GOA’s FHP and the Bank's CPS. Project design built upon that of the FESP I and incorporated some modifications to reflect changes in implementation conditions. As a whole, Page 27 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) implementation readiness was adequate, except in the case of the two pilots, which were not central to PDO achievement. The scope may have been overly ambitious, particularly when taking into consideration the risks of weak implementation capacity and the high turnover among health officials and authorities at both the national and provincial levels. However, the broad scope also served as a risk diversification mechanism, providing the flexibility needed to adjust to changes in the relative priorities throughout the operation's lifetime. Quality of Supervision 58. Bank's performance in supervision is deemed Satisfactory. The Project was intensively supervised, and timely and accurate supervision reports were provided. The same Team involved in preparation remained on board through the operation's lifetime, which provided continuity. Although there was a change in the Team's leadership, the transition was seamless. The fact that the key technical staff were based in Argentina provided strong advantages in terms of their in-depth sector knowledge, enhanced dialogue with the Borrower and ongoing support, as well as more swift identification and resolution of problems as they emerged. The team also played a key role in transferring knowledge and instilling ownership among incoming provincial and national health officials and authorities that were appointed as a result of changes of leadership, including (but not limited to) the 2015 elections. Project management and fiduciary challenges were also met in a timely and effective manner. The country-based fiduciary team worked closely, enhancing its effectiveness. It developed and closely supervised the Governance and Accountability Action Plan (GAAP), leading to improvements in FM and Procurement. Likewise, problems in Project management were flagged early, and solutions were agreed with Government. Although Bank worked closely with the Borrower to address implementation bottlenecks arising from weaknesses in the procurement of civil works, progress was temporarily interrupted in 2015- 2016 during the transition in national authorities. Safeguards (environmental and indigenous peoples) were also adequately supervised. Justification of Overall Rating of Bank Performance 59. Given the satisfactory performance in ensuring quality at entry and adequate supervision, Bank performance is deemed Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 60. Risk to the sustainability of the Project's outcomes is deemed Substantial. Although there has been notable progress toward the institutionalization of the some of the Project's outcomes, significant risks hindering the sustainability of the operation's outcomes still persist. 61. Some of the most notable progress toward the institutionalization include: i) a large proportion of the human resources hired as part of the operation's technical assistance have been formally transferred to the MSN; ii) the national directorate responsible for the Safe Blood program is adopting the PHA results- based mechanism to allocate its own budgetary resources to the provinces; iii) the Safeguards Unit that was created under the operation has been permanently incorporated into the UFI; thus, all safeguards procedures will be systematically applied to all programs with external financing; and iv) procurement and FM are moving out of the UFI to be a part of the regular administration of the MSN. Page 28 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 62. There are, however, significant risks affecting sustainability, including: i) the continuous improvement of the clinical accountability and efficiency gains of the PROFE Program, which has been transferred outside the orbit of the MSN and is now conceived as a program to serve the disabled; and ii) the budgetary constraints posed by macroeconomic conditions, which limits budgetary allocations to the health sector. 63. The structural lack of mechanisms to ensure national-provincial coordination as well as provincial compliance with national norms, standards, and reporting constitutes perhaps the most pervasive hindrance to ensure the long-term sustainability of the operation's outcomes. 11 In this regard, there are several Bank- financed operations that will help ensure this coordination within their specific orbits. 12 With respect to those areas of intervention that are not captured by any specific project, the sustainability will depend on the continuous efforts to build consensus and buy-in on the part of provinces from the corresponding directorate or program management unit at the national level. V. LESSONS AND RECOMMENDATIONS 64. Focusing on essential public health functions can lead to the improved effectiveness of public health programs. Both FESP operations sought to simultaneously address weaknesses in EPHFs at the national and provincial levels while ensuring the delivery of critical health services under PPHPs. This dual approach resulted in EPHFs being treated both as independent pillars as well as cross-cutting themes incorporated into the various PPHPs. This, in turn, created synergies that helped achieved the intended Project outcomes in terms of both stewardship and service delivery. 65. Results-based financing has proved to be an effective mechanism for delivering services to beneficiaries and supporting institutional development. Setting targets and linking disbursement to results helps strengthens governance and accountability and contributes to developing a public-sector culture that rewards performance. 66. The operation successfully promoted the articulation between the national and provincial levels by supporting consensus building and collaboration between the national and provincial levels. Through "soft" activities (e.g., symposiums and conferences, joint working sessions, seminars and workshops, filed visits) that fostered periodic face-to-face contact between national and provincial counterparts helped build consensus over the specific strategies and coordinate their implementation, thus provided an adequate response to the country's federal structure. 11 NOMIVAC serves as an illustration of the need to ensure provincial beyond the operation's lifetime. Specifically, the implementation of this registry constitutes a significant achievement, as it records vaccinations at the level of the individual newborn, including his/her physical address. However, its implementation is yet to be consolidated, as the percentage of live births registered in NOMIVAC in relation to the total number of live births still exhibits large fluctuations from year to year (e.g., it decreased from 70 and 57 percent in 2016 and 2017, respectively). 12 These operations include: the Supporting Effective Universal Health Coverage Project (P163345; US$658 million; approved on May 18, 2018); the Protecting Vulnerable People Against Non-communicable Diseases Project (P133193; US$437.5 million; approved on June 11, 2015); and the Provincial Public Health Insurance Development Project (P154431; US$200 million; approved on July 7, 2015). Page 29 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 67. In federal contexts, Project design has to account for disparities exhibited by sub-national governments in terms of their socio-economic, geographic and demographic characteristics as well as their fiscal and institutional capacity. In the case of the FESP operations, the negotiation of annual targets (brechas) for individual PHAs between national directorates and/or national units responsible for PPHPs provided the flexibility needed to adjust to institutional capacities of different provinces and changing priorities. 68. The signing of 5-year umbrella agreements between federal and provincial levels outlining each party’s responsibilities helped protect against political risk. The same mechanism could potentially be used to ameliorate political risk at the national level, by which national directorates and PPHPs formally subscribe agreements under the operation reflecting their responsibilities during the operation's lifetime. 69. Project evaluation methodologies need to be defined during preparation to ensure the needed information is available. Waiting until after the Project has been under implementation for some time (or near its completion) hinders the ability to carry out a rigorous analysis. The relevance and usefulness of the Project Impact Evaluation was very limited as a result of difficulties with the data. . Page 30 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased kilograms of blood- Number 33000.00 40000.00 40000.00 43557.00 plasma produced by MSN and provincial labs 20-Dec-2010 30-Jun-2016 30-Apr-2018 30-Mar-2018 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Reduction of the tobacco Percentage 30.00 27.00 27.00 25.10 consumption prevalence in adults 18-64 20-Dec-2010 30-Jun-2016 30-Apr-2018 30-Mar-2018 Comments (achievements against targets): Page 31 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion At least 200 out of the 700 Number 0.00 200.00 200.00 200.00 participating municipalities certified as Health 20-Dec-2010 30-Jun-2016 30-Apr-2018 30-Mar-2018 Responsible Municipalities Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased percentage of Percentage 0.00 80.00 80.00 75.00 certified departments or local territories with satisfactory or 20-Dec-2010 30-Jun-2016 30-Apr-2018 30-Mar-2018 highly satisfactory epidemiological surveillance nodes (C2). Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 32 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Increased percentage of Percentage 93.60 95.00 95.00 88.00 children under one year of age vaccinated with 20-Dec-2010 30-Jun-2016 30-Apr-2018 30-Apr-2018 pentavalent vaccine. Comments (achievements against targets): A.2 Intermediate Results Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased participation of Number 0.00 24.00 24.00 24.00 provinces in training of health personnel in public 20-Dec-2010 30-Jun-2016 30-Apr-2018 31-Oct-2017 health Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of pilot studies with Number 0.00 2.00 2.00 0.00 nutritional sprinkles component (to reduce iron 20-Dec-2010 30-Jun-2016 30-Apr-2018 31-Oct-2017 Page 33 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) deficiency anemia) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Health personnel receiving Number 0.00 800.00 800.00 1665.00 training (number) 31-Mar-2011 30-Jun-2016 30-Apr-2018 31-Oct-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of women Percentage 10.00 60.00 60.00 90.80 between 40 and 70 years old in the province of Tucuman 20-Dec-2010 30-Jun-2016 30-Apr-2018 31-Oct-2017 benefiting from at least one mammography provided by the health system. Percentage of women Percentage 10.00 60.00 60.00 76.60 between 35 and 64 in Tucuman benefiting from at 01-Nov-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 least one PHV and/or Page 34 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) citology test provided by the health system Percentage of women Percentage 10.00 60.00 60.00 76.60 between 35 and 64 in Misiones benefiting from at 01-Nov-2010 30-Jun-2016 30-Apr-2018 31-Oct-2017 least one PHV and/or citology test provided by the health system Percentage of women Percentage 10.00 60.00 60.00 90.80 between 50 and 69 years old in the province of 01-Nov-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 Misiones benefiting from at least one mammography provided by the health system. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased number of PROFE Number 0.00 4000.00 4000.00 2562.00 population from Tucuman and Misiones enrolled in a 20-Dec-2010 30-Jun-2016 30-Apr-2018 31-Oct-2017 clinical governance program for kidney disease prevention Page 35 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) and control. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of intensive health Number 0.00 400.00 400.00 633.00 care units evaluated and properly registered in the 20-Dec-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 system. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Blood donations per 1000 Percentage 25.60 30.60 30.60 30.90 inhabitants over 18 years old 20-Dec-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 Comments (achievements against targets): Page 36 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of operational Percentage 0.00 70.00 70.00 70.60 HIV/AIDS Counseling and Testing Centers. 20-Dec-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of PROFE Percentage 0.00 50.00 50.00 83.30 beneficiaries with low incidence high cost diseases 20-Dec-2010 30-Jun-2016 30-Apr-2018 31-Oct-2017 included in the PROFE health care chain. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of PROFE newly Percentage 0.00 80.00 80.00 100.00 diagnosed patients with renal chronic dialysis 20-Dec-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 Page 37 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) evaluated within six months of beginning dialysis. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of PROFE renal Percentage 0.00 80.00 80.00 95.10 chronic dialysis patients evaluated and included in 20-Dec-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 SINTRA. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of PROFE women Number 0.00 40000.00 40000.00 8477.00 with more than 7 children having signed their letter of 20-Dec-2010 30-Jun-2016 30-Apr-2018 31-Oct-2017 rights. Comments (achievements against targets): Page 38 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of tobacco free Number 0.00 140.00 140.00 142.00 municipalities. 20-Dec-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of technical audits Number 0.00 2.00 2.00 3.00 on clinical effectiveness of the diseases of low incidence 20-Dec-2010 30-Jun-2016 30-Apr-2018 31-Oct-2017 and higher cost, renal disease and haemophilia. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Health facilities constructed, Number 0.00 12.00 12.00 1.00 Page 39 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) renovated, and/or equipped 20-Dec-2010 30-Jun-2016 30-Apr-2018 29-Dec-2017 (number) Comments (achievements against targets): Page 40 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) B. KEY OUTPUTS BY COMPONENT PDO I - To improve the stewardship role of the federal public health system, through the strengthening of the EPHFs Regulation + Quality Assurance The systematic and standardized registration of health care professionals has made it possible for the first time to adequately determine the number, qualifications, geographical location of the country's "health care workforce," a critical input for need assessment and planning of health care delivery and workforce development. Outcome Indicators The systematic and standardized registration of private and public health care facilities has allowed to determine the number, level of service, geographical location, and other characteristics of health care facilities throughout the country. This constitutes a critical input for need assessment and planning of health care delivery. For example, the registration of Intensive Care Units for adults, children and neonates provides a valuable tool for referrals, emergency and catastrophic situations. The Federal Registry of Health Establishments (REFES), which had been established in 2009, was developed and implemented. It currently includes all public and private health care facilities in the country, which, since 2011, are required to be identified by single identification code. It currently includes over 25,000 registered health care facilities; over 13,000 registered pharmacies; and over 1,100 registered drug companies. The Federal Registry of Health Care Professionals (REFEPS) was formally established in 2015 and subsequently developed and implemented utilizing a single identification code for each health professional (e.g., medical doctors, nurses, technicians). It currently includes over 850,000 registered health care professionals, over 105,000 medical licenses, 220 health care professions, 154,000 health care specialists, as well as 1,502 teaching institutions. Intermediate Results Indicators Enhanced quality assurance mechanisms for health care and patient safety, including: i) the preparation and adoption of Guidelines for the Organization and Functioning of Health Services and Clinical Practice as well as Protocols for Standardization of Health Care Delivery Processes; ii) the consolidation of the Public Network of Evaluation of Health-Related Technologies (REDARETS); and iii) the ongoing evaluation and registration of primary care health care facilities, image diagnostic centers, and intensive care units. The Sanitary Legislation of the Argentine Republic (LEGISALUD), provides online access to health-related legislation at the international, national, provincial and municipal level, has been implemented and is available online. It includes over 25,000 pieces of enacted health-related legislation and regulations that are gathered daily based on the Official Gazettes published by the national government and 24 provincial jurisdictions and subsequently systematically catalogued. Strengthened human resources within the National Directorate to plan, coordinate and monitor regulatory and quality assurance strategy at the national and provincial level. Key Outputs under Component 1 Capacity building and training in regulation and quality assurance, including logistical support for: i) training workshops of managers and operators of new registers (i.e., REFES and REFEPS) in the SISA platform; and ii) workshops on quality assurance, focusing on the standardization of the health-care processes, patient safety, and external evaluation of health care services, among others. Page 41 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Knowledge generation, including several studies utilizing the information included in the various registries, such as: i) guidelines and protocols for health care delivery; ii) identification of evaluation processes and qualified evaluators of health care technologies; iii) annual mapping and characterization of health care professionals (i.e., medical doctors and nurses); iv) demographic profile of medical specialists. In addition, the portal LEGISALUD was further expanded and strengthened, including: redesign of the LEGISALUD portal; systematization of 163 thematic categories in the Federal Atlas; systematization of 2,284 sanitary norms at the national and provincial levels; publication of daily and weekly bulletins; and the launching of the Encyclopedic Dictionary of Health-Related Legislation in cooperation with OPS-WHO, with over 8,000 visits to its online site. A total of 287 health facilities for primary care (CAPs) were evaluated (REG 001; REG 002). CAPS Annual Verifications (REG 002). Key Outputs under II 1.1 - The number of intensive health care units evaluated and properly registered in the system increased from 0 to 633 between 2010 and 2017, Component 2 equivalent to 158 percent level of achievement (REG 003). (i.e., PHAs) A total of 551 Image Diagnostic Centers were evaluated (REG 004). The Health Services Information System - SIISA was updated and further developed, including the development and incorporation of several registers, including health facilities (REFES), pharmacies (REFAR), drug companies (REDRO) and medical professionals (REFEPS) (REG 005). A total of 56 Registers to track completion of medical residences have been established. Epidemiology, including Surveillance and Situation Rooms Epidemiological surveillance of specific events has improved significantly, providing the opportunity for timely clinical interventions. For example, the reporting of suspected arbovirosis events (e.g., dengue and Zika) is now done automatically, at an individual level, and in an average of two days after testing, compared to the consolidated information that was manually submitted by provinces on a weekly basis in 2009. Likewise, the Outcome notification of the results of syphilis testing among expecting mothers increased from 48 percent to 81 percent of live births between 2008 and Indicators 2016. Enhanced National Surveillance System, with 14 provinces achieving or exceeding the 80 percent end-target and four additional provinces with at least 75 percent of critical surveillance nodes performing at the satisfactory or highly satisfactory levels in 2017. KPI 2 - The percentage of certified departments or local territories with satisfactory or highly satisfactory epidemiological surveillance nodes increased from 68 to 75 percent between 2010 and 2017. Although below the 80 percent end-target, the increase in this composite index Intermediate Results represents improvements in terms of timeliness (i.e., opportunity), frequency, decentralization, and coverage of surveillance reporting by Indicators individual nodes representing 25 percent of all the nodes and 75 percent of all reporting. Coverage, in particular, exhibited the most notable increase at the system level, from 78 percent in 2011 to 84 percent in 2017. Individual provinces have made progress toward the establishment and consolidation of their Situation Rooms, including enhanced information flows, the configuration of working teams for the analysis of health information and the dissemination of findings, progress has been uneven Page 42 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) across provinces and not all of the achievements have been maintained over time. Specifically, provincial Situation Rooms are active in 19 provinces. New epidemic surveillance system (SNSV 2.0) was developed and is now operational under the SISA platform. Strengthened capacity within the National Surveillance Directorate to plan, coordinate and monitor the implementation of Situation Rooms at the provincial level, as a result of expanded human resources at the national and provincial levels (i.e., hiring of consultants) and activities aimed at enhancing national-provincial articulation. Key Outputs under Capacity building aimed at enhancing the effective utilization, analysis and dissemination of health data and information, including: i) workshops Component 1 on GPS Tools Applied to the Environment and Health; ii) specialization in epidemiology for professionals hired for or assigned to Situation Rooms; and iii) virtual Course on epidemiological surveillance of non-communicable diseases; and iii) a National Surveillance Symposium. Improved infrastructure, with a total of 838 surveillance nodes and 759 labs in 23 provinces provided with ITC and technical equipment. Design and ongoing implementation of SNVS 2.0 in SISA, incl. training of over 4,000 users. Operational improvements of the National Surveillance System at the provincial level, incl. updating of roster of establishments, and monitoring of reporting (VIG 001). A total of 20 provinces adopted formal mechanisms for information flows for their provincial Health Situation Rooms (SDS 004). Key Outputs under Component 2 A total of 22 provinces set up a virtual Health Situation Room ( SDS 005). (i.e., PHAs) A total of 14 provinces established a provincial Health Situation Analysis Committee (SDS 008.1). A total of 8 provinces produced a Situation Analysis Project (SDS 008.2). A total of 18 provinces created or updated a Contingency Health Situation Room (SDS 009). A total of ten provinces disseminated SSD information within the province (SDS 010). Monitoring and Evaluation Outcome Indicators N.A. Intermediate Results The Health Services Information System (SISA) was further developed and expanded, including the incorporation of several registers that were Indicators developed or expanded under FESP II, as follows: - Federal Registry of Health Care Professionals (REFEPS) - Federal Registry of Health Establishments (REFES) Page 43 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) - Federal Register of Nominalized Vaccination (NOMIVAC) - Management system of HIV/AIDs patients (SVIH) - Integrated Monitoring System of Medical Supplies (SMIS) implemented for vaccinations, blood, and HIV/AIDS. - Registry of Medical Residences (SNARES) - National Surveillance System II (SVNS 2.0) Other registers created or expanded under the FESP II include: - Register of Blood Donors (REDOS) - National Registry of Hemotherapy Providers - National Registry of Institutional Friends of Voluntary Blood Donors - Register of Plasma Providers - National Registry of Health Research (RENIS) See corresponding section under individual programs and functions. Key Outputs under Component 1 Key Outputs under See corresponding section under individual programs and functions. Component 2 (i.e., PHAs) Health Promotion, including Communications and the Healthy Municipalities and Communities (MCS) Program Outcome Indicators KPI 4 - 200 out of the 700 participating municipalities have been certified as Health Responsible Municipalities (100 percent level of achievement). A total of 36 new or improved health promotion agreements, and a total of 37 new regulations designed to promote healthy environments, Intermediate Results products and behaviors have been issued. Indicators A total of 1,011 bakeries have joined the "Less Salt, More Life" initiative, and 81 percent of them have been evaluated and certified by the Healthy Argentina Plan (PRO 003). Key Outputs under Training activities and technical assistance at the national and provincial levels, including planning instruments, results-based management Component 1 modality, development and management of MCS Public Health Activities. Page 44 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) A total of 23 Healthy Municipalities and Communities (MCS) Units have been created within provincial health ministries. A total of 23 MCS institutional agreements have been subscribed between provinces and the national government, as well as 450 territorial MCS agreements involving the national, provincial and local levels (MCS 001). A total of 1,148 municipalities are now members of the National Program of Healthy Municipalities and Communities, equivalent to 51 percent of all municipalities in the country. A total of 813 inter-sectoral working groups have been established at the municipal level. Of them, 231 have Key Outputs under conducted diagnosis studies of local health status and determinants, while 205 have constituted Situations Rooms to monitor determinants of Component 2 local health. A total of 819 health projects and plans have been financed (MCS 002). (i.e., PHAs) A total of 190 municipalities have undergone the self-evaluation process required to be certified as a Health Responsible Municipality. More than 80 of them meet the conditions to begin the self-evaluation process to be accredited as Healthy Municipalities (MCS 003). The 23 Provinces have elaborated an Provincial Strategic Plan for Health Promotion (PRO 001). More than half of the provinces monitor the actions included in their respective Strategic Health Promotion Plans, which have resulted in, among others, 50 communication campaigns, and 42 workshops on Health Promotion and Prevention of NCDs (PRO 002). Human Resources Development Outcome Indicators N.A. Enhanced access to information for decision making as a result of the creation of the Federal Network of Human Resources in Health (RHUS) and Intermediate Results the Federal Observatory of Human Resources in Health (OFERHUS), which was formally established as a technical area within the NMH (Resolution Indicators 1775 - E/2016). ORRHUS collects, produces and disseminates information on human resources within the health sector at the provincial and national levels. Implementation of the National System of Accreditation of Medical Residences that had been formally established in 2005 and 2006, including developing standards for 12 areas of specialization and establishing the foundations for a Single Residency Examination. Strengthened capacity within the National Human Resources Directorate to plan, coordinate and monitor human-resources strategies at the national and provincial levels, as a result of expanded human resources at the national level (i.e., hiring of consultants) and activities aimed at enhancing national-provincial articulation. Key Outputs under Capacity building focused on: i) training in the management of human resources in the health sector for provincial human-resources units within Component 1 the PMHs; and training of h, surveillance and control in the 19 Chagas-endemic provinces, including: training in Diagnosis, Treatment and Surveillance of health personnel at the provincial level, including: II 1.4 - A total of 1,665 health personnel from the MSN and provinces receiving training in public health labs, blood banks, and vaccination for more than 40 hour (208 percent level of achievement). Page 45 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) II 1.6 - Increased participation of provinces in training of health personnel in public health between 2010 and 2017, from 0 to 24 provinces (100 percent level of achievement). Equipment, mainly training equipment, such as interactive boards, multimedia projectors, and photo cameras. A total of 18 provinces have implemented the first phase of the Information System on Human Resources in Health, including adopting Key Outputs under standardized indicators and generating the corresponding baseline. Of them, ten provinces have subsequently updated the information. Component 2 (i.e., PHAs) A Registry of Medical Residences (SNARES) was created, with 550 and 2,790 medical residents formally registered in SISA in 2016 and 2017, respectively. This reflects partial progress towards its implementation, with three provinces fully meeting their targets for 2016 and 2017, and ten meeting them only partially. Six provinces have made no progress. Public Health Research Outcome Indicators The level of development of Argentina's National Health Research System was considered High during the period 2011-2015, compared to Medium for the period 2005-2010 (Iberoamerican Ministerial Network of Learning and Health Research - RIMAIS, 2015). Strengthened Provincial Health Research Systems, which, in turn, constitute the National Health Research System, as denoted by: i) In 2017 there were 23 formally constituted Health Research Areas at the provincial level, compared to ten before FES II. Intermediate Results Indicators ii) The National Registry of Health Research (RENIS) was composed by 19 provincial registries in 2017, compared to six before FES II. iii) In 2017 there were 16 formally constituted Provincial Committee of Ethics in Health Research, compared to 7 before FES II. Ten of them have been accredited and are under regular supervision. Knowledge production and training, including: i) tutoring program for young researchers; ii) annual health research meetings with the participation of 300 researchers from provincial jurisdictions; and iii) annual public health research conference in which the results of the research funded by Key Outputs under the Ministry of Health were presented to experts and decision makers. Component 1 Knowledge management and communication, including the publication of, among others, the "Revista Argentina de Salud Pública," the Health Research Grants Yearbook, as well as several books, brochures, and dissemination materials. Health research policy, including the establishment and coordination of Inter-Ministry Network of Heath Research (REMINSA), including two annual meetings with the participation of research leaders from all provinces. Key Outputs under A total of 18 diagnostic studies were conducted to assess the strengths and weaknesses of health research within individual provinces (SI 001). Component 2 (i.e., PHAs) A total of 13 new Areas of Health Research were created at the provincial level (SI 001). A total of nine new Ethic Committees for Health Research at the provincial level (SI 002). Page 46 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) A total of 13 new provincial registers were created and incorporated into the National Register of Health Research (SI 003). PDO 2 - To increase the coverage and clinical governance of PPHPs 1. Non-Communicable Diseases (NCDs) KPI 5 - The prevalence of tobacco consumption in adults 18-64 decreased from 30 to 25.1 percent between 2010 and 2017, exceeding the 27 percent end-target (163 percent level of achievement). Regular exposure to second-hand smoke decreased from 40.4 to 36.3 percent between 2009 and 2013. Outcome Indicators Daily sodium intake / inhabitant decreased by 2.02 grams between 2011 and 2015, from 11.2 to 9.2 grams/ per capita. Based on the various agreements with food manufacturers and food industry organizations, average sodium consumption is expected to reach 5 grams/day compared to 12-13 grams/day at the beginning of the Project. Intermediate Results In 2012, Argentina enacted one of the first sodium reduction laws in the world. Indicators II 2.1 - The number of tobacco-free municipalities increased from 0 to 142 between 2010 and 2017, equivalent to 101 percent level of achievement. Strengthened human resources within the National Directorate to plan, coordinate and monitor Health Promotion and NCD strategy at the national and provincial level. Capacity building and training in NCD and Health Promotion, including: "Epidemiological Surveillance of NCDs" course (60 hours) in 2014; "Model of Key Outputs under care of people with NCDs" course (70 hours) in 2014; "Prescription and Promotion of Physical Activity: Physical Activity is Health!" course (78 hours) Component 1 in 2014, as well as training on various topics such as diabetes, healthy eating, health promotion, smoking cessation. Knowledge generation on NCDs, including the implementation of the National Surveys of Risk Factors (2009, 2013), and an Impact Evaluation Study of New Graphic Health Warnings. Page 47 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) A total of 1,661 public and private institutions have been certified as Free of Tobacco Smoke Institutions (ENT 001). Key Outputs under A total of 465 organizations with at least 30 workers have been certified as a healthy work environment. Component 2 74 percent of the provinces have elaborated their quarterly report with the information of all the cases of injury by external causes and loaded it (i.e., PHAs) onto SISA. A total of 5,831 CAPS have been assessed and certified regarding their approach to NCDs. 2. Federal Health Program (PROFE) Outcome Indicators N.A. Intermediate Results N.A. Indicators II 1.2 - The number of PROFE population from Tucuman and Misiones enrolled in a clinical governance program for kidney disease prevention and Key Outputs under control increased from 0 to 4,000 between 2010 and 2016 (64 percent level of achievement). Component 1 II 1.5 - A total of 3 technical audits on clinical effectiveness of the diseases of low incidence and higher cost, renal disease and hemophilia were conducted between 2010 and 2016 (150 percent level of achievement). II 2.2 - The number of PROFE women with more than 7 children having signed their letter of rights increased from 0 to 8,477 between 2010 and 2016 (21 percent level of achievement). II 2.3 - The percentage of PROFE renal chronic dialysis patients evaluated and included in SINTRA increased from 0 to 95.1 percent (119 percent level Key Outputs under of achievement). Component 2 II 2.4 - The percentage of PROFE newly diagnosed patients with renal chronic dialysis evaluated within six months of beginning dialysis increased (i.e., PHAs) from 0 to 100 percent between 2010 and 2016 (125 percent level of achievement). II 2.5 - The percentage of PROFE beneficiaries with low-incidence, high-cost diseases included in the PROFE health care chain increased from 0 to 83.3 percent (167 percent level of achievement). Page 48 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 3. Vaccine-Preventable Diseases KPI 1 - As opposed to an increase in the registered coverage of the pentavalent vaccine with respect to the baseline, there was a drastic decrease Outcome Indicators during the operation's lifetime, from 93.5 to 88 percent between 2010 and 2017. The drop of the registered vaccination is also reflected in the decrease in the percentage of applied doses relative to the total number of doses distributed, from 71 to 48.4 percent between 2016 and 2017. Federal Register of Nominalized Vaccination (NOMIVAC) is now operational in 22 provinces, with over 40 percent of vaccination events registered in NOMIVAC in 2017. The implementation of this registry constitutes a significant advance, as it records vaccinations at the level of the individual, Intermediate Results including his/her physical address. However, its implementation is yet to be consolidated, as the percentage of live births registered in NOMIVAC in Indicators relation to the total number of live births exhibits large fluctuations from year to year (e.g., 70 and 57 percent in 2016 and 2017, respectively). Enhanced M&E of vaccines due to the utilization of the SMIS. Strengthened human resources within the National Directorate to plan, coordinate and monitor vaccination activities at the provincial level. Development and implementation of ITC systems, including: i) support for the digitization and reporting in the Federal Registry of Nominalized Vaccination (NOMIVAC); and ii) development and consolidation of the Sanitary Supplies Monitoring System (SMIS) at the national and provincial levels. Capacity building and training, including: Integral Training in Immunizations, and workshops at the provincial, regional and national levels focusing on NOMIVAC, Regional Immunization, Integrated Monitoring PAI-DIREPI, among others. Key Outputs under Knowledge building, including studies on: Lost opportunities in vaccination; Simultaneous or deferred administration of yellow fever and triple viral Component 1 vaccines in 12-month-old children; Prevalence of use of vaccines in adults, pregnant women and adolescents in the framework of the Second National Survey of Nutrition and Health; Seroprevalence studies in children vaccinated with a single dose of hepatitis A; Incidence of hospitalization due to community-acquired pneumonia and pneumococcal meningitis in adults in Argentina; and Multicenter, prospective, observational study to be carried out in institutions with admission of adults from various cities in the country. Physical infrastructure and equipment, i.e., the design and ongoing construction of six vaccine storage centers and two cold chambers (i.e., cold chain). Equipment, including: the acquisition of refrigerators, cold rooms, furniture, computer equipment and vehicles. 80 percent of the Provinces with Annual Management Diagnosis for the strategic planning prepared (PIP 001). 2,160 vaccinations and supervised vaccination records (PIP 002). Page 49 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Key Outputs under 7 jurisdictions with systematized information nodes on conformed immune-preventable diseases (PIP 003). Component 2 82 percent of the jurisdictions with quarterly report of doses applied in referenced form to the place of residence performed (PIP 004). (i.e., PHAs) 78 percent of jurisdictions with rapid coverage monitoring in territories at risk (PIP 005). 4. Vector-Borne Diseases 4.1 Chagas Outcome Indicators Significant progress in vector control, with two provinces--San Luis and Tucumán--succeeding in interrupting vectorial transmission. Significant improvement in prevention of vector transmission, with at least 95 percent of dwellings in the endemic area subjected to entomological surveys and spraying in 52 out of 162 departments in 2017 (equivalent to 32 percent Intermediate Results Strengthening in the reporting of Chagas events, with reporting coverage of pregnant women tested for Chagas in the SNVS averaging 60 percent in Indicators the last 3 years (2014-2017). Strengthening in early diagnosis and timely treatment, with an increase of 38.8 percent in treatments between 2010 and 2016. Strengthened capacity within the National Chagas Program to plan, coordinate and monitor Chagas-focused strategy at the provincial level, as a result of expanded human resources at the national level (i.e., hiring of consultants) and activities aimed at enhancing national-provincial articulation. Key Outputs under Capacity building in Chagas prevention, surveillance and control in the 19 Chagas-endemic provinces, including: training in Diagnosis, Treatment and Component 1 Surveillance of Chagas Disease; workshops for entomological surveillance of provincial health agents; and training to community leaders, schools and health teams in entomo-epidemiological surveillance with community participation. Equipment, including vehicles and GPS equipment for field teams responsible for vector control, computers and other lab equipment for provincial lab networks to enhance early diagnosis and reporting. 22 out of 22 provinces in which Chagas control is relevant have completed a diagnostic study and developed a Provincial Strategic Plan as well as an Operational Plan for Chagas Disease Prevention, Surveillance and Control (VEC 001). Key Outputs under Component 2 Monitoring of indicators within the Provincial Operational Plans (such as reporting of vertical transmission) has improved gradually, with 19 provinces conducting regular monitoring and reporting in 2017, compared to 10 provinces in 2013 and none in 2010 (VEC 002). (i.e., PHAs) Chagas prevention, surveillance, and control is included in the 12 provincial Situation Rooms, some of them with a dedicated focus on vector- transmitted diseases. Page 50 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) There are ten provinces with active Integrated Working Tables focused on Chagas disease, with the participation of national, provincial and municipal levels as well as non-public actors. 4.2 Mosquito-Transmitted Diseases (i.e., Dengue, Zika, Yellow Fever) Epidemiological surveillance of mosquito-transmitted diseases has improved significantly, providing the opportunity for timely clinical interventions. Outcome Indicators Specifically, the reporting of suspected arbovirosis events (e.g., dengue and Zika) is now done automatically, at an individual level, on an average of two days after testing, compared to the consolidated information that was manually submitted by provinces on a weekly basis in 2009. System of real-time notifications of routine and emergency actions of the personnel of the National Coordination of Vector Control (CNCV) was implemented under the Project and is operational. Particularly during outbreaks, It is a critical tool to strengthen emergency management, allowing for evidence-based decision making and rapid resource deployment. Intermediate Results Indicators Permanent training system for CNCV personnel has been implemented and is operational. Surveillance of mosquito-transmitted diseases under LIRA (Levantamiento de Índices Rápidos) has been implemented and is operational in six provinces. Strengthened capacity within the NMH team responsible for the National Plan for the Control of Mosquito-Transmitted Diseases to plan, coordinate and monitor implementation at the provincial level, as a result of expanded human resources at the national level (i.e., hiring of consultants) and regional and national-level activities aimed at enhancing national-provincial articulation as well as annual international events focusing on Dengue Key Outputs under to complement financial support from PHO and WHO. Component 1 Knowledge building, including pilot field tests of a new tool for Dengue Risk Stratification. Equipment, i.e., computers for members of the GT Denge group. Planning: The Strategic Plans for the Prevention and Control of Dengue and other arbor-viruses of nine provinces have been evaluated and are fully or partially operational. Vector control activities, including: 191,016 blocks in municipalities with medium-high environmental risk being monitored (DEN 001); 758,534 Key Outputs under dwellings inspected and treated (DEN 002); elimination of breeding sites in 13,055 public spaces (DEN 003); 10,781 suspected cases of dengue were Component 2 tested (DEN 004); 638,076 households visited and educated in prevention and control of dengue (DEN 005); 2,317 community events on dengue prevention and control were carried out (DEN 006); 20 community interventions were carried out in municipalities identified with medium/high (i.e., PHAs) environmental risk (DEN 012). Surveillance and reporting activities, including: 10,781 suspected cases of dengue were tested (DEN 004); 17 LIRA reports were elaborated (DEN 008); five Situation Room reports of risk scenario in the sentinel localities with risk of dengue outbreak were elaborated (DEN 009); 30 monthly reports of cases were reported to the SNVS (DEN 011). Page 51 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) 5. HIV/AIDS Argentina has the largest coverage of antiretroviral therapy in Latin America, with 81 percent of Argentines living with HIV (approximately 69,200 people) receiving antiretroviral therapy, compared to a global average coverage of 41 percent. Outcome Indicators As a result of the implementation of the SVIH system, the application and approval times of both antiretroviral treatments and viral load authorizations have been reduced substantially from 15 days to 48 hours, and from 30 days to less than a day, respectively, between 2010 and 2017. II 2.6 - The percentage of operational HIV/AIDS Counseling and Testing Centers increased from 0 to 70.6 percent (101 percent level of achievement). Intermediate Results Indicators Enhanced M&E of HIV/AIDs medications due to the utilization of the SMIS. Strengthened capacity within the HIV-STD Directorate to produce the 2013-2017 HIV-STD Plan and to plan, coordinate and monitor its Key Outputs under implementation at the provincial level, as a result of expanded human resources at the national level (i.e., hiring of consultants) and regional and Component 1 national-level activities aimed at enhancing national-provincial coordination. HIV-STDs Strategic Plan 2013-2017 carried out; HIV-STDs Strategic Plan 2018-2021 prepared. Monitoring and evaluation HIV-STD Directorate carried out. Development and implementation of ITC systems for HIV-STD treatment, including: i) system for the monitoring of patients (SVIH) improved and operational, with standardized patient database, currently with 1,353 users; and ii) systems for the monitoring of inputs (SMIS) improved and operational, ensuring traceability of medicines and strengthening the management of supplies. Capacity building in prevention, surveillance and treatment of HIV and other STDs , including: Knowledge generation in prevention, surveillance and treatment of HIV and other STDs, including the following studies: Evaluation and monitoring of the current status of the preventive and assistance response to HIV-AIDS, STIs and viral hepatitis in the Argentine Republic, Profile of people dying of AIDS in Argentina; Young people with recent diagnosis of HIV in the AMBA; Meanings around HIV; and Cascade of the continuum of HIV care in health services. Provision of HIV drugs and medical supplies (US$26 million) and implementation of centralized purchasing mechanisms. A total of 476 Centers for Testing and Counseling (CEPATs) created (HIV 001) Key Outputs under Component 2 A total of 3,046 preventive and community awareness activities were carried out by the CEPATs in all 24 provinces; 94,585 persons received council and/or were tested (HIV 002). (i.e., PHAs) Evaluation of 34 public health professionals that work with HIV/AIDS patients (HIV 003). Awareness and training in HIV/AIDS of actors involved in the management of penitentiary population (HIV 004). Page 52 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Supervision of 783 CEPATs and prevention centers - CEPs (HIV 005). 6. Safe Blood Priority Public Health Program More effective management of blood intake, as shown by the decrease in the percentage of deferred blood donors from 21% to 17% between 2009 and 2017. Safer blood (i.e., less risk of transmission of infections through blood transfusions) and less wastage of resources, as shown by a decrease in reactive serology (i.e., positive results detecting the presence of infections and other analytes) from 9% to 4.7% between 2009 and 2016 at the national level. Outcome Indicators Enhanced efficiency in blood processing, as shown by: i) the increase in the percentage of transfusions of fractionated blood units from 84% to 95.3% between 2009 and 2016; and ii) the decrease in transfusions of whole or unprocessed blood from 8,406 units in 2009 (1.6% of transfusions) to 1,160 units (0.17% of transfusions) in 2016. KIP 3 - Increase in the number of kilograms of blood-plasma produced by the public labs within National Blood Network increased from 33,000 kg in 2009 to 43,557 kg in 2017--thus, exceeding the end-target of 40,000 kg. Increased supply of recurrent safe blood - The increase in voluntary non-remunerated blood donors provide the foundation of a safe, sustainable blood supply. Specifically, voluntary blood donations per 1000 inhabitants over 18 years increased from 25.6% in 2009 to 30.9% in 2016--thus, exceeding the end-target of 30.6%. Voluntary blood donations now account for 10% of all blood processed by public sector blood centers. Enhanced production capacity of the blood network as a result of: i) reducing the number of hospital-based blood production centers from 238 to Intermediate Results 147 between 2011 and 2016; and ii) increasing the number of centralized blood center from 27 to 38 during the same period. Indicators Increasing access to safe blood by: i) increasing the transfusion centers from 244 to 861 between 2011 and 2016. Enhanced M&E of blood production supplies due to the utilization of the SMIS, with 100% of users tracking incoming supplies and 75% of users tracking their utilization. New policy regulation, including: i) new ministerial regulations, focusing on selection procedures of blood donors (MR 1507-1509/15) and prohibition of required blood donations (MR 1508/15); and ii) the drafting of the National Blood Law, which has been agreed with all relevant actors and is currently under consideration. Key Outputs under Creation of national registers, including: i) Register of Plasma Providers incl. quality standards and enforcement mechanisms (Administración Component 1 Nacional de Medicamentos, Alimentos y Tecnología Médica - ANMAT; Resolution 1538-1638 /12); and ii) Registry of Blood Donors - REDOS (Ministerial Resolution MR 761/13). Physical infrastructure and equipment for blood supply network, including: i) design and ongoing construction of four centralized blood production centers (i.e., Jujuy, Formosa and two in the province of Buenos Aires); ii) equipment and smaller infrastructure works (i.e., cold chain). Page 53 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Capacity building throughout the blood network, including: i) the training of 53 hemotherapy specialists with more than 1,200 hours of training; ii) 111 health personnel with at least 40 hours of training; and iii) a number of training and capacity building events with participation of health care authorities, medical personnel, lab professional and technical staff. Provision of blood processing supplies (US$37 million) and implementation of centralized purchasing mechanisms. Implementation of Medical Supply Monitoring System (SMIS), including user training in 24 provincial centralized blood production centers. A total of 4,894 blood drives carried out throughout the country, and 670 blood drives by centralized blood banks in high-demand areas (i.e., Buenos Aires, Cordoba and Santa Fe); design and implementation of public awareness and dissemination campaigns; and 2,456 events to train community promoters (SAN 001) Key Outputs under Component 2 A total of nine Volunteer Donors Management and Social Communication Units have been established at the provincial level (SAN 002). An additional of 838 providers of hemotherapy providers have been added to the National Registry of Hemotherapy Providers--equivalent to 41% of (i.e., PHAs) total number of establishments (SAN 003). An additional of 261 organizations and businesses have been incorporated to the National Registry of Institutional Friends of Voluntary Blood Donors, which now includes 426 members (SAN 004). 7. Tuberculosis Significant expansion of coverage of TB treatment, with the percentage of patients tested positive for pulmonary TB patients receiving treatment Outcome Indicators increasing from 54 to 98.6 percent between 2007 and 2017. Intermediate Results Universal TB treatment guaranteed, with 9,000 drug treatments for sensitive and resistant TB provided in 2016. Indicators Strengthened capacity of the NMH's team responsible for the National Program of Control of Tuberculosis and Leprosy to plan, coordinate and monitor actions at the provincial level, as a result of expanded human resources at the national level (i.e., hiring of consultants) and regional and Key Outputs under national-level activities aimed at enhancing national-provincial coordination. Component 1 System for the monitoring of TB drugs and medical supplies (SMIS) improved and operational, ensuring traceability of drugs and strengthening the management of supplies. TB Strategic Plan 2013-2017 carried out; TB Strategic Plan 2018-2021 prepared. Capacity building in TB prevention, surveillance and treatment, including: Mercosur meetings held to strengthen TB detection and prevention in migrant populations; TB monitoring and supervision in jails; TB in children; clinical approach of cases; Strictly Supervised Treatment (TAES) approach; and regional and national meetings. Page 54 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Key Outputs under Provinces submitted Treatment Monitoring Report twice a year in 2017; provinces sent the follow-up report of patients with TB every six months in Component 2 2017s, complying with the indicators of coverage and evaluation of the cases. (i.e., PHAs) Supervision of reference heath centers. Enhanced M&E of TB medications due to the utilization of the SMIS. Page 55 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Fernando Lavadenz Task Team Leader Amparo Gordillo-Tobar Senior Economist Vanina Camporeale Operations Officer Luis Orlando Perez Senior Public Health Specialist Marie Chantal Messier Senior Nutrition Specialist Rocio Schmunis Public Health Specialist Adam Wagstaff Research Manager Victor Ordonez Financial Management Specialist Keisgner Alfaro Senior Procurement Specialist Alejandro Solanot Financial Management Specialist Alejandro Alcala Senior Counsel Veronica Jarrin Senior Program Assistant Gabriela Moreno Zevallos Program Assistant Santiago Scialabba Program Assistant Isabel Tomadin Social Safeguards Consultant Francis Fragano Senior Environmental Specialist Maryanne Sharp Senior Operations Officer Julie Ruel Bergergon Consultant Vanessa Victoria Consultant Supervision/ICR Luis Orlando Perez Task Team Leader(s) Alvaro Larrea Procurement Specialist(s) Luz Maria Meyer Financial Management Specialist Page 56 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) Vanina Camporeale Team Member Maria Gabriela Moreno Zevallos Program Assistant Marcelo Roman Morandi Environmental Safeguards Specialist German Nicolas Freire Social Safeguards Specialist Maria Gracia Lanata Briones Team Assistant Luciana Garcia Team Assistant Florencia Chaves Team Assistant B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY10 29.445 167,517.04 FY11 40.052 183,072.45 Total 69.50 350,589.49 Supervision/ICR FY11 .803 28,170.00 FY12 13.403 90,058.31 FY13 25.830 200,787.59 FY14 51.769 274,702.90 FY15 33.374 150,926.61 FY16 33.662 162,919.65 FY17 14.575 58,530.55 FY18 15.613 100,300.23 FY19 3.477 11,432.00 Total 192.51 1,077,827.84 Page 57 of 75 The World Bank Essential Public Health Functions Programs II Project (P110599) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (%) Strengthen the Ministry of Health's (MSN) and Provincial Ministries' Stewardship 123.2 87.8 71.3 Capacities and Improve the Public Health Infrastruc ture Improve Results of Public Health Priority Programs and 322.0 334.9 104.0 the Provincial Level Administration, Monitoring, 14.5 22.0 151.7 and Evaluation Total 459.7 444.7 96.7 Page 58 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) ANNEX 4. EFFICIENCY ANALYSIS 1. The economic efficiency of the operation is considered Substantial to reflect its High economic efficiency and Modest implementation efficiency. The results of the economic analysis performed at appraisal remain valid in view of the operation's implementation experience and outcomes. The benefits that were not accrued from the anticipated coverage in pentavalente vaccinations are more than offset by the benefits from other Project outcomes that were not included in the original economic evaluation. However, there were considerable delays in the procurement of civil works, which extended their construction beyond the operation's closing date and postponed the accrual of the economic benefits associated with them. Specifically: 2. Economic efficiency: The operation's economic efficiency is deemed High. The economic analysis performed at Appraisal showed that the Project was expected to yield a net present value of benefits, after investment and recurrent costs, of about US$9.1 billion, and produce an internal rate of return above 50 percent over a ten-year period. In addition to project costs of US$461 million, the analysis included the costs of capital and recurrent expenditures related to the management of the seven selected programs, to be sustained once the main interventions of the Project were completed. Project interventions were anticipated to produce substantial savings over the medium to long-term. Savings were expected to accrue through reduction in morbidity and mortality rates associated with communicable and non-communicable diseases, reduction of the risk factors, and corresponding savings in hospital costs, outpatient costs, and treatment costs. 3. In the case of the non-accrued benefits from vaccination, there are several factors that indicate that they might not be considerable over a ten-year horizon. First, it can be expected that the actual drop in coverage is less than the one shown in the data due to misreporting as a result of the transition to the NOMIVAC register. Reporting is expected to improve once this transition is completed. Second, the actual coverage of pentavalente vaccination increases once late vaccination events included (i.e., while the coverage of on-time vaccination events was 88 percent in 2017, it increases to 89.5 percent when all infants one-year old or less are included, regardless of the timing of the vaccination as long as it took placed within the first 12 months after birth. Third, the drop exhibited in 2016 and 2017 is likely to be reversed in 2018, as the MSN now has experience with the purchases of vaccines from the new providers. 4. In addition, there were several economic benefits that were not included in the PAD's economic analysis, which can are considerable, including: • Savings from NCD-prevention activities (i.e., "Less salt, more life"; institutional agreements for the reduction of trans fats, healthier living in 400 municipalities, etc.) as well as the reduction in the tobacco consumption were not included in the analysis. According to estimates from the MSN, savings resulting from the prevention of heart attacks, strokes, and cardiovascular deaths are approximately US$3.765 billion over the same ten-year horizon. • Savings from Chagas prevention efforts supported under the operation. According to studies in other countries in the region, the cost of Chagas prevention accounts for roughly 2-4 percent of the cost of treatment, for which it is highly efficient intervention from an economic perspective. • Savings from the increased efficiency of the restructuring of the blood network, which are estimated at US$5 million per year once the civil works are completed (National Directorate of Blood and Hemotherapy). Page 59 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) • Savings from the implementation of the SMIS for the monitoring of the health supplies. Together with the implementation of centralized purchases, saving the implementation of the SMIS in the case of blood production supplies alone, are estimated at ARG$1.5 million per year (equivalent to US$50,000 per year). 5. Implementation efficiency: The operation's implementation efficiency is deemed Modest. As mentioned earlier, the closing date was extended 22 months to allow sufficient time for the implementation of civil works, which, in turn, had experienced considerable procurement delays. While one of the planned civil works was completed at operation's closing date, the remaining works were still under implementation and expected to be completed in their totality in early 2019 (see Table 1.1). From an efficiency perspective, these delays have resulted in the deferral of the economic benefits associated with these civil works. As mentioned above, savings from the increased efficiency of the restructuring of the blood network are estimated at US$5 million per year once the civil works are completed. Table 1.1 Civil Works under Construction as of October 25, 2018 - FESP II Page 60 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS FESP II 2010-2018 EXECUTIVE SUMMARY ABBREVIATIONS AND ACRONYMS NCD Non-communicable diseases PYLL Potential Years of Life Lost MSN National Ministry of Health NEA Northeastern Argentina NOMIVAC Federal Electronic Immunization Registry SAMIC Community Integrated Health Services NOA Northwestern Argentina PAD Project Appraisal Document PDO Project Development Objectives PEN National Executive Branch PACBI Capita net payment and high-cost, low-incidence lending CM Framework Agreement CGP Provincial Management Agreement PHA Public Health Activities NO No Objection DNPOIC National Directorate of Projects with International Organizations DA Designated Account CISA Certified Information System Auditor COFESA Federal Health Council AEC Concurrent External Audit CU Project Coordination Unit PROFE Federal Health Program UHC Universal Health Coverage 1. INTRODUCTION During the Project lifetime (2010-2018), Argentina went through an intense economic recovery and deep political and institutional changes as a result of the 2001-2002 national crisis. Since 2010, the Government has addressed new sector-wide priorities by continuing policy reforms defined in the Federal Health Plan (2010-2016 PFS). In this context, the Essential Public Health Functions Project II was designed to support the Argentina Government-led health policies and their goals to reduce mortality and morbidity linked to collective diseases and contribute to strengthen the stewardship and governance roles of the National Ministry of Health and Provincial Health Ministries (MSP) by developing Essential Public Health Functions and strengthening a series of prioritized programs in line with health goals set. Page 61 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) Having these objectives in mind and built on progress achieved by its predecessor (FESP I), FESP II was designed to make emphasis on institutional strengthening and capacity building in both provincial and national levels and linkage with prioritized programs objectives and goals. The PDOs proved to be critical for the Federal Health Plan (2010-2016). Priority Public Health Programs Essential Public Health Functions and *Non-communicable diseases *Policy Regulation *Incluir Salud (Ex Profe) *Surveillance *Vaccine preventable diseases *Monitoring & Evaluation *HIV/AIDS *Health promotion *Safe blood *Social Participation *Tuberculosis *Planning and Policy-making *Equity *Human Resource Development *Quality Assurance *Public Health Research *Emergency and Disasters. The Project focused on: a) Strengthening the MSN's Stewardship Capacities through the Essential Public Health Functions (FESP); and b) Implementing governments’ priority public health programs. Interventions at provincial level were engineered around a Public Health Activities (PHA) mechanism. The achievement of objectives/goals fixed by provinces revolved around bridging gaps and maximizing structural actions that rendered sustainable outcomes upon completion of operations. 2. CONTEXT The health sector in Argentina is immersed in a complex federal structure (political, institutional and administrative). Although the national level governs and guides the overall health sector, the provincial governments, framed in their own constitutions, are responsible for guaranteeing the citizens’ right to health. The high fragmentation, extensive array of stakeholders, and poor internal linkage across the health system translate into a poorly coordinated coexistence of bulky subsystems - national and provincial social security, public provision and private insurance - with their own political and institutional arrangements that hinder the implementation of proposals promoting integrated health actions. In this context, important efforts were undertaken to strengthen the essential public health functions (EPHF), mainly those linked to: (i) Planning and development of sectorial policies, (ii) Regulation and control, (iii) Quality assurance and (iv) Social Participation, both nationally and provincially Furthermore, in Argentina there was no unified comprehensive health promotion program that included action plans and policies at both national and provincial level, and measurement of risk factors for burden of disease at national level. Therefore, strengthening disease prevention and health promotion strategies to deal with chronic diseases was deemed a critical step. In addition, they identified the need to develop a comprehensive approach for chronic diseases under INCLUIR SALUD (ex PROFE) Program. Page 62 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) Thus, the FESP II goal consisted of strengthening the governance and coverage of key non- communicable diseases programs, using a results-based approach, without neglecting communicable diseases. 3. INSTITUTIONAL AGREEMENTS On March 3, 2011, the Presidential Decree Nº 263/11 approved the draft IBRD Loan Agreement. . The IBRD 7993-AR Loan Agreement was signed between Argentina and IBRD on April 11, 2011, to be implemented over a five-year timeframe. The FESP II was declared effective on August 3, 2011. The closing date was June 30, 2016. In that year, it was extended for additional fifteen months and the new closing date was April 30, 2018 During implementation, between 2011 (the period prior to the formal start date of the Project) and 2018 (extended closing year), two presidential elections and provincial authorities were held, resulting in authority changes (the elections took place in 2011 and 2015). All jurisdictions, except one, elected new governors and appointed at least 62 Ministers of Health in the 24 target provinces during the Project lifetime. These changes altered the working approach held by the lead team, which made adjustments and adaptations as needed for maintaining or improving the implementation of activities, the day-to-day performance and adherence to the sustainability plan. 4. PROJECT DEVELOPMENT OBJECTIVES (PDO) The Project Development Objectives were: FESP II Development Objectives and Components PDO 1 PDO 2 To improve the stewardship role of the public health To increase the coverage and clinical system, through the strengthening of Essential Public governance of Priority Public Health Programs. Health Functions. Component 1: Strengthening of MSN's and Provincial Component 3: Ministries' Stewardship Capacities and Improve Administration, Component 2: the Public Health Infrastructure of the Priority Monitoring and Improve Results at the Provincial Level in Public Health Programs Evaluation Priority Public Health Programs Works ENFR Health Pilot Information Projects Systems The strategies that were implemented through the Project by Priority Public Health Programs, reflected the best international practices recognized by PAHO and WHO. 13 13 See Annex to document body: Overall Results Chain Page 63 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) 5. PROJECT RESULTS This box summarizes the Project Overall Results Chain: COMPONENTS COMPONENT OBJECTIVES OVERALL RESULTS The Component supported the implementation of key activities aimed at upgrading quality, safety, epidemiological surveillance, and health promotion by developing healthy municipalities, reducing the prevalence of smoking, distributing supplies, regulations that are intrinsic to the stewardship role of national and provincial health ministries. The specific results are substantial. Component 1. * Strengthen the national and provincial The public health care system infrastructure was also improved, focusing on strengthening the National Safe Blood Strengthening of stewardship capacities, using the Network, by refurbishing or building six regional blood banks, expanded storing capacity of vaccine storage facilities MSN's and essential public health functions and rehabilitating the National Biotechnology Laboratory. Provincial framework and modernizing While a laboratory building had been completed by the closing date of the Project, other 13 physical works are still Ministries' management of selected programs. Also, under construction, displaying different completion rates, ranging from 32% to 91%. Overall financial performance Stewardship further strengthen the epidemiological averages 70%, delayed by overruns beyond the scope of the Project. All civil works are expected to be completed in Capacities and surveillance of chronic diseases, through August 2018 under the formal assumption the national government will take on financial responsibility for Improve the reengineering the MSN to address NCDs completion of these works Public Health and providing technical assistance and In addition, one of the relevant purposes under this Subcomponent consisted of financing two pilot projects in Infrastructure of training. This component also Misiones and Tucumán provinces, aimed at promoting innovations in health promotion and reducing risk factors. the Priority strengthened the public health This implied enforcing the Framework Agreements (CM) signed between the MSN and the participating provincial Public Health infrastructure, including, blood banks, ministries of health, specifying covenants, objectives, activities and indicators used to measure performance, using Programs cold chain and supply monitoring result-based payment mechanisms, and complying with Bank safeguard policies. systems. Both projects were actually launched in July and September 2013 following the signing of an amendment to the Annual Provincial Management Agreements (CGP) for implementing the Pilot Projects in both provinces. In that document thereof, the implementation and monitoring arrangements and planned investments were set forth explicitly. Page 64 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) The process to have the Annual Management Agreements signed with participating provinces began in the third On the one hand, the Project would quarter of 2012. During the third and fourth quarters of that year, 23 provinces signed their Management support the achievement of intermediate Agreements for implementing the PHA. On December 31, 2012 the Project had signed Framework Agreements with health results at provincial and municipal 23 jurisdictions. In 2015, Autonomous City of Buenos Aires adhered to the Project, so that year saw the Framework levels through incentives for the selected Agreements were signed with the 24 target jurisdictions nationwide. programs. Bank financing was used to reimburse provinces partly for delivery of selected Public Health Activities (PHAs), in order to strengthen essential public health functions and/or programs. Priority public health programs made significant achievements (including tuberculosis, safe blood, NCD-Diabetes- Hypertension, cervical and breast cancer, kidney diseases, lifestyle-related diseases-decline in tobacco prevalence, decreased consumption of salt). However, the most recent two-year trends in Pentavalent immunization coverage Component 2: are failing. Specifically, instead of continuing a rising trend toward the 95% goal, the percentage of children under Improve Results the age of 1 year that were immunized, after peaking 94.1% (2014), failed in maintaining an upward performance In parallel, in the first half of 2012, the at the Provincial over the final period of the Project, falling back to 93.8% in 2015, 91.5% in 2016 and 88% in 2017. process of reviewing the FESP II PHA Level in Priority In 2017, they identified specific circumstances (delayed procurement of vaccines as well as unavailability of timely protocols launched, whose proposal was Public Health budgetary funds) that contributed a stalled immunization coverage (that is, the additional burden on the system’s partially approved by the WB (Non Programs capacity due to increased number of compulsory vaccines from 12 to 20 in recent years and the nominalized system Objection N ° 1), with an abridged menu introduced to keep immunization records). Note here that the current Pentavalent immunization coverage rate does to be implemented on third quarter of not put the child population at risk. In addition, Pentavalent coverage peaks 95.3% when measured in children up to 2012. The Review Mission in September 18 months old, which is greater or similar than many other countries, including the USA and OECD LICs. 2012 revised the comprehensive On the other hand, the Component sought to improve the Federal Health Program efficiency and performance proposal, and finally the World Bank (INCLUIR SALUD). The sub component pursued to increase the Program efficiency and performance by equipping it issued Non Objection in favor of the full- with modern administrative, financial and health management tools to transform it into a public health insurance fledged PHA Protocol (version 1.6) on system that provides verified benefits for high-cost and low-incidence conditions and disabilities for the most March 26, 2013 (NO N° 19), amending vulnerable population with no explicit health coverage. Annex IX of Project Operational Manual In this regard, the Project partially financed the capitation payments for Incluir Salud population, for an amount that reflected the decreasing INCLUIR SALUD capitation cost in relation to the monthly rate of capita execution and improvements in Incluir Salud management. The WB’s decision to finance Incluir Salud expenses (USD 280 million) is equivalent to 23% of Incluir Salud total expenses (USD 1,200 million) estimated for the entire project implementation period prior to its restructuring. Page 65 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) General Indicators Performance 17 out of 21 FESP II indicators achieved or exceeding targets at the Project end. 1 was crossed out and 3 had failing performance: Not FESP II INDICATORS Achieved achieved 1. Increased percentage of children under one year of age vaccinated with Pentavalent vaccine X 2. Increased percentage of certified departments or local territories with satisfactory or highly satisfactory epidemiological surveillance nodes (C2). Methodological PDO LEVEL adjustment (approved on November 2016). X RESULTS 3. Increased kilograms of blood-plasma produced by MSN INDICATORS and provincial labs. X 4. At least 200 out of the 700 participating municipalities certified as "Health Responsible Municipalities”. X 5. Reduction of tobacco consumption prevalence in adults 18-64. X 1.1 Number of intensive health care units evaluated and properly registered in the system. X 1.2 Increased number of PROFE population from Tucuman and Misiones enrolled in a clinical governance program for kidney disease prevention and control. X 1.3 Increased number of national and provincial public health labs and blood banks constructed and/or rehabilitated. X 1.4. Increased number of health personnel from the MSN and provinces receiving training in public health labs, blood INTERMEDIATE banks, and vaccination for more than 40 hours. X RESULTS 1.5 Number of technical audits on clinical effectiveness of INDICATORS the diseases of low incidence and higher cost, renal disease COMPONENT 1 and hemophilia. X 1.6 Increased participation of provinces in training of health personnel in public health. X 1.7 Percentage of women between 34 and 64 years old in the two pilot provinces benefitting from at least one HPV and/or cytology test provided by the health system. X 1.8 Percentage of women between 50 and 69 years old in the two pilot provinces benefitting from at least one mammography provided by the health system. X 1.9 Number of Number of pilot studies with nutritional NOT APPLICABLE Sprinkles component (to reduce iron deficiency anemia). INTERMEDIATE 2.1 Number of tobacco free municipalities X RESULTS 2.2 Number of PROFE women with more than 7 children X Page 66 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) Not FESP II INDICATORS Achieved achieved INDICATORS having signed their letter of rights. COMPONENT 2 2.3 Percentage of PROFE renal chronic dialysis patients evaluated and included in SINTRA. X 2.4 Percentage of PROFE newly diagnosed patients with renal chronic dialysis that have been evaluated within six months of beginning dialysis X 2.5 Percentage of PROFE beneficiaries with low incidence high cost diseases included in the PROFE health care chain. X 2.6 Percentage of operational HIV/AIDS Counseling and Testing Centers. X 2.7 Blood donations per 1000 inhabitants over 18 years old. X Robust monitoring proved to be essential for effective EPHF implementation and progress toward achievement of objectives. The Project supported the MSN in (i) Collecting routine national health statistics; (ii) Monitoring public health supply; (iii) Conducting national epidemiological surveillance; (iv) Undertaking external audits for measuring results at provincial level. Overall results show substantial achievements, as three out of five Project Development Objectives (PDO) were successfully accomplished. On the one hand, the indicator: “Increase in the percentage of certified departments or local territories with satisfactory or highly satisfactory epidemiological surveillance nodes” progressed from 70.2% to 75% (80% Target), which means departmental nodes had a 93.75% achievement. However, the indicator “Increased percentage of children under one year of age vaccinated with Pentavalent vaccine” (PDO1), based on the latest December 2017 data reported on April 2018, revealed that actual achievement was 88% (versus 95% target). Regarding the three Intermediate Results Indicators: 1. Number of tobacco free municipalities. 2. Percentage of operational HIV/AIDS Counseling and Testing Centers. 3. Blood donations per 1000 inhabitants over 18 years old, 16 provinces out of 24 achieved them successfully. Overall Indicators Evolution Target Target Target Target Target Baseline 2011 2012 2013 2014 2015 2016 2017 2011 2012 2013 2014 2015 Increased percentage of children under one year of age vaccinated with Pentavalent vaccine 93,6% 93,8% 91,7% 94,0% 91,3% 94,3% 94,7% 94,6% 94,1% 95,0% 93,8% 95,3% 88% Increased percentage of certified departments or local territories with satisfactory or highly satisfactory epidemiological surveillance nodes (C2). 0% 35% 36,6% 50% 48,5% 60% 52,0% 70% 50,1% 80% 51,8% 49,9% 62% Methodological adjustment (approved on November 2016). Page 67 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) Target Target Target Target Target Baseline 2011 2012 2013 2014 2015 2016 2017 2011 2012 2013 2014 2015 0% 35% 67,7% 50% 68,5% 60% 70,8% 70% 64,7% 80% 68,0% 67,8% 75% Increased kilograms of blood-plasma produced by MSN and provincial labs. 43.557 33 34 39 35 40.79 36 45.4 37.5 37.96 40 40.13 41.6 kg At least 200 out of the 700 participating municipalities certified as "Health Responsible Municipalities”. 0 30 s/d - - 100 102 - 120 200 143 166 200 Reduction of tobacco consumption prevalence in adults 18-64. 30,1% - 30,1% - - - 25,1% - 25,1% 27,1% 25,1% 25,1% 25,1% The Project achieved substantially the Project Development Objectives (PDOs): Improve the stewardship role of the public health system, through the strengthening of essential public health functions and increase the coverage and clinical governance of Priority Public Health Programs 6. RESULTS-BASED FINANCING: PHAs The Public Health Activities (PHAs) were a pivotal instrument of FESP II, whose backgrounds and further improvement are built on the lessons learned from the implementation of FESP I. Under this scheme conditional cash transfers were subject to the actual implementation of measurable health actions and payment is subject to the achievement and verification of results defined in advance. 14 During the implementation of FESP II, the subcomponent was executed 100% financially speaking while physical progress landed at 71%. It supported the following activities: (i) Public health surveillance activities, and other epidemiological technical systems; (ii) Disease control activities; (iii) Monitoring of compliance with national and provincial norms and standards including training and supervisory activities to improve public health quality; (iv) Health promotion, communication, education and social participation activities with an emphasis on healthy lifestyles; (v) Regulatory activities carried out by the MSPs; (vi) Institutional strengthening and design and sustainability of health structures such as PAIERC 15, indigenous health, bio hazardous wastes or research areas. An attempt was made to overcome the programmatic fragmentation and address policies from integral strategies, including: (i) Assessment of health status in provinces and their trends by setting up and developing health situation rooms, (ii) Achievements on information systematization, standardization of activities and use of protocols, (iii) Frameworks for monitoring and evaluating results; (iv) Process standardization; (v) Sustainability in monitoring mechanisms and internal and external audits. Results successfully achieved in terms of provincial planning and management include: (i) Creation of Jurisdictional Units and/or Areas, e.g. Health Research Area, Research Ethics Committee, Health Situation 14 While comparisons between FESP I and FESP II go beyond the scope of this paper, note that some critical changes include PHA incentive system. While incentives under FESP I were “negative” or penalty for non-compliance (constrained cash flows for noncompliance with indicators agreed), FESP II opted for “positive” incentives (increased cash flows for successful compliance with results). 15 Comprehensive CKD Management Program. Page 68 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) Rooms, Environmental Health Units, Indigenous Health Units, Provincial Kidney Health Units; (ii) Use of computer systems, including the SMIS module, SIISA for integrating information linked to both Intensive Care Units, Health Facilities, doctors who successfully completed their residency program, as well as CKD patients; (iii) Implementation of specific monitoring: DOTS (Tuberculosis) Treatments, Health Promotion and Chagas Disease; (iv) Implementation of FESP measurement tool, whose interdisciplinary exchange nurtured an enriching debate about the tool arrangement, interpretation of indicators and particularities of target provinces. 7. EPHF, COVERAGE, GOVERNANCE AND STEWARDSHIP 16 The Project largely focused on developing actions to strengthen stewardship, governance and coverage, in addition to information monitoring and systematization actions. The table below shows, based on the information shown, the contribution (if any, indicated as "X") attributable to each Program/Directorate to each EPHF, as depicted in the following scheme: National Programs/Directorates Contributing to EPHF Program/Area contributing to EPHF 1 2 3 4 5 6 7 8 9 10 11 Health Surveillance Area/Directorate of Epidemiology X X X Directorate of Health Promotion and NCD Control X X X X X X Directorate of Vaccine Preventable Disease Control X X Department of Regulation and Quality of Health Services X X X X X Directorate of Blood and Hemoderivatives X X X X Environmental Safeguards X X X X X X INCUCAI Kidney Disease Management Program X X X Directorate of Healthy Cities, Municipalities and Communities X X TB Program X X AIDS and STDs X X X X X National Plan for Dengue and Yellow Fever Prevention and Control X X National Directorate for Human Capital and Occupational Health X X Directorate for Health Research X X National Department for Health Emergency X X EPHF 1. Monitoring, evaluation and analysis of health status 2. Surveillance, research and control of the risks and threats to public health 3. Health promotion 4. Social Participation in health 16 For an in-depth analysis, it is suggested to see Annex V of final report (document body). Page 69 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) 5. Strengthening of public health regulation and enforcement capacity 6. Evaluation and promotion of equitable access to necessary health services 7. Human resources development and training in public health 8. Quality assurance in personal and population-based health services 9. Research, development and implementation of innovating health solutions 10. Reduction of the impact of emergencies and disasters on health 8. FINANCIAL PERFORMANCE Performance per Investment Category At the end of the Project, 17 execution totaled US$502,558,869, including US$445,785,893 financed by the World Bank and US$56,772,975 by local matching funds. According to progress, the categories of Public Health Activities, Capitations and Front-end fees were fully executed (100%). The category of Medical Supplies and Goods, Consultant/Non-Consultant Services & Training resulted in 97% and 91% execution rate, respectively. Likewise, Operating Costs category reached 99% of execution. Finally, Works, Goods and Consulting Services had an execution of 65%. Investment Matrix per Investment Category as at 31/10/18 (USD) Available as at Budget Approved Cumulative as at 31/10/2018 Investment Category % Progress 31/10/201 8 BIRF Local Total BIRF Local Total BIRF 1- (i) Goods, Works, and Non 10.159.48 Consultant Services Part 1. B 29.436.000 12.000.000 41.436.000 19.276.512 5.570.694 24.847.205 65% 8 (ii), (iii), (iv) y (v) of the Project 2- Eligible medical supplies 44.300.000 28.000.000 72.300.000 43.017.066 21.597.260 64.614.326 97% 1.282.934 3- Sub projects 0% - - - - - - - 4- Goods, Consultant Services and Non Consultant Services and Training under the Project, 48.305.000 11.962.333 60.267.333 44.172.723 26.636.879 70.809.602 91% 4.132.277 excluding Categories (2), (3), (5) and (6) 5- Essential Public Health 54.518.500 54.518.500 54.897.892 54.897.892 101% -379.392 Activities(PHA) - - 6-Cápita payments under Part 280.000.00 280.000.00 280.000.000 280.000.000 100% 2.B (i) of the Project - 0 - 0 - 7- Operating Costs 3.288.000 650.000 3.938.000 3.269.200 2.968.143 6.237.343 99% 18.800 8- Front-end fee 1.152.500 1.152.500 1.152.500 1.152.500 100% - - - 513.612.33 502.558.86 15.214.10 Total 461.000.000 52.612.333 445.785.893 56.772.976 97% 3 9 7 Source: Administration and Finance - DGPPSE 17 Information available as at 31/10/2018. Page 70 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) When analyzing the evolution of expenditure categories, note that category 5 had an upward execution until 2015. On the other hand, expenditure categories 1, 2 and 3 had performance peaks in certain years. Likewise, expenditure category 7 remains practically steady throughout the Project while category 6 had escalating performance in 2011 and 2012 and then it decreased. Performance per Components and Sub-components In line with a description per Component and Sub-component, at the end of Project, Component 2 and Front-end Fee had a 100% execution. Component 3 had a 95% completion rate. Finally, Component I had an 86% completion rate. Available as at COMPONENTS Y Budget Approved Cumulative as at 31/10/2018 % Progress 31/10/2018 SUBCOMPONENTES BIRF Local Total BIRF Local Total BIRF COMPONENT 1: Strengthen the MSN's and provincial Ministries' stewardship capacities and 102.216.000 47.733.333 149.949.333 87.758.291 44.074.738 131.833.029 86% 14.457.709,35 Improve the Public Health Infrastructure of selected Priority Programs (a) Sub-Component 1: Strengthening stewardship 28.480.000 7.733.333 36.213.333 24.753.368 16.706.022 41.459.390 87% 3.726.632 in public health (b) Sub-Component 2: Modernizing the 73.736.000 40.000.000 113.736.000 63.004.923 27.368.716 90.373.639 85% 10.731.077 infrastructure of the public health system COMPONENT 2: Improve results of selected public 334.518.500 - 334.518.500 334.897.892 - 334.897.892 100% -379.392 health priority programs at the provincial level (a) Sub-Component 1: Financing public health 54.518.500 - 54.518.500 54.897.892 - 54.897.892 101% -379.392 results in provinces (b) Sub-Component 2: Improving the efficiency and 280.000.000 - 280.000.000 280.000.000 - 280.000.000 100% - performance of Federal Health Program (PROFE) COMPONENT 3: Administration, Monitoring 23.113.000 4.879.000 27.992.000 21.977.211 12.698.237 34.675.448 95% 1.135.789 and Evaluation (a) Sub-Component 1: 16.113.000 3.212.333 19.325.333 15.374.179 10.497.227 25.871.406 95% 738.821 Administration (b) Sub-Component 2: 7.000.000 1.666.667 8.666.667 6.603.032 2.201.011 8.804.042 94% 396.968 Monitoring & Evaluation Front-end fee 1.152.500 - 1.152.500 1.152.500 - 1.152.500 100% - Total 461.000.000 52.612.333 513.612.333 445.785.893 56.772.976 502.558.869 97% 15.214.107 Page 71 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) Procurement and Contracts The ongoing contracts are commitments signed in 2017 to build different infrastructure works in different provinces. As at 10/31/2018, works contracts performance accounted for 63%. The National Government committed itself to complete unfinished works. Project Works Province In Argentina Pesos Renovation of Vivarium and Insectarium at National Institute of FESP II Parasitology “Dr. Mario Fatala Chaben” and Laboratories and Health CABA 6.209.324 Institutes “Dr. Carlos G. Malbran” FESP II Storage Facility – Mendoza Mendoza 16.979.000 FESP II Storage Facility - Santa Fe Santa Fe 24.740.883 FESP II Storage Facility - Entre Ríos Entre Ríos 19.660.924 FESP II Storage Facility – Chaco Chaco 16.160.004 FESP II Storage Facility – Córdoba Córdoba 25.151.494 FESP II Storage Facility - Sgo del Estero Sgo del Estero 19.595.093 FESP II Storage Facility - Tucumán Tucumán 17.709.926 FESP II Construction Cold Room for Vaccine Storage in Formosa Formosa 15.996.115 FESP II Hemotherapy Center in Jujuy Jujuy 57.967.474 FESP II Hemotherapy Center in Formosa Formosa 69.921.713 FESP II Hemotherapy Center in Buenos Aires (3 de febrero) Buenos Aires 118.836.184 FESP II Hemotherapy Center in Buenos Aires (La Plata) Buenos Aires 121.700.000 Total 530.628.134 9. LESSONS LEARNED AND SUSTAINABILITY Focusing on Essential Public Health Functions may lead to more effective public health programs in countries in the Americas. 18 This approach resulted in seeing EPHF as independent pillars and crosscutting themes embedded in different health programs. Results-based financing proved to be an effective mechanism to improve health services reaching out the population and support institutional development. The operation successfully promoted harmonization between the national and provincial levels by building consensus and collaboration among them. The signing of framework agreements between federal and provincial levels clearly establishing the responsibilities of each party helped mitigate political risks. The same arrangement may be followed to reduce political risks at national level, whereby National Directorates and priority health programs formally sign agreements clearly stating their responsibilities during the term of the operation. 18 The EPHF describe the spectrum of competencies and actions that are required to reach the central objective of public health, improving the health of populations (…) The EPHF have been defined as the structural conditions and aspects of institutional development that permit better performance in terms of public health practice. Pan American Health Organization. Public Health in the Americas. Conceptual Renewal, Performance Assessment, and Bases for Action. Washington DC: OPS, 2002. Page 72 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) The most notable advances towards institutionalization include: i) A large share of human resources hired as part of the operation's technical assistance has been formally transferred to the MSN; ii) the Directorate of Blood and Hemoderivatives is adopting the results-based mechanism to allocate its own budgetary resources to the provinces; iii) The Safeguards Unit transferred under this operation has been permanently incorporated into the Ministry's dominion; therefore, all safeguard procedures will be applied systematically to all externally funded programs; (iv) The immunization program is adopting management agreement as a planning standard for working with provinces; and v) The procurement processes are walking away of UFI and embracing the day-to-day administration of MSN. In addition, FESP II contributed to the creation of provincial units that would enable diagnoses, establish plans with lines of action and establish the jurisdictions’ priorities. It leaves as legacy a rated capacity for planning, managing, monitoring & evaluating health policies. It also encouraged the establishment of an activity monitoring system: 22 provinces measure EPHF performance through the EPHF Performance Measurement Instrument to monitor their evolution over time. Today, the country has two EPHF compared performance measurements in 22 out of 24 provinces, carried out between 2011 and 2014. Such EPHF measurements have become an observatory for each of these provinces. Another positive outcome resulting from the Project implementation refers to a structured accountability system in place, by consolidating and incorporating in full independent concurrent auditing service (measurement of outputs). This independent concurrent audit intended to verify the accuracy and appropriateness of PHAs in conformity with requirements compulsory and previously agreed between the national authority and each of the participating provinces. The observations noted by the audit enabled the debits for future reimbursements and triggered improvement plans for implementing future activities, based on the recommendations made by the Central Coordination Unit. In view of the above, it is evident that the Project became the basis for reshaping relations with the programs at both national and provincial levels, facilitated the harmonization of the national policy across different health system levels, favored the standardization of information, and the allocation of resources, linking the financial support from the Nation to the Provinces through measurable health outcome activities. Risks include: i) Completion of works under construction, including the supply of appropriate equipment and commissioning of the Regional Hemotherapy Centers and vaccine storage facilities; and ii) Clinical responsibility and increased efficiency of PROFE Program, which has been transferred beyond the MSN domain, reengineered as a PLWD program under the National Executive Branch. The basic operational challenges faced by the Immunization Program, e.g. predictable budget allocation, capacity for planning procurement of essential medical supplies, and collection of timely information, raise a red flag, so does as the half-done performance of Plan of Works in support of the Immunization Program and the National Blood Plan (which excelled in other aspects). Although these factors were beyond the Project, they still pinpoint weaknesses in clinical governance and vaccine- preventable disease service delivery at national level, possibly one of the most important priority public health programs for MSN and the Project. Final Report (Executive Summary, Body and Annexes) written by Carolina Casullo. Coordinator FESP II 2015-2018. Page 73 of 74 The World Bank Essential Public Health Functions Programs II Project (P110599) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) Page 74 of 74