FOR OFFICIAL USE ONLY Report No: ICR00054 INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD-8593 ON A LOAN IN THE AMOUNT OF US$420 MILLION TO THE REPUBLIC OF COSTA RICA FOR THE STRENGTHENING UNIVERSAL HEALTH INSURANCE IN COSTA RICA PROGRAM-FOR-RESULTS JULY 31, 2024 Health, Nutrition & Population Global Practice Latin America And Caribbean The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2024) Currency Unit = Costa Rican Colones (CRC) CRC$523 = US$1 US$1.31 = SDR 1 FISCAL YEAR July 1 - June 30 For Official Use Only Regional Vice President: Carlos Felipe Jaramillo Country Director: Michel Kerf Regional Director: Jaime Saavedra Practice Manager: Tania Dmytraczenko Task Team Leader (s): Laura Di Giorgio ICR Main Contributor: Maria Cecilia Zanetta The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT ABBREVIATIONS AND ACRONYMS ATAP Technical Primary Health Care Assistants (Asistentes Técnicos de Atención Primaria) CCSS Costa Rican Social Security Administration (Caja Costarricense de Seguro Social) COVID 19 Coronavirus disease CPF Country Partnership Framework DLI Disbursement-Linked Indicator DM2 Diabetes Mellitus Type II EDUS Unique Digital Health Record e-health Package (Expediente Digital Único de Salud) ESSA Environmental and Social Systems Assessment FM Financial Management GoCR Government of Costa Rica ICR Implementation Completion and Results Report IMAS Mixed Institute of Social Assistance (Instituto Mixto de Ayuda Social) IP Indigenous Peoples For Official Use Only IRI Intermediate Results Indicator ISR Implementation Status and Results Report LAC Latin American and the Caribbean LGBTQI Lesbian, gay, bisexual, transgender, queer, and intersex persons M&E Monitoring and Evaluation NCDs Non-communicable Diseases PAD Program Appraisal Document PAHO Pan-American Health Organization PDO Program Development Objective PDOI Program Development Outcome Indicator PforR Program-for-Results PHC Primary Health Care PNDIP National Development and Public Investment Plan (Plan Nacional de Desarrollo e Inversión Pública) RF Results Framework SASHI Strategic Agenda for Strengthening the Health Insurance (Agenda Estratégica para el Fortalecimiento del Seguro de Salud) SIFF Integrated Family Record System (Sistema Integrado de Ficha Familiar) SINIRUBE Single System of State Beneficiaries (Sistema Nacional de Información y Registro Único de Beneficiarios) SISA Environmental Sustainability Information System (Sistema de Información de Sostenibilidad Ambiental) ToC Theory of Change The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT TABLE OF CONTENTS DATA SHEET ................................................................................................................................................. i I. PROGRAM CONTEXT AND DEVELOPMENT OBJECTIVES .....................................................................................1 A. CONTEXT AT APPRAISAL ................................................................................................................................................ 1 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ......................................................................... 7 II. OUTCOME .......................................................................................................................................................8 A. RELEVANCE OF PDOs AND DLIs ..................................................................................................................................... 8 B. ACHIEVEMENT OF PDOs (EFFICACY) ............................................................................................................................ 10 C. JUSTIFICATION OF OVERALL OUTCOME RATING ........................................................................................................ 20 D. OTHER OUTCOMES AND IMPACTS (IF ANY) ................................................................................................................ 20 For Official Use Only III. KEY FACTORS AFFECTED IMPLEMENTATION AND OUTCOME .......................................................................... 24 A. KEY FACTORS DURING PREPARATION......................................................................................................................... 24 B. KEY FACTORS DURING IMPLEMENTATION.................................................................................................................. 25 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .................................... 27 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................................................... 27 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ....................................................................................... 28 C. BANK PERFORMANCE .................................................................................................................................................. 29 D. RISK TO DEVELOPMENT OUTCOME ............................................................................................................................ 30 V. LESSONS AND RECOMMENDATIONS .............................................................................................................. 30 ANNEX 1. RESULTS FRAMEWORK, DISBURSEMENT LINKED INDICATORS, AND PROGRAM ACTION PLAN .................... 32 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION .............................................................. 39 ANNEX 3. PROGRAM EXPENDITURE SUMMARY ........................................................................................................ 41 ANNEX 4. BORROWER’S COMMENTS ........................................................................................................................ 42 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT DATA SHEET @#&OPS~Doctype~OPS^dynamics@icrpfrbasicdata#doctemplate BASIC DATA Product Information Operation ID Operation Name P148435 Strengthening Universal Health Insurance in Costa Rica Product Operation Short Name Program-for-Results Financing (PforR) STRENGTHENING UNIVERSAL HEALTH INSURANCE Operation Status Approval Fiscal Year For Official Use Only Closed 2016 IPF Component? Original EA Category CLIENTS Borrower/Recipient Implementing Agency Republic of Costa Rica Caja Costarricense del Seguro Social DEVELOPMENT OBJECTIVE Original Development Objective (Approved as part of Approval Package on 14-Mar-2016) The objectives of the Program are to contribute to: (i) improving the timeliness and quality of health services; and (ii) enhancing the institutional efficiency of the CCSS. @#&OPS~Doctype~OPS^dynamics@icrfinancing#doctemplate FINANCING i The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Financing Source Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 420,000,000.00 420,000,000.00 420,000,000.00 IBRD-85930 420,000,000.00 420,000,000.00 420,000,000.00 Total 420,000,000.00 420,000,000.00 420,000,000.00 RESTRUCTURING AND/OR ADDITIONAL FINANCING Amount Disbursed Date(s) Type Key Revisions (US$M) • Loan Closing Date Extension 29-Apr-2022 Portal 390.00 • Implementation Schedule 20-Dec-2023 Portal 420.00 • Loan Closing Date Extension @#&OPS~Doctype~OPS^dynamics@icrkeydates#doctemplate For Official Use Only KEY DATES Key Events Planned Date Actual Date Concept Review 26-Feb-2015 26-Feb-2015 Decision Review 01-Oct-2015 01-Oct-2015 Authorize Negotiations 06-Jan-2016 08-Jan-2016 Approval 15-Mar-2016 15-Mar-2016 Signing 21-Apr-2016 Effectiveness 18-Nov-2016 17-Nov-2016 ICR/NCO -- Restructuring Sequence.01 Not Applicable 29-Apr-2022 Restructuring Sequence.02 Not Applicable 20-Dec-2023 Mid-Term Review No. 01 16-Mar-2020 20-Oct-2020 Operation Closing/Cancellation 31-Jan-2024 31-Jan-2024 @#&OPS~Doctype~OPS^dynamics@icrratings#doctemplate RATINGS SUMMARY Outcome Bank Performance M&E Quality ii The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Highly Satisfactory Highly Satisfactory High ISR RATINGS Actual Disbursements No. Date ISR Archived DO Rating IP Rating (US$M) 01 20-Jun-2016 Satisfactory Satisfactory 0.00 02 21-Dec-2016 Satisfactory Satisfactory 0.00 03 28-Jun-2017 Satisfactory Satisfactory 105.00 04 27-Dec-2017 Satisfactory Satisfactory 105.00 05 26-Jun-2018 Satisfactory Moderately Satisfactory 105.00 06 26-Dec-2018 Satisfactory Satisfactory 225.00 For Official Use Only 07 26-Jun-2019 Satisfactory Satisfactory 225.00 08 19-Dec-2019 Satisfactory Satisfactory 285.00 09 19-Jun-2020 Satisfactory Satisfactory 390.00 10 14-Jan-2021 Satisfactory Satisfactory 390.00 11 29-Jul-2021 Satisfactory Moderately Satisfactory 390.00 12 10-Feb-2022 Satisfactory Satisfactory 390.00 13 10-Nov-2022 Satisfactory Satisfactory 390.00 14 16-May-2023 Satisfactory Satisfactory 420.00 15 21-Dec-2023 Satisfactory Satisfactory 420.00 @#&OPS~Doctype~OPS^dynamics@icrsectortheme#doctemplate SECTORS AND THEMES Sectors Adaptation Mitigation Major Sector Sector % Co-benefits (%) Co-benefits (%) Health Health 100 0 0 Themes Major Theme Theme (Level 2) Theme (Level 3) % iii The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Human Development Health System Health Systems and Policies 100 and Gender Strengthening For Official Use Only iv The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT ADM STAFF Role At Approval At ICR Practice Manager Daniel Dulitzky Tania Dmytraczenko Regional Director Luis Benveniste Jaime Saavedra Global Director Timothy Grant Evans Juan Pablo Eusebio Uribe Restrepo Practice Group Vice President Keith Hansen Mamta Murthi Country Director J. Humberto Lopez Michel Kerf Regional Vice President Jorge Familiar Carlos Felipe Jaramillo Fernando Xavier Montenegro ADM Responsible Team Leader Laura Di Giorgio Torres Co-Team Leader(s) ICR Main Contributor Maria Cecilia Zanetta I. PROGRAM CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Costa Rica stands out for being among the most politically stable, progressive, and prosperous countries of the Latin America and the Caribbean (LAC) region. The country’s development model had resulted in important economic, social, and poverty dividends, with sustained growth, upward mobility for a large share of the population, important gains in social indicators, and one of the lowest poverty rates in LAC. Using the poverty and extreme poverty lines of US$4 per day and US$2.5 per day respectively, just 12 percent of the Costa Rican population was considered poor in 2014 (less than half of the LAC average), 4.7 percent was considered extremely poor (about one-third of the LAC average).1 2. The country’s success was also reflected in health outcomes indicators, among the best in LAC. Costa Rica’s signature universal public health insurance (Seguro de Salud) managed by the Costa Rican Social Security Fund ( Caja Costarricense de Seguro Social - CCSS) had provided access to health care to its entire population, including the poor and bottom 40 percent. The country’s universal health care system was considered one of the key reasons for its strong health outcomes, with around 95 percent of the population formally enrolled. As in other high-middle income countries, however, emerging structural issues threatened the sustainability of Costa Rica’s public health sector achievements, including: (i) a demographic transition resulting in a rapidly aging population and increased prevalence of non- communicable diseases (NCDs), (ii) stagnation of quality-of-care improvements to the national network of primary health care (PHC) services to respond to NCDs and epidemics; and (iii) financial sustainability issues related to increasing health 1Acosta, P. (2015), Central America social expenditures and institutional review: Costa Rica, Report No.: 97416-CR, July 2, 2015, Washington, D.C.: World Bank Group. [http://documents.worldbank.org/curated/en/927631468011103545/Central-America-social-expenditures-and-institutional-review-Costa-Rica] Page 1 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT care costs. Sectoral and Institutional Context 3. Costa Rica’s public health insurance system is the largest health care provider in the country, being virtually the sole public provider of health services at all levels of care to the entire population. As the percentage of the population with formal insurance fluctuates around 95 percent, the government effectively subsidizes individuals from poor and indigent households that cannot afford any contribution to health insurance.2 The CCSS is also responsible for public health insurance revenue collection and pooling, which provides a single risk pool that allows cross-subsidization and the ability to use its purchasing power for negotiating better prices for several costly pharmaceuticals. As both the provider and insurer for the health services, the CCSS had unique comparative advantages in improving efficiency and quality of care, including having a body of decision makers at a centralized level to make strategic decisions and the ability to facilitate changes without the need for difficult inter-agency coordination. 4. The increasing prevalence of chronic conditions such as hypertension, diabetes, and different types of cancer, along with the growing demand for specialized care of an aging population was putting a strain on the health system. Likewise, recent global health emergencies, such as the Ebola crisis in 2015 and the then ongoing Zika crisis, had highlighted the need for a system flexible enough to respond to unexpected challenges. Other challenges included improving coordination among the PHC and specialized services (both outpatient and inpatient) to, in turn, eliminate bottlenecks, shorten waiting times, and increase patient satisfaction. 5. In 2014-2015, the CCSS carried out an in-depth review of the main institutional capacity challenges to improving quality and efficiency of care. Three priority areas of weakness were identified: (i) the delivery of health care had not fully adapted to the changing needs of the population, given the rise of NCDs; (ii) the CCSS required more effective institutional capacity to manage the system (e.g., streamlining the central management structure, strengthening the efficiency of budget allocation, ensuring a managed approach to strategic investments, and introducing tools for shifting from inputs towards a prospective resource allocation system); and (iii) financial management within the CCSS had to be improved and modernized to allocate resources more efficiently, including the development of financial information systems for both accounting and budgeting practices. 6. To guide the needed transformation of Costa Rica’s health system, the CCSS Board of Directors3 had approved its Strategic Agenda to Strengthen Health Insurance (Agenda Estratégica para el Fortalecimiento del Seguro de Salud - SASHI). The SASHI, developed using international best practices with the support of various national and international experts, including the World Bank, was approved by the CCSS Board of Directors in November 2014. With the dual goals of increasing the efficiency and quality of the CCSS health insurance system, it represented a comprehensive and ambitious institutional capacity building program with a focus on three priority areas: (i) strengthening the health care model to better integrate PHC with secondary level care in a given catchment area and network of providers in order to improve prevention, early diagnosis, and timely management of NCDs and ensure more efficient use of health care resources; (ii) enhancing the institutional management of the CCSS while increasing accountability and responsiveness to users; and (iii) adopting international best practices relevant to Costa Rica to improve financial management of its public health insurance. This operation provided financial support for the implementation of SASHI and related actions through a Program-for-Results (PforR), initially planned for six years but extended to seven. 2 World Bank (2016), Costa Rica - Program-for-Results: Strengthening Universal Health Insurance in Costa Rica, PAD, Washington, D.C.: World Bank Group. [http://documents.worldbank.org/curated/en/168981468184170068/Costa-Rica-Program-for-Results-Strengthening-Universal-Health-Insurance-in-Costa-Rica] 3 The CCSS Board of Directors oversees CCSS operations and is made up of representatives from the Government, trade unions, and the private sector. Page 2 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Theory of Change (Results Chain) 7. The Program Development Objectives (PDOs) were to contribute to: (i) improving the timeliness of health services; (ii) improving the quality of health services; and (iii) improving the institutional efficiency of CCSS.4 In turn, these PDOs were to contribute to the higher goal of enhancing the financial sustainability of Costa Rica’s public health insurance. 8. To achieve these PDOs, the operation’s design focused on the three priority areas under SASHI: (i) strengthening the health care model with emphasis on improving PHC via the strengthening of integrated health care networks and a more effective approach to prevention, early diagnosis, and control of chronic NCDs; (ii) improving institutional management; and (iii) optimizing financial management. 9. The three detailed Results Chains (one for each priority area) for SASHI as a whole are included in the Program Appraisal Document (PAD).5 These, in turn, fully captured the operation’s Theory of Change (ToC), showing how the Program activities within each Priority Area (shown in the boxes with the dark blue border) were expected to contribute to the achievement of the Intermediate Results (IRs) and the three PDOs (see Figures 1.a, 1.b, 1.c). It should be noted that, under SASHI’s ToCs, PDOs 1 and 2 are “packaged” into a single PDO. 10. There were several implicit assumptions underlying the operation’s ToC, including: (i) shared support and adequate institutional capacity across the CCSS to ensure the Program’s successful implementation; and (ii) no drastic changes in external conditions (e.g., continuous political and macroeconomic stability, no natural disasters or health emergencies. Figure 1. Theory of Change 1.a Priority Area 1 - Health Care Model With PforR support 4 The original PDO description was “improving the timeliness and quality of health services and the institutional efficiency of the CCSS. For the purpose of this ICR, it was “unpacked” into the three different PDOs described above. This is in line with the I CR Preparation Guidelines for PforR Operations (see Bank Guidance - Implementation Completion and Results Report (ICR) for Program- for-Results (PforR) Operations; OPS5.04-GUID.119, December 13, 2021; paragraph 27). 5 World Bank (2016), Costa Rica - Program-for-Results: Strengthening Universal Health Insurance in Costa Rica, PAD, Report No. 99213, February 18, 2016, Washington, D.C.: World Bank Group. (See pp. 34, 35 and 36.) [http://documents.worldbank.org/curated/en/168981468184170068/Costa-Rica-Program-for-Results-Strengthening-Universal-Health-Insurance-in-Costa-Rica] Page 3 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT 1.b Priority Area 2 - Institutional Management With PforR support 1.c Priority Area 3 - Financial Management With PforR support Rationale for PforR Support, and Program Scope and Boundaries 11. The PforR operation was fully aligned with the 2016-2020 Country Partnership Framework (CPF),6 which aimed to support the Government of Costa Rica’s (GoCR) objectives of reducing constraints to productive inclusion and bolstering fiscal, social, and environmental sustainability, as reflected in its 2015-2018 National Development Plan. 7 The PforR remains relevant also under the new CPF, which supports, among others: (i) enhancing fiscal and debt management, (ii) strengthening and preserving human capital in the face of shocks, (iii) enhancing equity and inclusion; and (iv) bridging 6 World Bank (2015), Costa Rica - Country partnership framework for the period FY2016-20, Report No. 94686-CR, April 23, 2015; Discussed by the Executive Directors on May 26, 2015; Washington, D.C.: World Bank Group. [http://documents.worldbank.org/curated/en/363291468179335481/Costa-Rica-Country-partnership- framework-for-the-period-FY2016-20] 7 Ministerio de Planificación Nacional y Política Económica (2015), Plan Nacional Desarrollo 2015-2018, San Juan: Costa Rica. [https://www.mideplan.go.cr/Plan- Nacional-Desarrollo-2015-2018] Page 4 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT institutional gaps. Likewise, the World Bank Group’s 2015 Systematic Country Diagnostic for Costa Rica8 had identified the need to contain the growth trajectory of key public expenditures, notably within the CCSS, as critical for ensuring fiscal and social sustainability of both the public health insurance and overall Government expenditure. Moreover, the PforR was also fully aligned with the World Bank’s corporate twin goals of ending extreme poverty and boosting shared prosperity. Specifically, by improving the quality and timeliness of health services, the operation aimed to improve the living standards of all Costa Ricans, including those in the bottom 40 percent, and to reduce inequality. Similarly, ensuring the CCSS financial sustainability was a prerequisite to continue its mission to provide services to the population, irrespective of their ability to pay. 12. The PforR was considered the most suitable lending instrument to support CCSS’ strategic agenda for several reasons. First, the CCSS had already developed and had full ownership of SASHI, the comprehensive program for improving the health insurance model supported by the PforR. Moreover, the CCSS sought to shift emphasis from the management of program inputs to management of program results and risks. Second, the GoCR aimed to use its own country systems in the implementation of the PforR in support of SASHI. The World Bank’s assessments confirmed that the national program systems had the necessary capacity to successfully implement the operation. 13. This PforR provided financial support to the GoCR for the implementation of SASHI over a seven-year period. The total cost of implementing SASHI from 2016-2023 was estimated at US$2.14 billion, including US$565 million for hospital replacement. Hospital replacement was outside the boundaries of the Program supported by this PforR. Thus, the Program supported by this PforR amounted to US$1.575 billion, 9 with US$420 million (i.e., 26.7 percent) being financed by this PforR and the remaining US$1.155 billion (i.e., 73.3 percent) by CCSS with its own sources of funding. Program Development Objectives (PDOs) 14. The PDOs were to contribute to: (i) improving the timeliness of health services; (ii) improving the quality of health services; and (iii) improving the institutional efficiency of CCSS.10 In turn, these PDOs were to contribute to the higher goal of enhancing the financial sustainability of Costa Rica’s public health insurance. Key Expected Outcomes and Outcome Indicators 15. Four PDO Indicators (PDOIs) were adopted to assess PDO achievement (see Table 1). In addition, eight Intermediate Results Indicators (IRIs) measured other important intermediate steps in the implementation of SASHI and PDO achievement (see Annex 1.a for the complete Program’s Results Framework - RF). Table 1. Program Outcome Indicators PDOIs Baseline End-Target PDOI 1. Percentage of major surgeries from priority list conducted in outpatient 18% 40% settings according to institutional guidelines. 8 World Bank (2015), Costa Rica’s Development: From Good to Better, Systematic Country Diagnostic, June 2015, Washington, D.C.: World Bank Group. [https://doi.org/10.1596/22023] 9 Excluding the cost of hospital replacement, the implementation of SASHI over an originally planned six-year period amounted to US$1.575 billion (i.e., US$2.14 billion minus US$565 million for hospital replacement). 10 The original PDO description was “improving the timeliness and quality of health services and the institutional efficiency of the CCSS. For the purpose of this ICR, it was “unpacked” into the three different PDOs described above. This is in line with the ICR Preparation Guidelines for PforR Operations (see Bank Guidance - Implementation Completion and Results Report (ICR) for Program- for-Results (PforR) Operations; OPS5.04-GUID.119, December 13, 2021; paragraph 27). Page 5 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT PDOI 2. Cumulative percentage of target population personally invited to undergo 0% 40% colon cancer screening in the five priority counties.11 PDOI 3. Percentage of the total number of primary health care units ( Áreas de Salud) with the Unique Digital Health Record e-health package (Expediente Digital 50% 80% Único de Salud - EDUS). Annual budget Annual budget PDOI 4. Development and execution of a comprehensive medium- and long-term does not use execution using plan to ensure the financial sustainability of the CCSS. prospective tools prospective tools Program Results Areas and DLIs The PforR supported the implementation of SASHI’s three priority areas: 16. Priority Area 1 – Health Care Model: By supporting the implementation of SASHI, this priority area aimed to boost the capacity of PHC to prevent and control NCDs and to better integrate PHC with secondary-level care. Key activities included: (i) expanding infrastructure and equipment at the primary and second levels of care; (ii) improving human resources to attend to the needs of patients; (iii) upgrading equipment; (iv) updating clinical guidelines and pathways, with an emphasis on chronic conditions; and (v) increasing the use of household data collected by the PHC teams. In addition, a pilot program was to be carried out to test new mechanisms for integrating primary and second-level services (i.e., hospitals) through integrated healthcare networks, including the development of new e-Health tools to facilitate information exchanges between medical staff at different levels of care. 17. Priority Area 2 – Institutional Management: This priority area aimed to enhance the institutional management of the CCSS while increasing accountability and responsiveness to users. A key activity under this priority area was the development of a new institutional strategic intelligence system by digitizing existing heath data and linking to existing E- Health tools to allow for data triangulation that would enable directors and managers to allocate resources more efficiently and equitably, monitor results, and improve patient satisfaction. Other activities supported under Priority Area 2 included the systematic measurement of patient satisfaction and implementation of improvements plans at the facility level, as well as the geo-referencing of Family Socio-Economic and Health Risks Information File (the so-called Ficha Familiar). 18. Priority Area 3 – Financial Management: This priority area aimed to enhance the management of financial risks affecting the public health insurance. Key activities included: (i) the identification of new strategies and mechanisms to monitor financial sustainability; and (ii) development of new financial sustainability strategy that considers the redesign of the financing model, a new model for resource allocation, and improvements in the efficiency of the management of health services. 19. The disbursement of funds under this PforR was based on seven Disbursement-Linked Indicators (DLIs) that were selected by the CCSS and agreed upon with the World Bank (see Table 2). In turn, each DLI included two or three sub- indicators (milestones), with predetermined disbursements linked to their achievement. 11 Priority counties (“cantones”) were those with the highest incidence of colon cancer. Page 6 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Table 2. Disbursement-Linked Indicators Disbursements Disbursement-Linked Indicators (DLIs) US$ million Expected Results (% of total proceeds) Priority Area 1 - Health Care Model DLI 1 - Percentage of major surgeries from Expansion of major surgeries in ambulatory priority list conducted in outpatient settings 60.0 (14.3%) settings would free up resources and reduce according to CCSS institutional guidelines. waiting time. DLI 2 - Cumulative percentage of target Early detection of colon cancer would improve population personally invited to undergo colon 60.0 (14.3%) health outcomes, contain costs, and improve cancer screening in five priority counties. quality of care. DLI 3 - Percentage of individuals diagnosed with Improvement of quality of care of individuals Diabetes Mellitus Type II that are under optimal 60.0 (14.3%) diagnosed with diabetes. clinical control. Design, implementation, and evaluation of DLI 4 - Pilot project on integrated health pilot for a new health care network networks approved by the CCSS Board and management model, improving continuity of implemented for a selected population and 60.0 (14.3%) care, efficiency in management of patient and territory and evaluated with the results publicly use of resources, and patient disseminated. outcomes/satisfaction, among others. DLI 5 - Percentage of total number of primary 60.0 (14.3%) Expansion of EDUS coverage. health care units (Áreas de Salud) with EDUS. Priority Area 2 – Institutional Management DLI 6 - Redesign, implementation, and use of data collected by the new annual survey to Evaluation of the results of the interventions 60.0 (14.3%) measure impact of new interventions to to improve patient satisfaction. improve patient satisfaction. Priority Area 3 – Financial Management DLI 7 - Development and execution of a Improvement in institutional capacity for comprehensive medium- and long-term plan to 60.0 (14.3%) financial management and budgeting. ensure the financial sustainability of the CCSS. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets, Result Areas, and DLIs 20. The PDOs and the Program’s results areas remained unchanged throughout the operation’s lifetime. The operational definitions, baselines, and end-targets for the PDOIs and DLIs also remained unchanged. The only exception was the upward modification of DLI 1’s baseline.12 This was done through the first Level-2 Restructuring of the Program13, which was approved on August 2, 2018. As shown in Table 3, DLI1 was revised to increase the baseline (from 18 percent to 41.6 percent), DLI 1.1 target (from 20 percent to 43 percent), and the end-of-program target (from 40 percent to 46 12The Original Loan Agreement (dated April 21, 2016) was amended under an Amendment Letter (dated August 2, 2018) to introduce these changes to DLI 1. 13 The first restructuring of the Project was handled offline due to technical difficulties and was not recorded in the system. However, the documentation has been uploaded to the Image Bank. Page 7 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT percent). The changes were the result of both the identification of a conceptual error in the methodology for defining DLI 1 during Program preparation during the first supervision mission; as well as desire to increase consistency of the proposed new end-of-Program targets with both international good practices and standards and CCSS plans to expand ambulatory surgery at the same levels and by OECD standards. Further, the new goal also considered internal factors affecting performance, such as the availability of anesthesiologists and other medical specialists in the country. 14 Table 3. Changes to Disbursement-Linked Indicators under the 2018 Amendment Targets Changes to DLI 1 Baseline DLI 1.1 DLI 1.2 Percentage of major surgeries from priority list Original 18.0% 20.0% 40.0% conducted in outpatient settings according to CCSS institutional guidelines. Revised 41.6% 43.0% 46.0% Other Changes 21. Restructuring 2 - A Level-2 Restructuring was approved on April 29, 2022, to extend the closing date by twenty months, from April 30, 2022, to December 31, 2023. The extension aimed to provide sufficient time to enable the achievement and verification of the DLIs that were still pending and to make up for the delays resulting from the COVID- 19 pandemic and the cyberattack that affected the CCSS information systems in May 2022 (see Section III for more detailed information). 22. Restructuring 3 - A Level-2 Restructuring was approved on December 18, 2023, to extend the closing date by one month, from December 31, 2023, to January 31, 2024. Considering the previous extension, the operation’s closing date was extended for 21 months. This extension was requested to ensure that sufficient time was provided for the achievement of DLI 7.3 (i.e., Annual budget of CCSS formulated using prospective budgeting tools), which required approval by the CCSS Board of Directors and the Republic General Comptroller, which were pending at the time of the request. Rationale for Changes and their Implication on the Original Theory of Change 23. There were no changes to the original ToC. II. OUTCOME A. RELEVANCE OF PDOs AND DLIs Assessment of Relevance of PDOs and Rating Rating: High 24. The operation has remained fully aligned with both the Bank’s and the GoCR’s strategic priorities. Specifically, 14While revised targets called for improvements more modest than the original ones, they reflected the fact that improvements with respect to a significantly larger baseline are harder to achieve. Page 8 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT PDOs continue to be in alignment the Costa Rica’s 2016-2020 CPF,15 which is the same that was in place at the time of preparation.16 Its aim was to support the GoCR’s objectives of reducing constraints to productive inclusion and bolstering fiscal, social, and environmental sustainability. The operation’s PDO s provided support to one of the CPF’s two pillars, i.e., Pillar 2 - Bolstering Fiscal, Social & Environmental Sustainability. Within this pillar, the PDOs directly support Objective 5 - Improve efficiency and quality of the health insurance system to improve results. In addition, the operation’s PDOs are also fully aligned with the World Bank’s corporate twin goals of eradicating extreme poverty and boosting shared prosperity. Specifically, by improving the quality of health services, the operation has contributed to improving the living standards of all Costa Ricans, including those in the bottom 40 percent. 17 Moreover, by helping secure the CCSS’ financial sustainability, the operation has helped ensure the continuity of Costa Rica’s universal insurance system, in particular the cross-subsidization of the poorest. 25. The PDOs are also in full alignment with Costa Rica’s latest National Development and Public Investment Plan (Plan Nacional de Desarrollo e Inversión Pública - PNDIP),18 which lays out the strategic priorities for the development of the country during the 2023-2026 period. The PNDIP’s objective is to generate well-being for all people living in the national territory through the implementation of concrete strategies and public policy actions that impact on the sustainable development of the country and on improvements in the quality of life. Within the health sector, the PNDIP’s aims to: (i) improve the health conditions of the population reflected in the number of years lived without disease as a result of the timely and integrated provision of health care services; and (ii) expand the coverage of the public health insurance. 19 In turn, the achievement of objectives is to be supported by several of the lines of actions that were supported by this PforR focusing on: (i) strengthening of the provision of health services and the development of integrated healthcare networks in the CCSS; (ii) implementing the national strategy for the integral approach to chronic NCDs and obesity; (iii) enhancing access and timeliness of health services, including cancer screening and ambulatory surgeries; and (iv) developing interoperable technological infrastructure that facilitates decision making. The PDOs continue to be fully consistent with the CCSS’s 2014 Strategic Development Plan (i.e., CCSS 2041: Una Mirada al Futuro), with the PDOs being reflected in its guiding principles, including guaranteeing: (i) the comprehensive implementation of PHC; and (ii) the financial and actuarial sustainability of the institution in the long term, considering the quality and timeliness of health care services.20 26. With a focus on improving the timeliness and quality of health care services, PDOs 1 and 2 are also central to the attainment of the United Nations’ Sustainable Development Goal 3 – Good Health and Wellbeing for All at All Ages, in particular Target 3.4 (i.e., to reduce premature mortality from NCDs through prevention and treatment) and Target 3.8 (i.e., to achieve universal health coverage, including financial risk protection, access to quality essential health-care services and access to safe, effective, quality and affordable essential medicines and vaccines for all). 15 The CPF was discussed by the Executive Directors on May 26, 2015 (Report No. 94686-CR). [https://documents1.worldbank.org/curated/en/363291468179335481/pdf/94686-CPS-P149582-R2015-0088-IFC-R2015-0118-MIGA-R2015-0027-Box391422B-OUO-9.pdf] 16 A new CPF is currently under preparation; the delays in preparing a new CPF were due to (i) the need to assess the duration and impacts of the COVID-19 pandemic, and the change in administration and time required to set priorities. 17 The use of health care services by the bottom 40 percent is high, with 87 percent of individuals in the first quintile (i.e., the poorest quintile) and 85 percent of those in the second quintile reporting use of outpatient health services through the CCSS. See World Bank (2015), Costa Rica: Estudio de Gasto Publico Social y sus instituciones – Educación, Salud, Protección Social y Empleo, World Bank Group. Washington, DC. [https://documents1.worldbank.org/curated/fr/168981468184170068/text/99213-PAD-P148435-OUO-9-R2016-0028-1.txt] 18 Ministerio de Planificación Nacional y Política Económica (2022), Plan Nacional de Desarrollo e Inversión Pública 2023-2026, December 2022, MIDEPLAN, San José, Costa Rica. [https://drive.google.com/file/d/1otcCNQGgjEKDl5hMEA8lG--RTmgzY6yK/view] 19 In addition to the two objectives mentioned above, the PNDIP 2023-2026 also aims to improve the coverage of the economically active population for economic and social protection in the face of the risks of disability, old age and death. 20 CCSS (2023), CCSS 2041: Una Mirada al Futuro, Presidencia Ejecutiva, Dirección de Planificación Institucional, CCSS, San José, Costa Rica. [https://www.ccss.sa.cr/web/flip/documents/mirada/pdf/full.pdf] Page 9 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Assessment of Relevance of DLIs and Rating Rating: High 27. The relevance of the DLIs is deemed High. The operation included seven DLIs that provided a solid platform for assessing progress toward the implementation of SASHI, with a focus on both performance outcomes (i.e., DLIs 1, 2, 3, 4) as well as relevant actions contributing to desired institutional change (DLIs 5, 6, 7). Together, they fully supported the operation’s ToC (see Figure 1), providing financial incentives for the achievement of intermediate results conducive to the PDOs. The criteria by which they were selected contributed to their relevance, including: (i) full alignment with CCSS priorities and correspondence to the key priority areas in SASHI, providing incentives for removing bottlenecks in the ToC; (ii) results-focus to stimulate performance; (iii) prioritization of government’s routine information systems and existing reporting mechanisms; (iv) balance between ambition (“stretch”) and feasibility (“realism”); and (v) a reasonably even distribution of disbursements, with some of the selected DLIs being designed to be disbursed proportionally to the quantitative achievement of the results (i.e., “scalability”). Rating of Overall Relevance 28. The operation’s overall relevance is deemed High, to reflect the high relevance of both its PDO and the DLIs. B. ACHIEVEMENT OF PDOs (EFFICACY) Methodological Notes 29. As noted earlier, under this Implementation Completion and Results Report (ICR), the original PDO 21 is “unpacked” into the following PDOs: PDO 1 – Improving the timeliness of health services; (ii) PDO 2 – Improving the quality of health services; and (iii) PDO 3 - Enhancing the institutional efficiency of the CCSS. 30. The drastic negative impact that the COVID-19 pandemic had on the delivery of non-urgent health services worldwide, including Costa Rica, required further analyses to determine the efficacy assessment of PDOs 1 and 2, which focus on service delivery, the most affected by the pandemic. In this specific case, DLI targets of PDOs 1 and 2 were achieved within the originally established timeline, prompting timely disbursements. The advent of COVID-19 led to an extension of the operation’s timeline to accommodate the achievement of the remaining indicators. Notably, the performance of PDOs 1 and 2, the targets of which had been met, deteriorated during this extended period. To assess efficacy of PDOs in this extraordinary implementation environment, this ICR assesses not only DLI achievement, but also the operation’s resilience and ability to regain and surpass deteriorated achievements. Evidence of recovery by closing is considered indicative of sustained results and institutional adaptability. It should be noted that this ICR extends the modifications made to DLI1’s baseline and targets 22 to PDOI 1, which was formalized through the first program restructuring. If considering PDOI 1’s original baseline (18 percent) and target (40 percent), actual achievements amply exceeded the target. Assessment of Achievement of Each Objective/Outcome 21The original PDO definition was to contribute to: (i) improving the timeliness and quality of health services; and (ii) enhancing the institutional efficiency of the CCSS. 22DLI 1’s original baseline and targets were 18 percent (baseline); and 20 percent (DLI1 1.1) and 40 percent (DLI 1.2). As noted earlier, the actual DLI 1 baseline was subsequently determined to be 41.6 percent, for which targets were subsequently adjusted to 43 percent (DLI 1.1) and 46 percent (DLI 1.2) through an Amendment Letter. Page 10 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT 31. PDO 1 - Improving the timeliness of health services. Rating: High 32. Definition - The operation provided financial incentives for a shift toward ambulatory care, with PDOI 1/DLI 1 focusing on the percentage of major surgeries from a priority list performed in ambulatory settings during the preceding six months.23 In turn, this shift was expected to improve the timeliness of health services by significantly reducing waitlist times (IRI 1), defined as the average number of waiting days in national hospitals for hip and knee replacement. 33. Results - The operation’s contribution toward improving the timeliness of health care has been High, with PDOI 1/DLI 1 exceeding the anticipated targets, both at the time of DLI achievement and at closing (see Table 4). The shift toward ambulatory surgeries also resulted in a reduction in the average waiting time for major surgeries in general and those specifically tracked that exceeded the anticipated target (IRI 1). In addition, the shift toward ambulatory care can also be expected to have resulted in quality improvements by reducing exposure to intra-hospital infections and shorter recovery times from less invasive procedures among others as well as efficiency gains by performing surgeries in less expensive settings and freeing up resources in higher intense care settings. The expanded ambulatory surgical capacity was mainly the result of the optimization of existing capacity of individual hospitals, with guidance and technical support from the Major Outpatient Surgery Commission.24 34. Specific achievements under PDO 1 under the operation’s RF are summarized in Table 4. 23The priority list included: varicotomies, hernias, salpingectomies, bone biopsies, osteo-synthetic devices removal, and laparoscopic cholecystectomies. 24Also see Rosado Valenzuela, A.L.; Sheffel, A.; Mussini, M.; Lara Salinas, A.M. and Di Giorgio, L. (2023), Increasing the Number of Major Outpatient Surgeries to Reduce the Waiting List in Costa Rica. Knowledge Brief, Health, Nutrition and Population Global Practice, October 2023, Washington, D.C.: World Bank Group. [http://hdl.handle.net/10986/40520] Page 11 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Table 4. PDO 1 Achievements PDO 1 – Improving the timeliness of health services. % of Target Achievement OUTCOMES At date of DLI At closing Achievement Shift to Ambulatory Care • PDO 1/DLI 1 - Percentage of major surgeries from priority list conducted in outpatient settings according to CCSS institutional guidelines (41.6 percent baseline; 46 percent end-target). At the time of DLI verification – Exceeded: The verification of DLI 1 achievement conducted by the Pan-American Health Organization (PAHO) determined that 187% 49.8 percent of major surgeries from the priority list were conducted in outpatient settings between April-September 2018 compared to the 41.6 percent baseline (equivalent to 187 percent level of achievement with respect to the 46 percent target).25 At closing - Exceeded: Showing a strong recovery after the severe drop in the production of ambulatory surgeries because of the COVID-19 pandemic, the percentage of major surgeries from the priority list conducted in outpatient 200% settings according to CCSS institutional guidelines reached 50.4 percent during the period January-September 2023 (equivalent to 200 percent level of achievement with respect to the 46 percent target).26 • IRI 1 - Cumulative percentage reduction in average waiting days for selected procedures (tracers): hip and knee replacement (0 percent baseline; 35 percent end-target). At the time of DLI verification – Exceeded: The average waiting time for hip and 169% knee replacement decreased by 59.1 percent (from 1,032 days to 422 days) between 2015 and 2018 (equivalent to a 169 percent level of achievement with respect to the 35 percent target). At closing - Exceeded: Despite the negative impact of the COVID-19 pandemic, the average waiting time for hip and knee replacement decreased by 38.8 111% percent (from 1,032 days to 632 days) between 2015 and 2023 (equivalent to a 111 percent level of achievement with respect to the 35 percent target). 27 INTERMEDIATE RESULTS AND OUTPUTS • Ambulatory surgical capacity was expanded through the optimization of existing capacity. Establishment of the Major Ambulatory Surgery Commission in July 2017 to oversee and support the implementation of the 2015 Institutional Strengthening Plan for Ambulatory Surgery. Specific outputs include, among others, the preparation of the Ambulatory Surgeries Operational Manual, which was submitted to the CCSS Medical Management in November 2018 as well as the monitoring of waiting lists for surgical procedures in collaboration with the CCSS’s Waiting Lists Technical Unit. 25 If considering PDOI 1’s original baseline (18 percent) and target (40 percent), actual achievements still exceed the target (equivalent to 145 percent and 147 percent level of achievement in 2018 and 2023, respectively). 26 CCSS, Area of Health Statistics, as reported by the CCSS Gerencia Médica, Comisión Cirugía Mayor Ambulatoria, Nov. 6, 2023. 27 CCSS, Area of Health Statistics, as reported by the CCSS Gerencia Médica, January 18, 2024. Page 12 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT 35. Additional evidence – The evolution of the total number of surgeries from 2013 onwards, including during the COVID-19 pandemic, provides additional evidence of the shift toward ambulatory care as a result of the reforms implemented under SASHI.28 After being largely stagnant between 2013 and 2018, the total number of major surgeries increased significantly between 2018 and 2019, with the increase driven by the number of surgeries conducted in ambulatory setting, reflecting the successful implementation of the reforms supported by SASHI (see Figure 2). Specifically, the total number of major surgeries (both ambulatory and non-ambulatory) increased by 5.3 percent between 2018 and 2019, compared to an average annual growth of only 0.4 percent between 2013 and 2018. Most remarkable, however, was the growth in the total number of ambulatory major surgeries, which increased by 17 percent between 2018 and 2019, compared to 2.5 percent for non-ambulatory major surgeries during the same period.29 36. However, as indicated by a report prepared by the CCSS at the request of the Ombudsman's Office, the COVID-19 pandemic had a devastating impact on the delivery of health services and waiting time in Costa Rica.30 In the case of surgeries, such report indicates that, from March to December 31, 2020, only 40,590 out of the 102,000 scheduled surgeries were carried out (i.e., equivalent to 60 percent of scheduled surgeries being cancelled).31 The impact of the COVID-19 pandemic was particularly pronounced in the case of major surgeries performed in ambulatory setting, which dropped by 55.9 percent between 2019 and 2020 compared to a 20.7 percent drop for non-ambulatory ones (see Figure 2). 37. Following the drastic drop in 2020, however, the production of major surgeries rebounded quickly once the first wave of the COVID-19 pandemic subsided. Specifically, the number of non-ambulatory surgeries performed annually increased by 11.6 percent between 2020 and 2022, to reach 88.4 percent of its peak 2019 value equivalent to an 18 percent increase. This rebound, however, was much stronger for ambulatory ones, which increased by 109.9 percent between 2020 and 2022 to reach 92.7 percent of its peak 2019 value.32 28 Note that DLI 1 and IRI are based on “selected” surgeries (i.e., hip and knee replacement only). 29 CCSS’s Annual Health Statistics 2013, 2022 (CCSS Area of Health Statistics). 30 Pacheco Jimenez, J.F. and Itriago Henriquez, D. (2022), Monitoring of the indicators of coverage, quality and financing of health services , Report of the Nation in Sustainable Human Development 2022, Report State of the Nation 2022, San José, Costa Rica. 31 Pacheco Jiménez at al. (2022). 32 CCSS, Anuarios Estadísticos 2013, 2022. Page 13 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Figure 2. Annual Number of Major Surgeries - Total and Ambulatory (2013 – 2022) 400,000 367,264 343,093 350,000 354,575 348,817 359,085 348,801 340,078 263,387 300,000 328,049 296,196 286,099 290,844 288,367 280,931 278,300 282,786 250,000 281,335 254,945 242,566 228,551 200,000 150,000 100,000 78,897 62,162 73,104 50,000 68,476 68,241 67,466 53,630 61,778 66,031 34,836 0 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 All Major Surgeries Ambulatory Major Surgeries Non-Ambulatory Major Surgeries Source: Own elaboration based on CCSS’s Annual Health Statistics 2013, 2022 (CCSS Area of Health Statistics). 38. PDO 2 - Improving the quality of health services. Rating: High 39. Definition - The operation provided financial incentives to improve the quality of health services by focusing on enhancing NCD detection, monitoring, and control, in particular for colon cancer, Diabetes Mellitus Type II (DM2) and hypertension. Specifically: ▪ Colon cancer - The operation provided financial incentives for the implementation of a large-scale colon cancer screening program in five priority counties that had been selected based on the high prevalence of colon cancer. Specifically, DLI 2/ PDO2 focused on the percentage of people aged 50 to 70 living in the five priority counties that were personally invited to undergo colon cancer screening (i.e., fecal occult blood test). ▪ Diabetes Mellitus - The operation provided financial incentives for improvements in the quality of care of individuals diagnosed with DM2. Specifically, DLI 3 focused on the percentage of individuals diagnosed with DM2 under optimal clinical control -- i.e., those with a hemoglobin A1C (HBAc1) test score below or equal to 7 percent in the last year. ▪ Hypertension - The operation’s RF (IRI 4) also focused on the percentage of individuals ages 20 to 64 diagnosed with hypertension under optimal clinical control -- i.e., those with a standard blood pressure target of less than 140/90 mmHg. 40. Results - The operation’s contribution toward improving the quality of health care, particularly with regards to the detection, monitoring, and treatment of NCDs is deemed Substantial. All end-Program targets corresponding to outcome indicators were fully achieved or exceeded by the time of DLI verification (i.e., PDI 2/DLI 2; DLI 3; and IRI 2). While the percentage of patients diagnosed with DM2 and hypertension under optimal clinical control dropped significantly during 2020 against the backdrop of the COVID-19 pandemic (i.e., DLI 3 and IRI 2), both have exhibited a strong upward trend from 2020 onwards and are likely to re-achieve the anticipated targets in the short term (see Table 5). As further explained below, it is important to note that the 2022 data on service provision is likely to be underestimated as a result Page 14 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT of the massive cyberattack which hit the CCSS that year. 41. These achievements were the result of the implementation of actions under SASHI focusing on NCDs and their risk factors, such as the preparation and dissemination of clinical guidelines for DM2, hypertension and obesity. In addition, the operation provided financial incentives for the adoption of new E-Health tools (DLI 5/PDOI 3) to facilitate information exchanges between PHC medical staff and specialists, thus contributing, among others, to the integration of primary and secondary levels of care. Finally, the operation provided financial incentives (DLI 4) for the implementation of a pilot program to test new mechanisms for integrating PHC with second-level services (i.e., hospitals) to improve the navigation of the patient across levels of care. The DLI and PDOI targets corresponding to these intermediate results were achieved or exceeded (see Section II.D for a detailed description). 42. Specific achievements under PDO 2 under the operation’s RF are summarized in Table 5: Table 5. PDO 2 Achievements PDO 2 – Improving the quality of health services. % of Target Achievement OUTCOMES At date of DLI At closing Achievement Colon cancer screening • PDO 2/DLI 2 – Cumulative percentage of target population personally invited to undergo colon cancer screening in five priority counties (0 percent baseline; 40 percent end- target). At the time of DLI verification – Achieved: The verification of DLI 2 achievement conducted 100% by the PAHO determined that by July 2022, a total of 13,592 people aged 50 to 70 living in the five priority counties had been personally invited to undergo colon cancer screening, representing 40.1 percent of the target population (equivalent to 100 percent level of achievement with respect to the 40 percent target). Optimal clinical control of Diabetes Mellitus • DLI 3 – Percentage of individuals diagnosed with Diabetes Type II that are under optimal clinical control (39 percent baseline; 43 percent end-target). At the time of DLI verification - Exceeded: Achievement of the DLI 3.1 target was verified in 2019, with 42.5 percent of DM2 patients receiving optimal care in 2017, (exceeding the 41 percent DLI 1.3 target). Likewise, achievement of the DLI 3.2 target was verified in 2022, with 45.7 percent of DM2 patients receiving optimal care in 2019 (exceeding the 43 168% percent DLI 3.2 target). Thus, the percentage of individuals diagnosed with DM2 under optimal clinical control increased from 39 percent to 45.7 percent patients between 2013 and 2019 (equivalent to 168 percent level of achievement with respect to 43 percent target). At closing – Likely to be achieved: The percentage of individuals diagnosed with DM2 under optimal clinical control in 2020 dropped drastically to 26 percent as a result of the Likely to COVID-19 pandemic (i.e., a 39.5 percent drop), gradually recovering since then to reach 38 be percent in 2022, the date of the last measurement available. While still below both the achieved baseline and 43 percent end-Program target, the rapid increase in the coverage of DM2 (see patients as well as the positive rate of recovery of those under optimal control indicates Figure 3) that the end-target is likely to be achieved or even already have been achieved again. Modeled estimates utilizing the average growth rate observed during post-pandemic Page 15 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT recovery years for diabetes patients under control, indicate that if the growth trend persists, the percentage of patients under optimal control is likely to have re-achieved the Program target in 2023 (see additional evidence below). Optimal clinical control of hypertension • IRI 2 – Percentage of individuals diagnosed with hypertension that are under optimal clinical control (62 percent baseline; 64 percent end-target). At the time of DLI verification – Exceeded: The percentage of individuals ages 20 to 64 diagnosed with hypertension under optimal clinical control increased from 62 percent to 185% 65 percent between 2013 and 2019 (equivalent to 185 percent level of achievement with respect to 64 percent target). At closing – Likely to be achieved: Although the percentage of patients with hypertension under optimal care dropped to 50 percent in 2020 and 44 percent in 2021 due to the impacts of the COVID 19 pandemic, it has gradually recovered since then to reach 56 percent in 2022 (still below both the baseline and the end- target). The swiftness with Likely to which the indicator recovered between 2021 and 2022 (from 44 percent to 56 percent) be sheds light on the resilience of the health system and is promising in terms of its capacity achieved to soon return to pre-pandemic values. Modeled estimates utilizing the average growth (see rate observed during post-pandemic recovery years for hypertension patients under Figure 4) control, indicate that if the growth trend persists, the percentage of patients under optimal control is likely to have re-achieved the Program target in 2024, year when the program closed. INTERMEDIATE RESULTS AND OUTPUTS Implementation of actions aimed at improving the detection, monitoring, and control of NCDs and risk factors • Development and dissemination of new clinical guidelines, including the 2020 Clinical Guidelines for Care of DM2. 33 • Adoption of guidelines for standardizing the detection and care of people with obesity at the PHC level in 2019. Performance is now routinely tracked and reported in the CCSS’ Annual Health Performance Reports, including detection, recording, and referrals to a nutrition specialist.34 Expansion of e-health tools • DLI 5/PDOI 3 – Percentage of total number of PHC units with EDUS (50 percent baseline; 80 percent end-target). At the time of DLI verification – Exceeded: The percentage of total number of PHC units with an operational EDUS increased from 50 percent to 90.1 percent between 2013 and 2021 (equivalent to 134 percent level of achievement with respect to the 80 percent 134% target). The EDUS includes three different information sub-systems: (i) patient identification, scheduling, and appointment (Sistemas de Identificación, Agendas y Citas), the Integrated Family Record System (Sistema Integrado de la Ficha Familiar - SIFF) and the integrated Medical Record System (Sistema Integrado del Expediente de Salud ). At closing – Exceeded: The percentage of total number of PHC units with an operational EDUS increased from 50 percent to 100 percent between 2013 and 2023 (equivalent to 167% 167 percent level of achievement with respect to the 80 percent target). Integration of primary and secondary levels of service 33 CCSS (2020), Guía para la Atención de la Persona con Diabetes Mellitus Tipo 2, Gerencia Médica, Dirección de Desarrollo de Servicios de Salud, Área de Atención Integral a las Personas, June 5, 2020, San José: Costa Rica. [https://repositorio.binasss.sa.cr/repositorio/bitstream/handle/20.500.11764/3487/GuíaDM.pdf?sequence=5&isAllowed=y] 34 CCSS (2019), Informe de resultados de la evaluación de la prestación de servicios de salud 2019 y monitoreo 2020 , Dirección de Compras de Servicios de Salud, CCSS, San José, Costa Rica. [http://hdl.handle.net/20.500.11764/3647] Page 16 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT • DLI 4 – A pilot project on integrated health networks approved by the CCSS Board and implemented for a selected population and territory and evaluated with the results publicly disseminated. • At the time of DLI verification – Achieved: The pilot project on integrated health networks Achieved to be implemented in the Huetar Atlántica region was approved by the CCSS Board and implemented from 2019 to 2021. Results were evaluated through two studies and the findings were publicly disseminated through publications.35 36 43. Additional evidence – An overall improvement in NCD detection, monitoring, and control as well as their risk factors is also observed when looking at evidence beyond the operation’s DLIs and PDOIs, as summarized below: ▪ Colon cancer – The screening campaign supported under the operation has had tangible impact on the early detention of colon cancer. As per PAHO’s recommendation as part of the verification of DLI 2, the CCSS began collecting data on the percentage of patients that underwent colon cancer screening after receiving an invitation. Data showed that 52 percent of those who received an invitation underwent testing, with 7 percent among them testing positive and subsequently undergoing a colonoscopy (61 percent of the latter subset resulted in positive findings - i.e., colon cancer or presence of polyps). ▪ Diabetes Mellitus – While the optimal clinical control for DM2 was negatively affected by the COVID-19 pandemic, the percentage of people with a diagnosis of DM2 who continued to receive care remained stable, largely due to the efforts of primary care teams to maintain contact with patients by phone and through virtual visits, and through household visits once these were reinstated.37 Specifically, the coverage of patients affected by DM2 under primary care increased from 40.1 percent in 2016 to 45 percent in 2019, remaining at that level in 2020 (see Figure 3).38 The positive trend in expanding coverage among those affected with DM2 continued during 2021 and 2022, reaching 49 percent and 51 percent, respectively.39 Multiple factors are likely to have contributed to the drop in the percentage of patients diagnosed with DM2 that were under optimal clinical control despite the expansion in coverage, including: (i) tests could not be performed in virtual and phone visits; (ii) reduced access to medications; and (iii) lifestyle changes during the COVID-19 pandemic, both in terms of restricted mobility as well as food intake and alcohol consumption. Modeled estimates, based on the average annual growth rate between 2020 and 2022, show a promising trajectory towards re-achieving the Program indicator target, with the percentage of those under optimal control expected to continue to increase and surpass once again the DLI 3 target in 2023, this is, before the program closure. Figure 3. DM2 Coverage and Optimal Clinical Control (2016 – 2023) 35 Desai, E., Ross, J., Velázquez, N. R., Bernal, O., Wiken, J., Siam, Z., Schwarz, D., Uribe, M. V. (2022). Análisis de la Capacidad de la Atención Primaria de Salud en la Región Huetar Atlántica de Costa Rica, Ariadne Labs: Boston. 36 Vega Martínez, L.C., Vilar-Compte, M., Gaitan Rossi, P. and Villar Uribe, M. (2022), Reporte integrado del análisis de capacidad del sistema de atención primaria y de la implementación del proyecto demostrativo de Redes Integradas de Prestación de Servicios de Salud en la Región Huetar-Atlántica. Washington, D.C.: World Bank Group. 37 The slight decrease in the percentage of DM2 patients under optimal clinical control in 2018-2019 underscores the challenges of keeping up concomitantly with increases in coverage (i.e., percentage of patients diagnosed with DM2) and maintaining optimal serum glucose concentration in DM2 patients, including patients’ behavior, and which may take some time before results are seen in practice. See, for example, Bin Rakhis, S. A. et al. (2022), Glycemic Control for Type 2 Diabetes Mellitus Patients: A Systematic Review, Cureus Jun 21;14(6). 38 Coverage is estimated based on three variables: (i) the official population pyramid for the year being evaluated provided by the Actuarial and Economic Directorate; (ii) DM2 prevalence among the population aged 20 years and older based on the latest Cardiovascular Risk Factor Surveillance Survey (e.g., 14.8 percent for 2018); and (iii) the number of people aged 20 years and older with a diagnosis of DM2 who had at least one consultation in the year being evaluated and whose diagnosis was registered in EDUS. 39 CCSS (2023), Evaluación de la Prestación de Servicios de Salud de Primer Nivel de Atención 2022 , Gerencia Médica, Dirección Compra de Servicios de Salud, San José: Costa Rica. [http://hdl.handle.net/20.500.11764/4088] Page 17 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Source: Own elaboration based on CCSS’s Reports on the Evaluation of the Provision of Health Services. 2016, 2022 (CCSS Area of Medical Management). ▪ Hypertension – While the optimal clinical control for hypertension was negatively affected by the COVID-19 pandemic, the percentage of people with a diagnosis of hypertension that continued to receive care remained stable, largely due to the efforts of primary care teams to maintain contact with patients by phone and through virtual visits, and through household visits once these were reinstated. Specifically, the coverage of patients aged 20 or more affected by hypertension under primary care increased from 39 percent in 2016 to 42 percent in 2019.40 While this percentage dropped to 40 percent in 2020, coverage among those affected with hypertension rebounded in 2021 and 2022, reaching 44 percent and 48 percent, respectively.41 As in the case of DM2, based on the average annual growth rate between 2020 and 2022, the percentage of hypertense patients under optimal control can be expected to continue its recovery and surpass the IRI 2 target by 2024, the year when the program closed (see Figure 4). 40 The numerator is the number of people with hypertension with at least one consultation. The denominator (the number of people aged 20 or more that are expected to suffer from hypertension) is estimated based on the prevalence estimated for the target population based on the findings of the survey of cardiovascular risk factors carried out by the Epidemiological Surveillance Area of the CCSS in 2018 (CCSS, 2021). 41 CCSS (2023), Evaluación de la Prestación de Servicios de Salud de Primer Nivel de Atención 2022 , Gerencia Médica, Dirección Compra de Servicios de Salud, San José: Costa Rica. [http://hdl.handle.net/20.500.11764/4088] Page 18 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Figure 4. Coverage and Optimal Clinical Control for People with Hypertension Aged 20 to 64 (2016 – 2024) Source: Own elaboration based on CCSS’s Reports on the Evaluation of the Provision of Health Services 2016, 2022 (CCSS Area of Medical Management). 44. PDO 3 - Enhancing the institutional efficiency of the CCSS. Rating: High 45. Definition - The operation provided financial incentives to improve the CCSS’ institutional efficiency, focusing on actions (i.e., Intermediate Results) aimed at providing the CCSS the necessary tools for ensuring its financial sustainability and enhance its institutional management capacity. In addition to financial incentives, the operation also contributed to the development and implementation of several tools aimed at enhancing CCSS’ institutional management capacity by including critical results in its RF (IRIs 3 thru 7), thus providing them with added visibility and the opportunity of collaboration between the technical units responsible for their implementation and the Bank Team. Specifically: ▪ Financial sustainability - The operation provided financial incentives for the development and implementation of a comprehensive medium- and long-term plan to ensure the financial sustainability of the CCSS (DLI 7/PDOI 4). It also contributed to the implementation of new international accounting standards and streamlined, automated processes (IRI 7). ▪ Institutional management - The operation provided financial incentives for the redesign, implementation, and use of data collected by the new annual survey to measure patients’ satisfaction and, subsequently, the impact of interventions aimed at its improvement (DLI 6). The operation also provided support to the implementation of several management tools and institutional improvements (IRIs 3-6). 46. Results - The operation’s contribution toward improving the CCSS’ institutional efficiency is deemed High. All end- targets were fully achieved or exceeded (i.e., DLI 7/PDI 4; DLI 6; IRIs 3-6), thus, providing the CCSS with robust tools to enhance the CCSS’s financial sustainability and its institutional management capacity. Specifically: ▪ Financial sustainability – All the milestones envisioned under DLI 7 were achieved and externally verified, Page 19 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT effectively improving the CCSS’s institutional capacity for financial management and budgeting. Specifically, all actions supported under DLI 7.1 were achieved and externally verified in February 2021, including: (i) the approval of the 2021-2030 Investment Portfolio; (ii) the adoption of improved prioritization criteria; (iii) financial projections to ensure the sustainability of the institution; (iv) compliance with environmental norms; and (v) the incorporation of recommendations from the Evaluation of Environmental and Social Systems (ESSA). Likewise, all actions supported under DLI 7.2 were achieved and externally verified by the PAHO in March 2020, including: (i) the approval of the “Actuarial Valuation of Health Insurance, 2017” by the CCSS Board of Directors on October 31, 2019; (ii) its public dissemination on the CCSS website; and (iii) the verification of the assumptions that were utilized and sensitivity analysis. Finally, the CCSS’s approved annual budget for 2024 was formulated using prospective budgeting tools (PDOI 4), which constitutes an important reform of the resource allocation for both PHC units and hospitals (see Section II.D for a detailed description). ▪ Institutional management – With the operation’s support, the CCSS’ institutional management capacity has been enhanced as the result of the implementation of important management tools, including: (i) an annual patient satisfaction survey (DLI6); (ii) the Integrated Family Record System’s (SIFF) expanded coverage (IRIs 3-4); (iii) the use of a newly developed institutional strategic intelligence system data to measure progress of SASHI’s implementation (IRI 5); and (iv) the institutional review of core institutional management functions, and operational plan for progressive implementation of streamlining central level management (IRI 6). (See Section II.D for a detailed description). Rating of Overall Efficacy Rating: High 47. The operation’s efficacy is deemed High to reflect the high level of achievement of PDO 1, 2 and 3. All the Program’s 15 milestones under the seven DLIs were fully achieved and externally verified. Therefore, the PforR was fully disbursed (i.e., US$420 million, equivalent to 100 percent disbursements). Although indicators focusing on optimal control of NCDs deteriorated significantly during the COVID-19 pandemic (i.e., DLI 3, IRI 2), their rapid improvement in the aftermath of the pandemic indicate the institutionalization of the improvements in the delivery of primary health services supported under this PforR, with modeled estimations for the indicators that dropped below target values showing a promising trajectory towards re-achievement of the Program targets by 2024, the year the PforR was closed. C. JUSTIFICATION OF OVERALL OUTCOME RATING Rating: Highly Satisfactory 48. The operation’s overall performance is deemed Highly Satisfactory to reflect its high relevance and high efficacy. It is worth noting that before the COVID-19 pandemic, all indicators were on track of being achieved or exceeded. The extension of the implementation period (21-month total extension) to deal with the impacts of the COVID-19 pandemic and the cyber-attack to CCSS’ information systems, showed the flexibility of the Bank’s operation in adjusting to external, unpredictable shocks. Other than this extension, the PforR was implemented as originally designed, with all DLI milestones being achieved without requiring any changes or waivers, pointing to the remarkable success of this PforR operation. D. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 49. In the context of the program aiming to strengthen patient treatment in a more inclusive manner, the Page 20 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Environmental and Social safeguards team participated in discussions about institutional changes throughout program implementation. 50. Over the past years, CCSS has worked at improving its gender inclusive strategy across multiple fronts. Although there are different institutional instances that provide gender-focused services or attention, CCSS consolidated these in early 2024 with a new institutional policy to promote gender equality in its services. The policy aims to ensure equitable treatment for men and women in healthcare, pensions, and other administrative services, including sensibilization regarding sexual orientation and gender identity. The policy aligns with national and international human rights and gender equality standards. It includes fifteen specific actions designed to eliminate gender discrimination and promote gender-sensitive health care to address the unique social determinants affecting women's health. 51. Additionally, CCSS launched a virtual tool to educate employees, students, and subcontractors about sexual harassment and workplace bullying, fostering respectful work environments. This training will be part of the induction for new staff and mandatory every two years for existing employees, aimed at strengthening institutional support for affected populations to prevent impunity and create positive work atmosphere. The policy also highlights the need for better hospital attire for women and comprehensive sexual and reproductive health services, ensuring safe and dignified maternity care. Institutional Strengthening 52. The operation supported the GoCR in its efforts to introduce important institutional innovations to improve both the timely access and quality of PHC services as well as the CCSS’s institutional efficiency. The operation’s support was through financial incentives (DLIs 4, 5, 6) or by including specific results under the RF (IRIs 3 thru 7), thus providing them with added visibility and the opportunity for collaboration between the technical units responsible for their implementation and the Bank Team. These institutional innovations can be summarized as follows: 53. Institutional innovations aimed at improving timely access and quality of PHC services: ▪ Expansion of e-Health tools – Costa Rica’s electronic health records system is one of the most comprehensive ones in Latin America. Conceived as more than an electronic record, EDUS integrates several types of patient- centered data that can be accessed by authorized users at different levels of care, including : (i) clinical (i.e., data derived from consultations and hospitalizations); (ii) administrative (i.e., appointments, referrals, and the delivery of support services such as radiology, pathology, laboratory, and pharmacy); as well as (iii) social (i.e., household demographic and economic data). The operation’s specific contribution has been on the expansion of EDUS coverage at the primary level, with the percentage of health care centers with a functional EDUS increasing from 50 percent to 100 percent during the operation’s lifetime, including those in remote areas without electricity or internet connectivity (PDOI 3/DLI 5). EDUS has become a vital tool for the provision of health services, contributing to their timeliness and quality by facilitating the integration of health care services across primary, secondary, and tertiary levels of care. It also facilitates access by offering patients web and telephone appointment requests as well as electronic access to their personal records through the cell phone application, has more than 5.6 million downloads, being the most popular application in the "Health and Wellness" category in the country. In 2019, EDUS was awarded the United Nations Public Service Award, recognized by the CCSS for its ability to digitally transform and better manage health.42 42For an in-depth description of EDUS’s implementation experience, including the strategies adopted to address the various implementation challenges, see Rosado Valenzuela, A.A., Sheffle, A. Lara Salinas, A. M., Mussini, M. and Di Giorgio, L. (2023), Expansion of the Coverage of the Single Digital Record (EDUS) in the Primary Page 21 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT ▪ Integrated Health Networks: The operation supported the design and implementation of a pilot project in the Huetar Atlántica Region as a first step in the roll-out of the Integrated Health Services Network model (DLI 4) at the national level. This model is based on the PHC strategy, which seeks to strengthen the first level of care and integrate it with specialized care when required. The pilot project, which was implemented during the period 2019-2021, has allowed the gradual adaptation of health services to the needs of the population of the Huetar Atlántica region, while developing a governance, implementation and optimization strategy that serves as a basis for future expansion efforts. This pilot provided important insights into important aspects that need to be taken into consideration for the successful roll-out of the model to the rest of the country, including strengthening of governance mechanisms at the network level, the need to develop guidelines to further enhance patient and community engagement and foster self-care, and include instruments to measure the quality of services and patient satisfaction. Results and the implementation process have been documented in two papers.43 54. Institutional innovations aimed at improving the CCSS’ institutional efficiency: ▪ Resource allocation reform: The operation provided financial incentives to reform the methodology used to allocate resources among PHC units and hospitals, moving away from budgeting practices focusing on historical values to a system based on prospective elements that introduces strategic purchasing mechanisms, aimed at enhancing expenditure efficiency, impact and quality of care, in the long term. With additional technical support 44 provided by the Bank, two models were developed taking into consideration the literature on state-of-art resource allocation practices in the health sector and country-specific conditions. First, a capitation model was found to be the most suitable for the allocation of resources among PHC units based on: (i) a base rate (i.e., based on the number of people registered in each health area) and (ii) a risk index (i.e., based on demographic factors, such as population under 18 and over 65, and the demand for health services). Second, a top-up budget model was developed for the allocation of resources among hospitals to incentivize: (i) productivity and accessibility (i.e., bed occupancy rate, bed turnover rate, and share of first time outpatient visits); (ii) quality of care (i.e., hospital-acquired infections); and (iii) budget performance (compliance with annual budget policy, execution of additional resources allocated for investment and maintenance activities).45 These two models were piloted for the preparation of the 2024 budget for eight of the 105 health areas and two of the 29 hospitals under CCSS (PDOI 4/DLI 7). ▪ Integrated Family Record System (SIFF): The operation supported the digitization of Family Socio-Economic and Health Risks Information File (Ficha Familiar) to be included under the SIFF. Now integrated under EDUS, the Family Health Record dates back from the early 1990s and contains data collected by Technical Primary Health Care Assistants (Asistentes Técnicos de Atención Primaria – ATAPs) on the socio-economic factors affecting households’ health (e.g., housing characteristics, location, and basic services). During the Program’s lifetime, the percentage of Health Care System in Costa Rica, Knowledge Brief, Health, Nutrition and Population Global Practice, February 2023, Washington, D.C.: World Bank Group. [https://openknowledge.worldbank.org/handle/10986/40524] 43 Desai, E., Ross, J., Velázquez, N. R., Bernal, O., Wiken, J., Siam, Z., Schwarz, D., Uribe, M. V. (2022). Análisis de la Capacidad de la Atención Primaria de Salud en la Región Huetar Atlántica de Costa Rica, Ariadne Labs: Boston. Vega Martínez, L.C., Vilar-Compte, M., Gaitan Rossi, P. and Villar Uribe, M. (2022), Reporte integrado del análisis de capacidad del sistema de atención primaria y de la implementación del proyecto demostrativo de Redes Integradas de Prestación de Servicios de Salud en la Región Huetar-Atlántica. Washington, D.C.: World Bank Group. Mireya Vilar-Compte; Gaitan Rossi,Pablo; Natalia Rovelo Velasquez; Bernal Acevedo,Oscar Alberto; Villar Uribe,Manuela. Fidelidad y Sostenibilidad de la Implementación del Proyecto Demostrativo del Modelo en Red de Servicios de Salud en la Región Huetar-Atlántica, Costa Rica (Spanish). Washington, D.C. : World Bank Group. Vilar-Compte, Mireya; Gaitán-Ross, Pablo; Rovelo Velazquez,Natalia; Bernal Acevedo,Oscar Alberto; Villar Uribe,Manuela. Evaluación de Sendas Causales de la Implementación del Proyecto Demostrativo del Modelo en Red de Servicios de Salud en la Región Huetar-Atlántica, Costa Rica (Spanish). Washington, D.C. : World Bank Group. 44 Technical support was provided by the Bank Task Team and external consultants funded through trust funds. 45 Rosado Valenzuela, A.L.; Sheffel, A.; Mussini, M.; Lara Salinas, A.M. and Di Giorgio, L. (2023) El desarrollo del presupuesto anual con modelos de pago prospectivos y por capitación en Costa Rica, Knowledge Brief, Health, Nutrition and Population Global Practice, Upcoming, World Bank, Washington DC. Page 22 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT households included in the SIFF increased from 70 percent to 100 percent (IRI 3). In addition, the operation also supported the georeferentiation of housing units, with the percentage of housing units with georeferenced information available online increasing from 0 percent to 66.9 percent (IRI 4). The georeferenced information of housing units, linked to the corresponding household socio-economic data and health records, constitutes the foundation to generate information on the concentration of epidemiological profiles and risk maps at the community level and will facilitate the management of future epidemics and natural disasters, as well as better target preventive interventions.46 ▪ Satisfaction surveys - The operation provided financial incentives for the design and implementation of CCSS’s satisfaction surveys aimed at giving greater visibility and impact to users’ perceptions, including those of indigenous populations. These surveys have been successfully carried out for five consecutive years, including during the COVID- 19 pandemic (DLI 6). Results of these surveys for individual health units are internally disseminated within the CCSS, fostering accountability and transparency. Best performers are showcased through the media and social networks, ¬fostering healthy competition. Results from satisfaction surveys also serve as critical inputs in the preparation of improvement plans by health units, thus including a direct feedback loop mechanism. Poverty Reduction and Shared Prosperity 55. By increasing access and timeliness of the PHC services provided by the CCSS, the operation can be expected to improve the health status of lower-income groups of the population. It is estimated that the use of health care services by the bottom 40 percent is high, with 87 percent of individuals in the first quintile (poorest) and 85 percent of those in the second quintile reporting use of outpatient health services through the CCSS.47 Another notable contribution was the expansion of EDUS among health care centers in remote locations and without internet coverage, thus increasing accessibility among rural and indigenous populations. In addition, other government institutions have signed confidentiality and information security agreements with the CCSS to access data from SIFF, including the Single System of State Beneficiaries (Sistema Nacional de Información y Registro Único de Beneficiarios - SINIRUBE) and the Mixed Institute of Social Assistance (Instituto Mixto de Ayuda Social – IMAS), which utilize the data as an input to allocate social protection resources, further supporting efforts to reduce poverty in the country. Other Unintended Outcomes and Impacts 56. Knowledge generation – The operation contributed to substantial knowledge generation benefiting individual interventions and the global community of development practitioners. On the one hand, as part of the process of verifying the achievement of DLI targets, the PAHO’s provided valuable technical advice and recommendations, going beyond its role as external verification agency. For example, as per the PAHO suggestion after the verification of DLI 2.1, data was also collected to measure the percent of those patients that underwent the screening after receiving an invitation. On the other hand, the Bank’s Task Team was extremely diligent in sharing lessons learned from the implementation of individual interventions as well as the PforR as an instrument through Knowledge Briefs, online publications from the Bank’s Health, Nutrition and Population Global Practice, of which five have already been published and one more is under review. Webinars were also organized to disseminate results among interested clients and stakeholders and other World Bank professionals. These dissemination events also contributed to sharing the strengths of the Costa Rican’s health sector with 46 Rosado Valenzuela, A. L.; Sheffel, A.; Mussini, M.; Lara Salinas, A.M.; Di Giorgio, L. (2023), The Integrated Family Record System (SIFF), a Key Tool for Monitoring the Social Determinants of Health in Costa Rica, Knowledge Brief, Health, Nutrition and Population Global Practice, August 2023, Washington, D.C.: World Bank Group. [https://openknowledge.worldbank.org/handle/10986/40525]. 47 World Bank (2015), “Costa Rica: Estudio de Gasto Público Social y sus Instituciones,” in Central America: Social Expenditures and Institutional Review, October 2015, World Bank Group. Washington, DC. [http://documents.worldbank.org/curated/en/998741468194981719/Estudio-de-gasto-público-social-y-sus-instituciones-Costa- Rica] Page 23 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT other client countries. 48 57. Strengthening of the CCSS’s environmental safeguards - The initial diagnostic study of the CCSS’s institutional capacity for implementing the Bank’s environmental guidelines set in motion a process of improving the CCSS’ environmental management capacity. Some of the most notable results include: (i) enhanced environmental management capacity, including the adoption of an institutional environmental policy, which has been embedded as a transversal axis in the CCSS’s Institutional Strategic Plan (Plan Estratégico Institucional); (ii) development of the Environmental Sustainability Information System (Sistema de Información de Sostenibilidad Ambiental - SISA) and training of CCSS staff; (iii) the inclusion of environmental investment projects within the CCSS’ investment portfolio; and (iv) a detailed environmental diagnosis of the CCSS’ institutional infrastructure is now underway. III. KEY FACTORS AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 58. Appropriateness of lending instrument – Costa Rica was the first country in Central America to utilize the PforR instrument, and the fourth one in the LAC Region. The PforR instrument lent itself well to support the implementation of the CCSS’ SASHI, as it provided financial incentives for the achievement of tangible program results while relying on the country’s own institutions, helping further strengthen the CCSS’ institutional capacity within the country and allowing a focus on results. 59. Political and institutional will – The operation capitalized on the ownership and institutional support that already existed for the SASHI Program, a homegrown initiative in response of an evaluation conducted by the CCSS between 2014 and 2016 that identified the priority areas that needed to be addressed (i.e., the need to adapt health care delivery to country’s changing epidemiological profile and enhance the CCSS’ institutional capacity to manage an increasingly complex system). These objectives, as well as the SASHI Program as a whole, benefited from the full support and buy-in of the CCSS Board of Directors. 60. Legal frameworks already in place – In addition to strong political support, some of the key interventions benefited from having the needed legal framework already in place. As an illustration, EDUS’s expansion was supported by a solid legal framework that included Law 9162, Regulation 8954 on EDUS, Law 8968 on personal data protection, and the publication of decrees by the CCSS Board of Directors. 61. DLI scalability – DLIs were well defined and agreed jointly with the CCSS. They were also scalable, providing incentives to stimulate and reward gradual, continuous improvements and to sustain motivation and commitment toward full PDO achievement. This scalability was introduced in two ways: (i) disbursements in four DLIs were proportional to the corresponding quantitative results, rewarding partial achievements; and (ii) within each area of intervention supported by a DLI, expected results were grouped into two or three milestones, thus introducing more flexibility rather than an all- or-nothing approach. 48 Find the various references above under Section II.D – Institutional Strengthening. See also Mussini, M., Lara, A.M., Rosado Valenzuela, A.L., Sheffel, A. and Di Giorgio, L. (2023), Lessons Learned through the Health Program for Results in Costa Rica, Knowledge Brief, Health, Nutrition and Population Global Practice, August 2023, Washington, D.C.: World Bank Group. [https://openknowledge.worldbank.org/handle/10986/40523] Page 24 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT B. KEY FACTORS DURING IMPLEMENTATION Factors Outside the Government’s Control 62. COVID-19 pandemic – In countries around the world, including Costa Rica, the COVID-19 pandemic resulted in interruptions in the provision of PHC services, disruptions to regular hospital level services, a sharp decline in the demand for services due to fear of contagion and mobility restrictions, and changes in lifestyle and psychological wellbeing. As of the end of 2021, 93 percent of countries in the LAC region reported interruptions in provision of essential health services for all modalities, with 26 percent of them reporting interruptions of 75 to 100 percent of the services, and with average interruptions in 55 percent of the 66 services analyzed49. In Costa Rica, the first COVID-19 case was detected on March 6, 2020. At the system level, a large share of the institutional resources was reassigned to address the pandemic by developing preventive actions and detecting and providing care to people with COVID-19-related symptoms. To preserve progress made under the PforR and more broadly to maintain the provision of essential services, the CSSS made various efforts. For instance, the CCSS implemented a strategy of expansion of mild, moderate, severe, and critical hospital beds, which included the referral of non-COVID-19 patients to private hospitals, it also implemented a reorganization of the care process from the first level to the most complex, to safeguard patients and staff against COVID-19. At the primary level, health services were reorganized to be provided through alternative means, including domiciliary visits and phone and virtual consultations. The government also made early efforts to secure access to COVID-19 vaccines, including through external financing. Finally, the CCSS, with support from the Ministry of Health, deployed a highly successful vaccination campaign, which limited the occurrence of severe COVID-19 cases, thus averting the diversion of resources and health care towards the management of COVID-19 cases. Despite these efforts, the percentage of the total population that received health care services at the primary level fell from 49 percent in 2019 to 43 percent in 2020, equivalent to a 5- percentage point drop. The negative trend reversed in subsequent years, with the percentage of the population that received care reaching 46 percent in 2021 and 48 percent in 2022, when it returned to pre-pandemic levels. This trend was also reflected in several of the operation’s DLIs and PDOIs, which were negatively affected by the COVID-19 pandemic, particularly those that required a “catch-up” period, such as those focusing on optimal control of NCDs. 63. The operational response of the fight against COVID-19 fell on the CCSS, the resulting diversion of significant material and human resources caused delays in the fulfilment of some of the indicators. This delay was addressed with the prompt request from the client for an extension of the implementation period. 64. 2022 Cyber-attack - During 2022, nearly 30 institutions of the GoCR, including the CCSS, were the target of a series of cyber-attacks, prompting the shutdown of various information systems. In the case of the CCSS, a cyberattack that took place on May 31 resulted in the shutdown of the EDUS systems from June to mid-August, and in some areas until mid- September. During this period, the provision of health care services had to be recorded manually in paper-based medical records. While these records were to be later uploaded by health personnel once the EDUS systems were restored, the effort was quite challenging due to the large amount of data, some of which were lost. Therefore, it is likely that the provision of health care services during 2022 is undercounted. Factors Inside the Government’s Control 65. Uneven pace of reforms – The implementation experience of the PforR operation illustrates that the pace of reform can vary significantly, with some DLI targets being fully achieved as early as 2018, such as the shift toward ambulatory surgeries (DLI 1). Conversely, other actions, such as the utilization of prospective budgeting tools for budget formulation, require deep system changes, and was therefore not achieved until 2023 (DLI 7). Although it may seem 49 PAHO: https://hia.paho.org/en/covid-2022/health Page 25 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT obvious, an operation’s lifetime is determined by the length of the more time-consuming reforms. The Program’s original lifetime of roughly six years (i.e., from March 2016 to April 2022) realistically estimated the time needed to implement the wide set of complex reforms supported under this PforR. 66. Amelioration measures – The CCSS adopted various measures to avoid disruptions in treatment and mitigate the deterioration of clinical control of NCDs, hence reducing the negative impact on the program indicators. Examples of these measures included the introduction of telemedicine, resuming home visits as soon as possible, and home delivery of medicines. These measures aimed to avoid disruptions in treatment and mitigate the deterioration of clinical control of NCDs.50 67. Value added of the PforR – Linking critical results to disbursements under this PforR operation and having a system of continuous monitoring and reporting during implementation supervision missions, provided effective incentives for the various technical units to focus its efforts and actions to meet these results within the established timeframe. At the same time, the joint follow-up and active communication between the CCSS and the Bank’s Task Team and experts contributed greatly toward addressing implementation barriers to achieve the DLIs targets in a timely manner. As reported by some CCSS managers, these institutional breakthroughs would not have been possible without the added visibility and technical support resulting from this PforR operation. 68. Value added of the additional Technical Assistance - While the PforR provides financial and non-financial incentives to achieve the agreed results, some of the reforms planned under this Program were complex to design and implement. While not highly visible, a substantial amount of TA went hand-in-hand with the financial support provided under this PforR operation, in particular for the design of reforms linked to the implementation of prospective budgeting and alternative resource allocation models. This technical assistance was critical to overcome technical challenges encountered during implementation to ultimately ensure PDO achievement. It is important to recognize the effort made by Bank Task Teams in identifying and securing the necessary resources to support the needed technical assistance through trust funds, limited Bank supervision budget and other sources of funding outside supervision budgets. While the Task Team managed to raise the funds needed to provide the technical assistance, it is advisable to assess the need for TA that may arise from PforR implementation at the PforR design phase and identify potential financing sources from the outset. Even clients with high capacity, such as the CCSS in Costa Rica, highly value and request the high-quality TA and the ability to share global experiences, that has been traditionally associated with Bank financing engagements. TA is particularly crucial for the achievement of “last mile” efforts required to achieve results. 69. Value added of the PforR Unit - Although not a requirement under a PforR Program, the CCSS made the decision to set-up a unit responsible for coordinating Monitoring and Evaluation (M&E) and reporting among the different implementing units and facilitate communication with the Bank. This unit, which was constituted by highly capable and well respected CCSS professionals, proved invaluable in ensuring the seamless implementation and reporting of the various lines of action under this PforR. 50 Specific examples include: (i) [https://www.scielo.sa.cr/scielo.php?script=sci_arttext&pid=S1409- 41422020000300023#:~:text=De%20una%20manera%20eficaz%20y,medicamentos%20y%20que%20éstas%20sean]; (ii) [https://semanariouniversidad.com/pais/la-ccss-habilitara-entrega-a-domicilio-de-medicamentos/]; and (iii)https://www.paho.org/es/historias/asegurando-continuidad-tratamiento-pacientes-cronicos-tiempos-pandemia-telesalud-costa] Page 26 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 70. The operation’s RF included four PDIs and a robust set of IRIs that adequately captured outcomes for PDOs 1 and 2 and the achievement of critical Intermediate Results in the case of PDO 3. The RF as well as the seven DLIs were jointly agreed with the CCSS. All DLIs had been clearly defined and responsibilities for data collection agreed upon. DLI achievement was to be verified externally. The DLI verification protocols included clear operational definitions, baseline and target values, and detailed procedures for their measurement through technical documents developed in agreement between the CCSS and the verification agency. In line with the spirit of the PforR instrument, the country's existing M&E systems were used for the measurement and verification of progress toward PDO achievement. Since the CCSS had already demonstrated its ability to successfully track results of SASHI, which was already under implementation, no major issue was expected regarding the country’s M&E capacity. Nevertheless, the timeliness and quality of the reported data was very satisfactory throughout the operation. M&E Implementation 71. Indicators under the RF were closely monitored by the Program Implementing Unit with data provided by the individual units responsible for the implementation of the various lines of action as well as the CCSS’s information systems, such as EDUS. However, one limitation was identified: once an end-target indicator was reached, subsequent updates were not incorporated into the Implementation Status and Results Report (ISR). Therefore, the ISR did not capture the deterioration in those PDOs and IRIs that were negatively affected by the COVID-19 pandemic. Nevertheless, it's essential to highlight that the CCSS and Task Team diligently monitored all indicators throughout supervision missions, persisting in their oversight even after targets were met, including during the pandemic. It should also be noted that EDUS shutdown in 2022 to protect the data from the cyberattack is likely to have resulted in the underestimation of some of the data reported for 2022 (DLI 3, IRI 4). 72. The PAHO was hired by the CCSS to independently verify the achievement of DLIs. DLI verification was done using rigorous methodologies consisting of the ex-post reviews of existing administrative data based on sound sampling methodologies. Verification reports gave detailed descriptions of the reviews’ methodological aspects of the reviews and DLI achievement. They also included recommendations that helped improve current CCSS practices (e.g., suggestion for further data collection on colon cancer screening; recommendations for the Investment Portfolio 2021-2030). M&E Utilization 73. Externally verified DLIs supported disbursement in a sound and transparent manner. The RF monitored implementation progress and PDO achievement. Though not reflected in further ISRs once an indicator had been achieved, the CCSSS continued the diligent monitoring of all indicators, with reporting to the World Bank done twice a year and results presented during missions and used it to inform interventions on the ground for continuous improvement of the areas affected by the PforR. For instance, the second restructuring of the program resulted from the fact that indicators were lagging due to the EDUS shutdown in 2022 and to the challenges posed by COVID-19. Regular monitoring facilitated the timely detection of these shortcomings and adjustments to implementation even for indicators not requiring a restructuring but “softer” pushes in implementation. Page 27 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT 74. There were significant positive externalities resulting from the strengthening of the CCSS’ information systems under this operation. Specifically, the expansion in EDUS’ coverage resulted in the improved quality and timeliness of online performance evaluations conducted regularly by the CCSS as part of the preparation of the annual evaluation of the provision of PHC services. Likewise, as mentioned earlier, other government agencies are also utilizing SIFF data. Justification of Overall Rating of Quality of M&E Rating: Substantial 75. The operation’s M&E quality is deemed Substantial, as it effectively served to monitor implementation progress and PDO achievement, as well as to operationalize disbursements in a transparent and effective manner. While it would have been desirable to continue reporting during the entire implementation period (i.e., rather than until the end-target was met), this shortcoming is being addressed in this ICR. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social 76. The ESSA that was carried out at Appraisal concluded that the PforR did not include activities that could have significant or unprecedented adverse impacts on the environment and/or affected people. Moreover, Costa Rica’s environmental, health and safety laws as well as the institutional capacity of its environmental regulatory authority was adequate, with mechanisms already in place within the CCSS and the environmental regulatory authority to assess the potential environmental impact and establish appropriate mitigation measures related to new construction and rehabilitation of facilities. The PforR also provided the opportunity to support more efficient and effective environmental, health, and safety management with the CCSS, including enhanced occupational safety procedures and the preparation of a technical proposal for the creation of an Environmental Directorate. 77. The Program's social system was also deemed adequate and without risk of substantial negative societal impacts; thus, the overall risk profile was rated as Low. Participation and citizen engagement processes were well- established within the CCSS and continued to improve during the PforR implementation. During the COVID-19 pandemic, measures were adopted to ensure inclusion for Indigenous Peoples (IP) and lesbian, gay, bisexual, transgender, queer, and intersex (LGBTQI) persons. The inclusion of an IP variable in data collection helped monitor pandemic-specific trends, which showed a higher contagion risk in IP communities but also effective vaccination rates. Similarly, additional local IP health staff has been hired that play the dual role of medical support and community engagement. For LGBTQI-oriented actions, CCSS carried out training plans and capacity building sessions with staff to raise awareness and build knowledge on LGBTQI issues and challenges in health delivery. Fiduciary 78. The fiduciary and governance frameworks were considered adequate to support the implementation of the PforR, with the fiduciary risk rated as Moderate during both the appraisal and implementation stages. Existing financial management and procurement systems were in place to facilitate budget planning and execution, procurement processes, transaction recording, and the generation of financial reports. Additionally, the CCSS is subject to oversight from Costa Rica’s Comptroller General's Office, i.e., the nation's Supreme Audit Institution, the Legislative Assembly, and the internal control framework applicable to the public sector. However, for the purposes of PforR, the services of the private firm Deloitte were engaged. The control environment was deemed sufficient, characterized by clearly established rules, Page 28 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT procedures, and assigned roles and responsibilities. Despite encountering some issues during implementation, particularly regarding the timeliness and efficiency of audit deliverables, the audit for the year 2022 was submitted, on June 14, 2024, with almost one year's delay due to additional work requested by the CCSS Presidency. Furthermore, the pending report for the period January 1, 2023, to January 31, 2024, is expected to be submitted by October 14, 2024. Nevertheless, these challenges did not impede the timely and reliable provision of information necessary for managing and monitoring Program implementation, including budget execution in accordance with approved budgets and financial statements. C. BANK PERFORMANCE Quality at Entry Rating: Highly Satisfactory 79. The operation capitalized on the CCSS’s strong ownership and institutional support for the SASHI program. The choice of lending instrument was adequate, as the PforR offered a good match to support the results-oriented SASHI Program using the robust CCSS’s institutional capacity. The technical design and overall readiness were robust, both for the operation and as a whole and for the various interventions within it. DLIs were highly relevant and supported with sound operational definition and verification procedures. They also provided incentives for gradual progress toward the end-results. While the reforms supported under SASHI were of a wide scope and ambitious, the risk evaluation proved to be realistic. Quality of Supervision Rating: Highly Satisfactory 80. The quality of supervision is deemed highly satisfactory. Supervision missions were carried out regularly, ensuring a fluid dialogue with the individual units responsible for implementing the various lines of intervention. Findings were recorded in detailed Aide Memoires and ISRs. The operation benefited from a strong and relatively stable Task Team, with seamless handovers from the original Task Team Leader to their successors. The Bank Team provided ongoing technical support and training on FM and social and environmental safeguards. The Bank Team also provided ongoing technical assistance on substantive areas, providing access to experts from the Bank and other countries to support the design and implementation of complex strategic reforms, such as the reform of budget allocation mechanisms. To this effect, the Bank Team was very active in securing additional sources of finance amounting to US$160,000 in Trust Funds (excluding staff time) to support various program-related activities, which would otherwise not have been possible under the PforR instrument. The ad-hoc technical assistance provided by the Bank, in combination with the financial incentives provided by the PforR, was critical in helping overcome technical challenges encountered in the path of reforms to achieve the program's objectives. In addition, the Team made a remarkable contribution to knowledge building, disseminating lessons learned from the implementation experience of various lines of intervention through a series of articles published under the “Knowledge Brief” series. Justification of Overall Rating of Bank Performance Rating: Highly Satisfactory 81. Overall Bank performance is deemed highly satisfactory to reflect its highly satisfactory in ensuring quality at entry and during supervision. Page 29 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT D. RISK TO DEVELOPMENT OUTCOME Rating: Negligible 82. The risk to the sustainability of the operation’s outcome is considered Negligible. Several factors contribute to the sustainability of the outcomes achieved under this operation, including strong ownership and continuous institutional support for the reforms under SASHI and the CCSS’s demonstrated implementation capacity. In addition, th e most significant reforms are supported by a legal framework and have been approved by the CCSS and, when needed, also by the Republic General Comptroller. The PforR also contributed to the CCSS' efforts to have a greater focus on results and accountability. This is reflected in the incorporation of even more robust and ambitious indicators into the CCSS institutional plans (i.e., 2023-2027 Management Tactical Plan, and the 2021-2025 Institutional Budget Plan). V. LESSONS AND RECOMMENDATIONS 83. PforR effectiveness – This operation underscores the effectiveness of the PforR instrument in supporting homegrown reform programs that have strong political and institutional support in countries with robust systems and processes. The key advantages are enhanced focus on results, further strengthening of country systems and processes, and lower risks to development outcomes. 84. DLI scalability – In contrast with an all-or-nothing approach, making DLIs scalable helps stimulate and reward gradual progress toward PDO achievement. It also introduces flexibility, recognizing that the pace of reforms can be uneven. In this operation, scalability was introduced in two ways: (i) in the case of DLIs supporting continuous quantitative results, making disbursements proportional to the achievements; and (ii) in the case of sets of specific milestones, grouping them into subsets. In this way, DLIs can measure incremental achievements, thus introducing more flexibility. 85. Realistic timeframes for reforms – By establishing an original lifetime of roughly six years (i.e., from March 2016 to April 2022), this PforR realistically estimated the time needed to implement a wide set of complex reforms. While the operation had to eventually be extended by an additional 20 months due to delays resulting mainly from the COVID-19 pandemic, the pace of individual reforms varied greatly, because of the technical and operational challenges they faced. 86. Continuous monitoring of M&E indicators despite achievement – Despite DLI indicators having been achieved, this PforR highlights the value of continuing to measure the results and of the institutionalization of the M&E function. While indicators were not reported in the ISRs, they continued to be discussed in the progress reports and during the missions. This continuous monitoring allowed to raise alerts that the indicators were starting to be off track and allowed for the CCSS to take the necessary countermeasures. 87. Demand for TA under PforRs – An important lesson derived from this PforR is the need for TA to support specific reforms, particularly those requiring specific know-how. In the case of the resource allocation reform, while the political will was there and technical capacity is high, the CCSS’ technical know-how was not sufficient. Some relevant takeaways can be derived from this particular experience: (i) there is an expectation on the part of the Borrower that they can count on TA on the part of the Bank, with the Bank’s ability to provide solid TA being one of its competitive advantages; (ii) when the need for TA arises in the context of PforR operations, the burden to obtain the necessary funding falls on the Bank Task Team; and (iii) the process of obtaining such financial support is not only time consuming, but also uncertain in terms of its eventual success. If the Team had not been successful in obtaining the relatively small ad-hoc financial support required for this TA, the quality of the solutions being adopted could have been compromised. As previously mentioned, Page 30 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT it is advisable to assess the need for TA during the PforR design phase and identify potential financing sources from the outset. 88. Value added of a PforR coordinating unit – Although not a requirement under a PforR Program, having a PforR coordinating unit contributed greatly to the program’s seamless implementation and reporting of the various lines of action under this PforR. 89. Value added of a reputable external verifier – The PAHO’s contribution went beyond its role as external verifier of DLI achievement, providing also sound technical suggestions and recommendations to further enhance ongoing reforms. 90. Knowledge generation – Ambitious reform programs such as the one supported under this PforR offer great potential for knowledge generation, not only at the program level but also within individual lines of action. In the case of this operation, the Task Team has effectively capitalized on this potential, disseminating important lessons learned and case studies among the wider community of development practitioners. Page 31 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT ANNEX 1. RESULTS FRAMEWORK, DISBURSEMENT LINKED INDICATORS, AND PROGRAM ACTION PLAN @#&OPS~Doctype~OPS^dynamics@icrpfrresultframework#doctemplate A. RESULTS FRAMEWORK PDO Indicators by PDO Outcomes Improving Timeliness and Quality of Health Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year Percentage of major surgeries from 18.00 Feb/2016 40.00 Mar/2021 46.00 Apr/2022 49.80 Dec/2020 priority list conducted in outpatient Comments on achieving targets Indicator achieved its end of Project target. settings according to CCSS institutional guidelines (Percentage) Cumulative percentage of target 0.00 Feb/2016 40.00 Mar/2020 40.00 Apr/2022 40.10 Dec/2021 population personally invited to Comments on achieving targets Indicator achieved and verified. undergo colon cancer screening in five priority counties. (Percentage) Improving the institutional efficiency of CCSS Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year Percentage of total number of 50.00 Feb/2016 80.00 Mar/2020 80.00 Apr/2022 90.10 Dec/2021 primary health care units (Areas de Comments on achieving targets Indicator achieved its end of Project target. Salud) with the Unique Digital Health Record e-health package (EDUS) (Percentage) Page 32 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Budget does not use Feb/2016 Annual budget May/2022 Annual budget Dec/2023 Target achieved in Jan/2024 Development and execution of a prospective tools. execution using execution using January 2024. comprehensive medium-and long- prospective tools. prospective tools. term plan to ensure the financial Comments on achieving targets Indicator achieved (DLI achievement confirmation proceeded during grace period, in February 2024, but the sustainability of the CCSS (Text) indicator was achieved in January 2024). Intermediate Results Indicators by Result Areas Health Care Model Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year Cumulative percentage reduction in 1032 days Feb/2016 671 (35% lower) Mar/2022 35% lower Apr/2022 573 days Jun/2020 average waiting days for selected Comments on achieving targets Indicator achieved its end of Project target. procedures (tracers): hip and knee replacement. (Text) Percentage of individuals diagnosed 62.00 Dec/2013 64.00 Mar/2020 64.00 Apr/2022 65.70 Jun/2020 with hypertension that are under Comments on achieving targets Indicator achieved its end of Project target. optimal clinical control. (Percentage) Institutional Management Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year Percentage of households covered by 70.00 Feb/2016 100.00 Mar/2022 100.00 Apr/2022 100.00 Dec/2021 the Family Socio-Economic & Health Comments on achieving targets Indicator achieved its end of Project target. (II3) Risks Information File. (Percentage) Percentage of households with 0.00 Feb/2016 16.00 Mar/2022 16.00 Apr/2022 66.90 Apr/2023 Family Socio-Economic & Health Risks Comments on achieving targets Indicator achieved its end of Project target. Information File digital geo- referencing data available online. (Percentage) Page 33 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT - Feb/2016 Two annual reports Apr/2023 System designed, Dec/2023 System designed, Dec/2023 published and approved and approved by Board Use of institutional strategic disseminated launchedTwo of Directors and intelligence system data to publish annual reports annual report and disseminate two annual reports published and published and measuring progress of SASHI (Text) disseminated disseminated Comments on achieving targets Indicator achieved. Final report on the implementation of NOVAPLAN was submitted by the CCSS in November 2023 and accepted by the Bank. Institutional Review of core - Feb/2016 Approved by Board Mar/2018 Approved by Board Apr/2022 Approved by the Dec/2021 institutional management functions, of Directors of Directors Board of Directors and operational plan for progressive Comments on achieving targets Indicator achieved its end of Project target implementation of streamlining central level management. (Text) Financial Management Indicator Name Baseline Closing Period (Original) Closing Period (Current) Actual Achieved at Completion Result Month/Year Result Month/Year Result Month/Year Result Month/Year - Feb/2016 Annual Financial Apr/2022 Approved Dec/2023 Annual Financial Nov/2022 Statement using Evaluation report of Statement using outputs from new first two outputs from new Financial statements produced using automated yearsAnnual automated new international public accounting streamlined system Financial Statement streamlined system. standards, streamlined and digital using outputs from automated processes. (Text) new automated streamlined system Comments on achieving targets Indicator achieved. Shadow budget constructed based on - Feb/2016 Completed Mar/2018 Completed Jan/2024 Achieved Dec/2023 gender and age risk adjusted Comments on achieving targets Indicator achieved its end of project target. capitations. (Text) Disbursement Linked Indicators (DLI) Page 34 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Period Period Definition Period 1 At achievement Baseline Period 1 1:Percentage of major surgeries from priority list conducted in Achievement Level: outpatient settings according to CCSS institutional guidelines. (Percentage ) Original/Revised Value 41.60 46.00 Allocated Amount 0.00 60,000,000.00 Actual Value 49.80 Actual Amount 60,000,000.00 2:Cumulative percentage of target population personally invited Achievement Level: to undergo colon cancer screening in five priority counties. (Percentage ) Original/Revised Value 0.00 40.00 Allocated Amount 0.00 60,000,000.00 Actual Value 40.10 Actual Amount 60,000,000.00 3:Percentage of individuals diagnosed with Diabetes Type II that Achievement Level: are under optimal clinical control. (Percentage ) Original/Revised Value 39.00 43.00 Allocated Amount 0.00 60,000,000.00 Actual Value 45.70 Actual Amount 60,000,000.00 4:Pilot project on integrated health networks approved by the Achievement Level: CCSS Board and implemented for a selected population and territory and evaluated with the results publicly disseminated. (Text ) Original/Revised Value - Allocated Amount 0.00 60,000,000.00 Actual Value Pilot project approved and evaluation of pilot disseminated Page 35 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Actual Amount 60,000,000.00 5:Percentage of total number of primary health care units (Areas Achievement Level: de Salud) with the Unique Digital Health Record e-health package (EDUS). (Amount(USD) ) Original/Revised Value 50.00 80.00 Allocated Amount 0.00 60,000,000.00 Actual Value 90.10 Actual Amount 60,000,000.00 6:Redesign, implementation and use of data collected by the new Achievement Level: annual survey to measure impact of new interventions to improve patient satisfaction. (Text ) Original/Revised Value - Allocated Amount 0.00 60,000,000.00 Actual Value Board approval of redesign of patient satisfaction survey (Milestone 1) and Report on impact of interventions to improve patient satisfaction approved by CCSS Board (Milestone 2) Actual Amount 60,000,000.00 7:Development and execution of a comprehensive medium- and Achievement Level: long-term plan to ensure the financial sustainability of the CCSS. (Text ) Original/Revised Value - Allocated Amount 0.00 60,000,000.00 Actual Value Milestone III: An annual budget executed having used risk-adjusted capitations as input for integrated networks of first and second level of care and DRGs as inputs for third level hospitals and other national specialized hospitals. Actual Amount 60,000,000.00 A. PROGRAM ACTION PLAN Page 36 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Action Timing Timing Value Achieved? Completion Measurement Contracting an Independent Verification Entity (IVE). Due Date 29-Jun-2018 Yes Contract signed Comments An IVE (PAHO) was hired and completed the verification of all DLIs of the PforR. To achieve and verify the indicator related to cumulative percentage of target population Due Date 30-Jun-2022 Yes Indicator Verified personally invited to undergo colon cancer screening in five priority counties. Comments Indicator achieved and verified. To achieve and verify the indicators related to the pilot project on integrated health networks approved by the CCSS Board and implemented for a selected Due Date 30-Dec-2022 Yes Indicator Verified population and territory and evaluated with the results publicly disseminate Indicator 4.1 achieved and verified. Indicator 4.2 achieved and disbursed, with dissemination of Comments results of the pilot completed during the mission carried out in October 2022. Verification took place in February 2023. To achive and verify the indicators related to redesign, implementation and use of data collected Due Date 30-Jun-2023 Yes Indicator verified by the new annual survey to measure impact of new interventions to improve patient satisfaction. Comments DLI 6.1 and 6.2 have been verified. Verification of milestone 6.2 took place in October 2023. To achive and verify the indicators related to the development and execution of a comprehensive Due Date 30-Dec-2023 Yes Indicator Verified medium- and long-term plan to ensure the financial sustainability of the CCSS Page 37 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT Comments All three milestones for DLI 7 (i.e., 7.1., 7.2 and 7.3) were achieved and verified. Page 38 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Laura Di Giorgio Team Leader Alvaro Gilberto Fernandez Trigoso Financial Management Specialist Sandra Lisette Flores De Mixco Financial Management Specialist Maria Camila Padilla Gomez Procurement Specialist Diacono Raul Vera Hernandez Environmental Specialist Ricardo Marten Caceres Social Specialist Maria Pia Cravero Counsel Tatiana Cristina O. de Abreu Souza Team Member Renata Pantoja Team Member Maria Virginia Hormazabal Team Member Micaela Mussini Team Member Viviana A. Gonzalez Team Member Fernanda Balduino de Oliveira Team Member @#&OPS~Doctype~OPS^dynamics@icrannexstafftime#doctemplate B. STAFF TIME & COST Staff Time & Cost Stage of Project Cycle No. of Staff Weeks US$ (including travel and consultant costs) Preparation FY14 11.713 91,439.06 FY15 24.803 175,496.38 Page 39 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT FY16 32.345 240,570.76 Total 68.86 507,506.20 Supervision/ICR FY14 0.000 4,140.00 FY17 22.950 130,024.34 FY18 30.418 192,037.17 FY19 35.122 331,380.20 FY20 25.063 145,891.63 FY21 21.538 142,067.19 FY22 23.590 171,989.52 FY23 20.190 140,300.21 FY24 26.469 165,244.90 Total 205.34 1,423,075.16 Page 40 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT ANNEX 3. PROGRAM EXPENDITURE SUMMARY Actual Expenditures (Disbursement) Source of Program Type of Co- Estimates at Financing (US$) Financing Appraisal Percentage of Percentage of Actual Appraisal Actual World Bank Loan 420.00 420.00 26.7% 26.7% Borrower - 1,155 1,155 73.3% 73.3% Other Partners - - - - - Total - 1,575 1,575 100% 100% Page 41 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT ANNEX 4. BORROWER’S COMMENTS 1. The following is a direct extract and summary of the preliminary closing report of the Strengthening Universal Health Insurance PforR prepared by the CCSS and submitted by the Ministry of Finance to the World Bank. This extract was translated to English by the Bank; any mistakes in translation are the sole responsibility of the Bank. The Borrower reviewed and approved the final draft version of the ICR. I. EXECUTIVE SUMMARY ▪ Program Coordinating Team: The program, designated as a "Special Project" by CCSS General Management in memo GG-2080-2020, was to be implemented by a Program Coordinating Team, authorized by the Project Director and with the ability to make interim staff appointments as necessary. The CCSS Board of Directors allowed for the temporary replacement of institutional personnel engaged in the program with substitutes under the same job code. The team, formed as of February 14, 2018, consisted of seven members, including a project coordinator, a deputy project coordinator, a M&E specialist, a secretary, and others, all overseen by the Director to ensure the project's progress. Further, its primary objective was the coordination and follow-up of the commitments acquired by the CCSS before the World Bank, the Legislative Assembly, the Comptroller General of the Republic, the Ministry of Finance and the Public Credit Directorate, for the successful management of the disbursements derived from the fulfillment of program indicators. ▪ Verification unit: On July 27, 2018, an agreement was signed with the PAHO, which translated into the successful execution of the verification of 14 milestones related to disbursements by the end of the year 2023. ▪ Restructuring/Adjustments: The Institution was required to submit two requests to Ministry of Finance’s Public Credit Directorate for an extension of the closing date of the Program in relation to Loan Agreement 8593-CR: one in 2022 due to the COVID-19 pandemic interference with the progress of several indicators and a subsequent cyberattack on CCSS systems; and the other in 2023. The first request resulted in the extension of the PforR until December 31, 2023; and the second, until January 2024. ▪ Results: At the time of Program closure, the PforR had reached 100% compliance, having completed all the necessary steps to comply with all the commitments defined in Loan Agreement No. 8593-CR, successfully verifying 15 disbursement indicators and complying with 8 intermediate indicators and 3 commitments related to environmental management, occupational health, and social management. All commitments outlined in the loan contract have been met. However, it is crucial to note the CCSS's interest in ensuring the sustainability of the improvements in metrics, efficiency, and service quality. II. PROGRAM OBJECTIVES 2. General Objective: To contribute to: (i) improving the timeliness and quality of health services; and (ii) enhancing the institutional efficiency of the CCSS. 3. Per CCSS internal version, the program's general objective was to manage the corresponding actions to achieve the objectives defined in Law 9396 "Approval of Loan Contract N°8593-CR and its Annexes between the Republic of Costa Rica and the International Bank for Reconstruction and Development, for the financing of the Page 42 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT PforR for the Strengthening of Universal Health Insurance in Costa Rica", in order to achieve the disbursement of financial resources for the DLIs and the compliance with the intermediate and cross-cutting indicators for the Strengthening of Universal Health Insurance. Specific Objectives: The PforR focused on three institutional priority areas: a. Strengthening the health care delivery model, with particular emphasis on increasing the impact of PHC, the integration of integrated health care networks, and a more effective approach to the prevention, early diagnosis, and control of chronic NCDs. This area focused on strengthening PHC coverage and capacity to prevent and control NCDs and integrating health services at different levels using international best practices to provide comprehensive and timely treatment and ensure continuity of care at different levels of the health system. b. Improving institutional management: Strengthen institutional capacities to improve the management of the CCSS and, in particular, the health insurance system. c. Optimizing financial management: Focused on improving the quality of financial data for the Executive Management and the Board of Directors. This will allow them to closely monitor actuarial analyses and trends in revenues and expenditures. III. PROGRAM ALIGNMENT WITH INSTITUTIONAL GOALS 4. The relationship between PforR objectives and the broader objectives of the CCSS Program is high, since its purpose is the strengthening of the Universal Health Insurance, through the fulfillment of a series of indicators which were focused on improving the health care model with a special emphasis on increasing the impact of PHC, the integration of integrated networks for health care, and a more efficient approach to prevention, early diagnosis, and control of NCDs, as well as the improvement of institutional administration, without neglecting the optimization of financial management. 5. All commitments outlined in the loan contract have been met. However, it is crucial to note the CCSS's interest in ensuring the sustainability of the improvements in metrics, efficiency, and service quality. In response, the program coordination unit consulted with the technical teams responsible for the indicators, seeking measures to sustain the achievements beyond the PforR life. The teams provided extensive feedback, proposing various measures and strategies. IV. RESULTS 6. Disbursement Linked Indicators (DLIs): The loan was conditioned on the achievement or fulfillment of commitments, specifically seven DLIs related to waiting lists, chronic diseases and medical care, service delivery model, technology and information systems, health service user satisfaction, and financial sustainability. For the achievement of each DLI, USD 60 million was allocated, for a total of USD 420 million under the loan agreement. Table 1. Disbursement indicators As of June 11, 2024 Status / Indicator Milestone Progress Achievement Page 43 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT DLI 1: Percentage of Target achieved. The milestone verification major surgeries from 1.1: 43% process by the PAHO 51 was completed in Verified the priority list December 2018 performed on an Target achieved. As agreed in the World Bank outpatient basis Mission of October 2022, the indicator is according to CCSS 1.2:46% Verified considered to be met (Letter of Scope institutional guidelines. Indicator 1.2, dated December 8, 2022). Target achieved. The milestone verification DLI 2.1: 10% process by the PAHO 52 was completed in Verified December 2018 Target achieved. The PAHO milestone verification process was completed in July DLI 2.1: Cumulative 2022. It is possible to verify compliance with percentage of target Milestone 2 of DLI2, where the CCSS has population invited in achieved the objective of asking at least 40% person for colorectal of the target population to receive the cancer screening in the DLI 2.2 40% Verified immunological test for fecal occult blood five priority cantons. screening. The PAHO confirms that the CCSS has satisfactorily achieved milestone 2 of DLI 2, reaching 40.06%. This percentage exceeds the target proposed in the program of inviting at least 40 % DLI 3. Porcentaje de Target achieved. The milestone verification individuos DLI 3.1: 41% process by the PAHO 53 was completed in Verified diagnosticados con June 2019 Diabetes Tipo 2 en Target achieved. The milestone verification Control Clínico Óptimo DLI 3.2 :43% process by the PAHO 54 was completed in Verified June 2019 DLI 4.1: Board Target achieved. The milestone verification DLI 4.1: Pilot project on of Directors’ process by the PAHO 55 was completed in Verified integrated health approval of the October 2018 networks approved by Pilot Plan the Board of Directors Verification performed and approved. The of the CCSS and verification process was carried out in DLI 4.2 implemented for a November and December 2022, including Assessment of selected population field visits by the PAHO consultant to various the pilot and territory and regions and facilities in the Huetar Atlantic Verified project evaluated with results zone. The result and final positive resolution disseminated to disseminated to the of the process is included in the Verification the public public Report sent to the World Bank through official letter PE-PRCCSS-BM-227-2022 dated 51 Final Verification Report: Disbursement-linked indicator #1/ Milestone 1: Percentage of major surgeries on the priority list performed on an outpatient basis according to CCSS Institutional guidelines. 2018. 52 Final Verification Report: Disbursement-linked indicator #2/ Milestone 1: Cumulative percentage of target population invited in person for colon cancer screening in the five priority cantons. 2018 53 Final Verification Report: Disbursement-linked indicator #3/ Milestone 1: Percentage of individuals diagnosed with Type 2 Diabetes in Optimal Clinical Management. 201 54 Final Verification Report: Disbursement-linked indicator #3/Milestone 2: Increase in the percentage of individuals diagnosed with Type 2 diabetes receiving optimal clinical control to 43%. 2019. 55 Final Verification Report: Disbursement-linked indicator #4/ Milestone 1: Pilot project on comprehensive health networks approved by the CCSS Board of Directors. 2018. Page 44 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT December 20, 2022. DLI 5. Percentage of the total number of Target achieved. The milestone verification Primary Health Care DLI 5.1: 60% process by the PAHO 56 was completed in Verified Units with the December 2018 electronic health package with Single Target achieved. The milestone verification Digital Health Record DLI 5.2 80% process by the PAHO 57 was completed in Verified (EDUS) December 2018 DLI 6.1: Redesigned DLI 6. Redesign, Target achieved. The milestone verification survey implementation and process by the PAHO 58 was completed in Verified approved by use of data collected in June 2019 the Board of the new annual survey Directors to measure the impact DLI 6.2 of new interventions to Evaluation of Target achieved. The milestone verification improve patient the pilot process by the PAHO 59 was completed in satisfaction. Verified project November 2023 disseminated to . the public DLI 7.1: Elaboration Ten-year and implementation of investment a medium and long- Master Plan Target achieved. The milestone verification term plan to guarantee approved by process by the PAHO 60 was completed in Verified the financial the Board of February 2021 sustainability of the Directors of the health insurance CCSS system. DLI 7.2: Elaboration Actuarial Target achieved. The milestone verification and implementation of Valuation of process by the PAHO 61 was completed in Verified a medium and long- Health March 2020 term plan to guarantee Insurance 56 Final Verification Report: Disbursement-linked indicator #5/ Milestone 1: Percentage of the total number of Primary Care Units with the electronic health package with Single Digital Health Record.2018 57 Final Verification Report: Disbursement-linked indicator #5/ Milestone 2: Percentage of the total number of Primary Care Units with the electronic health package with Single Digital Health Record. 2018. 58 Final Verification Report: Disbursement-linked indicator #6/ Milestone 1: Redesign, implementation and use of data collected in the new annual survey to measure the impact of new interventions to improve patient satisfaction. 2019. 59 Final Verification Report: Disbursement-linked indicator #6/ Milestone 2: Evaluation of outcomes of interventions to improve patient satisfaction: Report on the impact of interventions to improve patient satisfaction. 60 Final Verification Report: Disbursement-linked indicator #7/ Milestone 1: Development and implementation of a medium- and long-term plan to guarantee the financial sustainability of the Health Insurance. 2021. 61 Final Verification Report: Disbursement-linked indicator #7/ Milestone 2: DLI #7.2: Development and implementation of a medium- and long- term plan to ensure the financial sustainability of the Health Insurance. 2020. Page 45 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT the financial system It was verified that milestone 2 of DLI 7 has sustainability of the been achieved with full satisfaction from the health insurance descriptive process detailed in the afore- system. mentioned final report: Actuarial Valuation of Health Insurance 2017, performed by the CCSS, its consecutive approval by the Board of Directors, its publication on the website of the same institution and all official documentation supporting such processes. Target achieved. The milestone verification process by the PAHO 62 was completed in April 2024 The delays caused by the cyber-attack forced DLI 7.3: Annual budget the Budget Department to formulate an formulated on the basis - Verified abbreviated budget for 2023; this prevented of prospective tools the incorporation of prospective tools. These tools will be incorporated, in agreement with the World Bank, for 2024 budgeting, so the indicator compliance date is expected for the fourth quarter of 2023, with verification to be carried out in the first half of 2024. 7. Intermediate Indicators: These indicators did not result in disbursements; however, they contributed to meeting the DLIs, as well as the development objectives of the PforR, such as strengthening PHC, with emphasis on prevention, early diagnosis, and control of chronic NCDs. Table 2. Intermediate Indicators Indicator Progress Status / Achievement II1. Cumulative percentage decrease in waiting days for selected procedures (markers): hip and knee Target achieved in replacements. September 2020. Achieved Target: 35% reduction in waiting days (Baseline of 1032 waiting days) II2. Percentage of people diagnosed with hypertension receiving adequate clinical control Target achieved in Target: 64% of people diagnosed with hypertension September 2020 Achieved with adequate clinical control (Baseline of 62% in optimal clinical control). II3. Percentage of households covered by the Integrated Family File System - IFFS Target achieved in Achieved Target: 100% of households entered in IFFS September 2020. (Baseline 0%) II4.Percentage of households with geo-referenced Target achieved in Achieved 62Final Verification Report: Disbursement-linked indicator #7/ Milestone 3: DLI #7.3: CCSS annual budget approved by the Board of Directors and CGR using prospective budgeting tools. 2024. Page 46 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT digital IFFS data available online. 2022. II5.Strategic institutional intelligence system, Pending Achieved progress of AEFSS II6. Review of the main institutional management functions and the operational plan for the strategic Target achieved in progressive implementation of central level September 2020. Achieved optimization. Target: Design proposal approved by the Board of Directors II7. The financial statements generated use the new Achieved Target achieved in International Public Sector Accounting Standards June 2024. (IPSAS). II8. Shadow Budget Target achieved in Achieved November 2023. 8. Social, Environmental and Occupational Health Commitments (Cross-cutting Indicators): Based on the Environmental and Social Assessment of the Program and the risk analysis carried out by the CCSS together with the World Bank, a series of actions were agreed upon to improve environmental, occupational health and social management, reflected in the following commitments that cut across the Institutional work. 9. As with the intermediate indicators, the commitments referred to as "cross-cutting" are ratified as fulfilled by the World Bank. Table 3. Cross-cutting Commitments Status/ Commitment Progress Achievement IIIa- Environmental: 1. Strengthening the Target achieved and approved by the World Achieved CCS Environmental Management System. Bank in November 2023. IIIb- Occupational Health: Occupational Target achieved and approved by the World Health Management Information System Achieved Bank in November 2023. (SIGACCSS). IIIc- Social / Vulnerable Populations: Strengthening and execution of the Target achieved and approved by the World Achieved Differentiated Attention Program for Bank in November 2023. Indigenous Populations. V. MAIN CHALLENGES ENCOUNTERED DURING IMPLEMENTATION AND ACTIONS TAKEN FOR THEIR RESOLUTION 10. The relevant events that delayed the fulfillment of goals and indicators during the development of program execution were as follows: a. Approval of Loan Agreement No. 8593-CR via Law No. 9396: This approval was secured five months after the agreement of Loan 8593-CR (April 21, 2016), which caused delays in the establishment of the necessary order to formally set up the structures for the execution and follow-up of the commitments within the institution. b. Late Conformation of the Program Coordinating Team: The Coordination Team was formed Page 47 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT 17 months after the approval of Law No. 9396, which meant delays in formalizing the supervision and defining the structures that would ensure continuous monitoring and effective communication with the technical teams and World Bank representatives. c. Hiring of the Verification Unit, which began its work at the end of July 2018, meaning uncertainty as to the entity that would carry out such processes in the initial stages of the Program. This made it impossible to determine verification methodologies with the technical teams in charge of the indicators with shorter compliance deadlines. d. Covid-19 Pandemic: Dealing with the arrival of the COVID-19 pandemic in March 2020 interfered with the progress of several indicators, as reported by the technical coordinators executing each of the indicators. Thus, it was concluded that targets would not be achieved before April 30, 2022, the program’s original closing date. In response to this, and after meetings held with WB representatives and Public Credit of the Ministry of Finance, an extension of the closing date of the Loan Contract was negotiated. The deadlines were subsequently extended, allowing the technical institutional executing teams to renew, adjust and modify their work schedules, strategies and actions so that the different indicators and commitments that make up the Program could be met and follow-up activities by international consultants carried out in a timely manner. COVID-19 also meant that technical visits to places and regions of the country that represent points of interest for the management of the different indicators were delayed. The request for extension of the program until December 31, 2023, submitted to the institutional authorities by the PforR Coordination Team with the support of the Executive Presidency, was approved on Thursday, January 27, 2022. e. Cyber-attack on CCSS information systems: As a result of the cyber-attack on the CCSS information systems that began on May 31, 2022, the Institution, through the Department of Technology and Communications, took several precautionary measures while the computer systems were being rebuilt, including disconnecting the Institution's networks and its platforms. CCSS services suffered a semi-paralysis for several months; gradually being re- activated at the end of July 2022. This implied the disconnection of the Single Digital Health Record (EDUS) platforms, even though the digital file platforms and those related to payroll and pensions were not compromised. Page 48 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT 1. The impact directly related to PforR implementation was as follows: a. The disconnection of the EDUS platforms affected health areas and hospitals in the production of reports and statistics that would not be updated for several months and whose recovery was very slow (healthcare model- focused indicators). b. The indicators designed to improve institutional management are developed at the central level and their objective was linked to aspects of management, health insurance, administrative and financial processes, and changes; and the use of IT tools and health tools to generate large data sets for decision-making at the managerial level. The disconnection of the Human Resources, Budget and Planning platforms had an impact on the development and application of new information systems and household data studies. c. Milestones to optimize financial management at an institutional level, designed to improve the quality of financial data at the management level, were affected by the cyberattack in the form of delays in the information that feeds these programs. The platforms could not be used, thus, the implementation of accounting standards, budget reports, development and start-up were delayed. 2. Regarding the direct impact of the hacking on the PforR in its day-to-day management, we can cite the following: a. Employees were asked to telecommute for institutional security reasons. b. Use of non-institutional virtual platforms such as Teams and Zoom. c. The inability to process budget changes and reports due to the deactivation of institutional platforms. d. Impact on general procedures related to human resources due to the disconnection of SOGERH (Human Resources Management Operating System63). e. Difficulties in preparing responses to the provisions of the Office of the Comptroller General of the Republic related to PforR management. 3. Bearing in mind that the Institution was recovering from the disruption caused by the COVID-19 pandemic, which had forced it to adjust the work programs of each of the indicators; this new impact led to the need to request the second extension of the agreement to Loan Contract 8593-CR, which was approved by the World Bank in December 2023. I. LESSONS LEARNED FROM IMPLEMENTATION ▪ As a starting point for the implementation of future projects, it is important to first consolidate and reach consensus on a strategic agenda to improve efficiency and effectiveness. ▪ It is important to hold initial discussions between the Bank and the Program Implementation Unit to identify priority issues and their positioning within the institutional agenda. ▪ Ensure joint follow-up and active communication between the parties to address implementation obstacles, identification of issues and timely achievement of indicators. ▪ A strength that directly translated into the achievement of the program's objectives was the existence of a comprehensive strategic program and the creation of a CCSS multidisciplinary coordination team that was exclusively dedicated to the follow-up of the existing contractual commitments. ▪ The type of lending instrument and its focus on results also helped implementation along, since disbursement against results in specific timeframes provides the necessary incentives to achieve them, thus contributing to 63 Sistema Operativo Gestión de Recursos Humanos in Spanish. Page 49 The World Bank Strengthening Universal Health Insurance in Costa Rica (P148435) ICR DOCUMENT improved accountability. ▪ Technical assistance from the World Bank and exchanges with experts from other countries were essential to support the design and implementation of complex strategic reforms. ▪ The financial support and technical assistance provided by the Bank to overcome the challenges facilitated the achievement of the program's objectives. ▪ The support and commitment of senior institutional authorities was crucial to the successful implementation of complex reforms and the achievement of program objectives. ▪ Critical to the development and achievement of the agreed commitments was the leadership and commitment of those responsible within the technical teams in charge of the indicators. ▪ The leadership and commitment of members of the World Bank team and the CCSS Coordinating Team were crucial to the development and achievement of the agreed commitments. ▪ Proximity and direct communication with the contract borrower, the Ministry of Finance, allowed for problems to be identified and resolved quickly and effectively. II. RECOMMENDATIONS ▪ Ensure the timely beginning of this type of program by expediting its initial phases and thus minimize delays that may occur later by way of extensions. ▪ The implementation of this type of program requires robust computer systems at the national level that incorporate modern cybersecurity concepts. ▪ Consideration should be given to the realities and specificities of each country in which a PforR is under negotiation or implementation. ▪ In this sense, aspects such as the economic situation, the social situation, the political situation, the particular behavior of the country should be taken into account in order to make the necessary adjustments. ▪ The World Bank is encouraged to maintain the continuity of the Project Manager and associated consultants throughout the life of the contract, as this will facilitate management and maintain a stable line of work. ▪ Throughout the development of the program, it is important for the implementing unit to maintain the support of the institutional authorities. ▪ The PforR model has proven successful in its implementation in the CCSS, making it a viable format for future projects, both within and outside the institution. Page 50