ICRR 10593 Report Number : ICRR10593 ICR Review Operations Evaluation Department 1. Project Data : Date Posted : 06/01/2000 PROJ ID : OEDID : L3527 P006639 OEDID: Appraisal Actual Project Name : Health Sector US$M ) Project Costs (US$M) 298.8 287.78 Country : Chile Loan/ US$M ) Loan /Credit (US$M) 90 90 Sector, Major Sect .: Other Population US$M ) Cofinancing (US$M) 208.8 197.78 Health & Nutrition , Population Health & Nutrition L/C Number : L3527; LP160 FY ) Board Approval (FY) 93 Partners involved : Government, and Closing Date 06/30/1998 06/30/1999 Japanese grant facility Prepared by : Reviewed by : Group Manager : Group : Chorching Goh John Glen Johnson Gregory K. Ingram OEDST 2. Project Objectives and Components a. Objectives To improve efficiency of public health care services through a range of policy and institutional changes, and to raise the quality of health care in selected areas . b. Components To decentralize the health service delivery by 1. redefining the missions and organization of the MOH and its decentralized agencies, 2. adopting modern management instruments, 3. developing human resources, and 4. establishing management information systems . To integrate the health service delivery by constructing /rehabilitating ambulatory service facilities, and hospitals in order to launch a new health care model and fill identified health service gaps in health areas serving the poor. To address cervical cancer and environmental health problems through Special Programs . c. Comments on Project Cost, Financing and Dates Except for Special Programs, actual costs were higher in all other components . If the project has not deleted several facility investments from the original plan, the total costs would be 25% (US$373 million) over the appraised estimate. The higher incurred costs and fewer constructed facilities balanced out with a 3.7% decrease in total project costs at closing, and full disbursement of the loan. 3. Achievement of Relevant Objectives : Decentralization of health services 1. The goal of refining missions and organization of the MOH, and its decentralized agencies is substantially fulfilled. The MOH reinforced the delegation of health service delivery to decentralized agencies (i.e., Health Service Areas (HSAs)) and increased their professional and managerial staff . It also developed sector-wide norms on health facilities. Other central agencies such as the Public Health Institute, the Central Supply Facility, and the National Health Fund have also restructured their institutions . 2. The process to adopt new management instruments has begun, but still leaving room for improvement . (i) The service provision agreements between MOH /FONASA and Health Service Areas now link resource -allocation to performances and productivity, instead of historical supply -based budgets. (ii) A payment system (PAD) based on diagnostic related group (which associated advanced payments with specific care ) was adopted to replace conventional ex-post fee-for-service system. This system intends to introduce transparency, improve efficiency, and develop a realistic cost accounting system . 3. A human resource department is established to serve as a training secretariat . Its training program emphasized improvement in administrative and financial efficiency, and the MOH's new mission . 4. Establishment of the MIS was delayed, and the quality of operation has been poor . The telecommunication system has not been developed because ownership, execution responsibility and cost -sharing could not be agreed upon. Integration of the health service delivery : The data to date indicated an increase in ambulatory services, and a reduction in internal referrals, and in average length of hospital stay. However, it is too early to attribute this improvement entirely to the project . Although infrastructure and equipment are in place, protocols for management of these facilities were not adequately developed, and needed human resources were not fully deployed. On special programs 1. to address cervical cancer, the project has increased coverage of screening from 20% to 66% (short of its targeted 80%); the timeliness of treatment has also improved though screening did not appear to give priority to those above 35, as planned. A national reference laboratory was also established to implement quality control and strengthen capacity in all laboratories . 2. to address environmental health problems, this project piloted an environmental education program that trained instructors, and monitors to raise public awareness at the community level; it also established an environmental network, and implemented micro -projects on waste management across Municipality of Santiago. 4. Significant Outcomes /Impacts : All central agencies have restructured to improve efficiency. E.g., the central supply facility (CENABAS) which served as a purchasing intermediary between private suppliers and HSAs has streamlined its operations and restructuring its personnel. It has developed a computerized bidding process to enhance competition, prepared a team of advisers for HSAs and hospitals, and established a communication network with HSAs and major hospitals. The Public Health Institute (ISP) has increased reliance on own operational income, reduced process time for medical product approval, and improved regulatory functions . The National Health Fund (FONASA), albeit still with some limitations, has defined a bill of rights for its beneficiaries, and instituted mechanisms for patients to file complaints . Even though the quality of diagnostic tests has not improved, the increased coverage, and improved timeliness of laboratory reporting and treatment on cervical cancers have partially contributed to a consistent reduction in the incidence of cervical cancer by 4% a year, during 1990-95. A law was modified to regulate contractual treatment of pre -existing diseases, and to make private health plans more transparent, and movement of policy holders between insurance institutions easier . 5. Significant Shortcomings (including non -compliance with safeguard policies ): There are no incentives for compliance in Service Provision Agreements between the National Health Fund/MOH and Health Service Areas (HSAs), and output or performance indicators used in the Agreement do not allow HSA managers to have discretion on how to reach those targets . Although compliance rates have improved, they have reached only 79%. The central MIS system linking implementing agencies has not yet been developed; many targeted hospitals and Health Service Areas (HSA) have not set up their MIS because they lack the capacity to define their needs and priorities on their own. The ICR suggested a twinning arrangement with agencies overseas to provide technical assistance and guidance . Only 61% of Health Service Areas have adopted the new payment system (PAD) for budget allocation. The reasons are partly delays in the MIS, and partly a more realistic calculation of services in the new system implies higher costs. Reform in human resources management and health financing was delayed and diluted because of opposition by stakeholders and interest groups, which was unforeseen during project's preparation . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory 7. Lessons of Broad Applicability : 1. Decentralized project implementation is likely to increase efficiency; however, it is still necessary to introduce competition and incentives among decentralized entities, and provide weaker agencies with technical assistance. Equally important, decentralization must be applied without losing efficiency from economy-of-scale or appropriate centralized organization . For example, the decentralized manner in setting up the Management Information System and communications networks was poorly coordinated, with each HSA contracting a software company to design its own system; a more productive approach may be to set up a central system in MOH, and then replicate to the others . 2. This project is too complex to be implemented effectively within the few years . Such a substantive health sector reform requires learning and capacity -building. The impact and results from the reform will likely be maximized through multiple sequential projects that undertake areas by areas gradually . 3. A strenuous evaluation effort only at the end of the implementation is less productive . Monitoring and evaluation should receive significant attention in the design and during implementation . An ongoing evaluation process enables implementing agencies to correct shortcomings promptly, and improve implementation. This process can also build local institutional capacity to continue with the program beyond the project's duration . 4. The government needs to reduce resistance to sectoral reform by educating the public of the reform's rationale, monitoring the project's impacts, and disseminating results . Similarly, adequate consultation with prospective users is crucial before construction of any facilities because users' objection that hinders execution is extremely costly . 8. Audit Recommended? Yes No Why? This project undertakes an interesting and substantive reform in the health sector, with potential lessons to be disseminated. Impacts of several component of the project were still unknown at project's closing, and a review at a later date will be informative . For example, benefits of the training program were not assessed, and the impact of its new health care model was too earlier to be examined at project's closing . 9. Comments on Quality of ICR : The ICR is very comprehensive, illustrating both the limitations and achievement of the project. The Borrower's contribution to the ICR is equally substantive .