Document of The World Bank Report No: 30441 IMPLEMENTATION COMPLETION REPORT (TF-25299 FSLT-70050) ON A LOAN IN THE AMOUNT OF EURO 9.0 MILLION (US$9.5 MILLION EQUIVALENT) TO THE REPUBLIC OF SLOVENIA FOR A HEALTH SECTOR MANAGEMENT PROJECT December 24, 2004 Human Development Sector Unit Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective October 27, 2004) Currency Unit = Slovenian Tolars (SIT) SIT 187.3 = US$ 1 US$ 0.005 = SIT 1 FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS AHIC Australian Health Insurance Commission APL Adaptable Program Lending CAS Country Assistance Strategy CEN European Center of Standards CHD Coronary Heart Disease CPG Clinical Practice Guideline DRGs Diagnostic Related Groups IBRD International Bank for Reconstruction and Development EU European Union GIN Guidelines International Network GOS Government of Slovenia HIIS Health Insurance Institute of Slovenia HISA Health Information Systems Architecture HSMP Health Sector Management Project IFMIS Financial Management Information Systems MCS Medical Chamber of Slovenia MOF Ministry of Finance MOH Ministry of Health NHDMC National Health Data Management Center NHIC National Heallth Information Clearinghouse OECD Organization for Economic Cooperation and Development PMU Project Management Unit QER Quality Enhancement Review QAG Quality Assurance Group TC251 Technical Committee 251 UMCL University Medical Center of Ljubljana Vice President: Shigeo Katsu Country Director Roger Grawe Sector Manager Armin Fidler Task Manager: Sarbani Chakraborty SLOVENIA HEALTH SECTOR MANAGEMENT PROJECT CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 1 4. Achievement of Objective and Outputs 5 5. Major Factors Affecting Implementation and Outcome 11 6. Sustainability 12 7. Bank and Borrower Performance 12 8. Lessons Learned 14 9. Partner Comments 15 10. Additional Information 26 Annex 1. Key Performance Indicators/Log Frame Matrix 27 Annex 2. Project Costs and Financing 28 Annex 3. Economic Costs and Benefits 30 Annex 4. Bank Inputs 31 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 32 Annex 6. Ratings of Bank and Borrower Performance 33 Annex 7. List of Supporting Documents 34 Annex 8. Beneficiary Survey Results 35 Annex 9. Stakeholder Workshop Results 39 Project ID: P051418 Project Name: HEALTH SECTOR MANAGEMENT PROJECT Team Leader: Sarbani Chakraborty TL Unit: ECSHD ICR Type: Intensive Learning Model (ILM) of ICR Report Date: December 30, 2004 1. Project Data Name: HEALTH SECTOR MANAGEMENT PROJECT L/C/TF Number: TF-25299; FSLT-70050 Country/Department: SLOVENIA Region: Europe and Central Asia Region Sector/subsector: Health (55%); Central government administration (38%); Compulsory health finance (7%) Theme: Health system performance (P); Law reform (S) KEY DATES Original Revised/Actual PCD: 05/09/1999 Effective: 06/30/2000 06/30/2001 Appraisal: 09/20/1999 MTR: 06/30/2002 04/22/2003 Approval: 01/20/2000 Closing: 06/30/2004 06/30/2004 Borrower/Implementing Agency: REPUBLIC OF SLOVENIA/MINISTRY OF HEALTH; REPUBLIC OF SLOVENIA/HEALTH INSURANCE INSTITUTE OF SLOVENIA Other Partners: Institute of Public Health, Clinical Center, Ljubljana STAFF Current At Appraisal Vice President: Shigeo Katsu Johannes Linn Country Director: Roger W. Grawe Roger W. Grawe Sector Manager: Armin Fidler Annette Dixon Team Leader at ICR: Sarbani Chakraborty Armin Fidler ICR Primary Author: Sarbani Chakraborty 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: S Sustainability: L Institutional Development Impact: M Bank Performance: S Borrower Performance: S QAG (if available) ICR Quality at Entry: S S Project at Risk at Any Time: No 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The project was Phase I of a two-phase APL designed to sustain the performance of the health system, in the face of adverse demographic and epidemiological trends, increases in health resource prices, and rising public and health provider expectations ­ while maintaining long-term fiscal sustainability. The specific objectives of Phase I of the Program were: (i) the Ministry of Health (MOH) provides health purchasing priorities to the Health Insurance Institute of Slovenia (HIIS), on evidence-based and health status principles, based upon appropriately processed health information data by the Institute of Public Health; and (ii) the participating health providers in the system pilot, including the University Medical Center of Ljubljana (UMCL), adopt improved business and case management practices, based on purchasing incentives from the Health Insurance Institute of Slovenia and updated practice norms issued by the MOH. 3.2 Revised Objective: The project development objectives were not revised during implementation. 3.3 Original Components: The total project costs for the two-phase APL were estimated at US$29.4 million of which US$12.9 million were loan funds and the remainder were government and other sources of funds. The total estimated cost of Phase I was US$13.30 million, of which US$9.50 million (71 percent) were loan funds. The project consisted of the following four components: Component 1: Health Policy Support (US$2.2 million total costs) The objective of this component was to support the working groups to analyze the existing legal, technical and institutional situation, technically develop and weigh reform options, assess international experience and best practice, as well as revise information, legal and other requirements to devise, launch and sustain the proposed reform elements. The component financed technical assistance and related knowledge-management and training support for four thematic working groups, including the participants from the MOH, the HIIS, the IPH, and the UMCL (and other providers) and the Ministry of Finance (MOF). Each working group addressed an aspect of health sector reform, based on the policy agenda of the Government. The groups were expected to work closely with ongoing projects such as the European Union (EU) Phase financed CONSENSUS project on Social Security Reform and the Quality of Care working group championed by the Medical Chamber of Slovenia (MCS). It was expected that the inter-institutional working groups would be guided by the approved policy agenda, put forward in the Policy Note for the APL Program - the White Paper (National Health Care Program of the Republic of Slovenia by 2004) produced by the MOH for Parliamentary approval and the EU Accession Strategy. Component initiatives for Phase I of the program included: Revision of Payment Mechanisms and Reimbursement Models for Primary, Secondary and Tertiary Care: The purpose of this working group was to assess current implications of the prospective and capped budget model for hospitals and propose more effective and quality oriented alternatives, based on international experience and best practice. A new reimbursement model was expected to achieve effective and efficient resource management at all service levels and create an incentive framework that would enhance productivity, resolution and quality at all service levels. The task force was expected to model and pilot test alternatives, with the participation of the UMCL and other pilot hospitals, and in close collaboration with the CONSENSUS project. The final goal was to work with international experts to develop, pilot test, adapt as necessary, and introduce a suitable model for Slovenia through the national roll-out supported by Phase II of the Program. - 2 - Definition and Introduction of Health Care Capacity Management Methodology and Practices in the Service Network: The objective of this working group was to support the strengthening of health care management, on both the institutional and national levels. Specifically, the task force was expected to focus on: (i) design and implementation of appropriate management training programs for the health sector, in particular within a new incentive framework for efficient management of hospitals and health centers; (ii) definition and introduction of appropriate models for planning within the public health network; (iii) definition and selection of models for system monitoring and evaluation to improve organizational planning and institutional management; (iv) definition and introduction of training programs for budget and deliverables negotiations in line with the introduction of new reimbursement systems; and (v) initiation of the preparatory legislative work to introduce hospital autonomy/corporatization. Improvement of Efficiency and Effectiveness in the Public Health Network and Introduction of Algorithms Linked to Evidence-based Medicine: The purpose of this working group was to define optimum resource allocation formulas for the public health network based on international experience. Expected outputs included: (i) agreements on specific indicators and benchmarks for efficiency and effectiveness that are internationally accepted and comparable; and (ii) a baseline survey on current performance levels of all service levels (PHC, district hospitals, tertiary care services). The task force was expected to define appropriate diagnostic protocols and establish treatment guidelines using evidence-based procedures depending on the care level and complexity; evaluate a pharmaceutical formulary to be used for HIIS reimbursement and linked to Diagnostic Related Groups (DRGs) and service levels. It was also expected to assess diagnostic, therapeutic and rehabilitation and prevention protocols that could be performed at the PHC level to strengthen the gate keeping function. This process was expected to facilitate the introduction of a unified classification of standard diagnostic and therapeutic procedures (algorithms), including defined human resource standards. The working group was expected to prepare proceedings for hospital accreditation standards and professional licensing policies. Project Component 2: Health Information Standards Formulation (US$ 0.8 million total costs) This objective of this component was to improve the management of the health sector, health institutions and health care services by establishing and promoting health information standards for clinical, economic, statistical, normative and other information types used in the sector. When the project was designed, health information was generated from disparate information systems (both within and from outside any single health institution). Given the heterogeneity of existing (and future) information system as well as the continuous process of change occurring in such systems, common information standards needed to be developed in order to facilitate communication among sector participants. Such common information standards needed to be established at a number of logical/conceptual levels. The component supported the adaptation of international standards and frameworks to country-specific needs. This component expected to exploit, among other international standards, the Health Information Systems Architecture (HISA), as devised by the Technical Committee 251 (TC251) of the European Center for Standards (CEN) which covers: (a) Subject of Care; (b) Health Characteristics; (c) Health Care Activities; (d) Health Care Resources; and (e) Health Information Systems Authorization (e.g. access, security, privacy, etc). Component 3: Health Information Systems Implementation (US$7.5 million total costs) Based on the work under the Health Policy Reform Component and under the Health Information Standards Formulation Component, the objective of this third component was to implement health information systems in Slovene health sector institutions and tie those institution-specific systems together in a nationally integrated health information system. Technologically, the objective of the component was - 3 - the implementation of standards-based information "brokers." At the national level, the component aimed to establish a National Health Information Clearinghouse (NHIC) (modeled on the clearinghouse functions employed in financial systems). The NHIC would serve as the central "hub" for the inter-agency exchange of health related information (clinical, economic, statistical, normative, etc.). As a single institutional point of focus and intermediary exchange of information, the NHIC was expected to reduce the complexity of designing, implementing and maintaining all the necessary bilateral links among health sector entities. 3.4 Revised Components: The original project components were never revised. 3.5 Quality at Entry: Quality at entry is rated satisfactory. The project underwent a Quality Enhancement Review (QER) in 1999. Since this was a QER rather than a formal Quality Assurance Group (QAG) Review, there are no ratings available. The main comments of the QER panel were as follows: (i) there was a need to further sharpen the project development objectives and strengthen the indicators to measure project results; (ii) although the project identified the governance arrangements for the health policy function, there was a need to make these institutional arrangements more explicit, for example, by strengthening the role of the MOH vis-à-vis other institutions (HIIS, IPH); and (iii) the project documents needed to describe the existing data systems in the country, what these systems were collecting, and the quality and use of the data. The Program was relevant for the country at the time it was designed. Slovenia, a small country with a population of about 2 million and a GDP per capita of US$14,000 (PPP dollars, 1999) had a better health system than most other transition economies. Health indicators had been steadily improving and were almost approaching EU averages. The fiscal situation in the health sector was in balance, although Slovenia spent 8-9 percent of GDP on health care (1999 data), which was one of the highest among transitional economies and about the level of expenditure in most EU countries. Nonetheless, problems were beginning to emerge in the system, and there was growing concern among a few policy-makers (Ministry of Finance, Ministry of Health) that, as the country continued its economic development and with the availability of more information, and more health provider and consumer demand for expensive medical technology and other medical inputs, the situation would fuel rapidly growing costs in the health sector. In the future, the Slovenian health system, like other EU health systems, would be facing the problem of rapidly rising costs in the health sector. The country was already looking ahead towards EU accession and recognized there was a need for the health system to begin to address these issues before they became major problems. There were also concerns regarding the quality of health services and there was an overall vision to establish a strong preventive and primary health care sector, cost-effective secondary and tertiary care, and continuity of care across all levels of care. To realize this vision, there was a need to formulate an operational strategy for health sector development and identify and implement critical areas for health system improvements. The lack of a strong information base for evidence-based policy-making and the need to strengthen the role of the Ministry of Health and other institutions in their core functions were recognized as key problems that the Program needed to address. The Program was consistent with the 1997 Country Assistance Strategy (CAS) which centered on four key development objectives for achieving high, sustainable growth as the country continued on the path towards EU accession: (i) restructuring public finances; (ii) completing the economy's systemic transformation; (iii) ensuring social sustainability; and (iv) achieving environmentally sustainable growth. To help achieve - 4 - these CAS development objectives, the Bank agreed with the client to support: (a) as part of the state asset management program, identification and implementation of improved ownership, management and institutional structures for health services; and (b) health and social security finance reform to ensure a sustainable financing base, quality and accessibility standards, and cost-effectiveness. Program preparation was streamlined with the identification and delivery of high quality and high value technical products that helped further shape the client's ideas and contributed to project design. For example, the Australian Health Insurance Commission (AHIC) carried out a situational analysis and long-term fiscal sustainability study of the health sector. Together with the World Bank sector paper of 1998, this study helped shape the design of the Program. The Bank was highly responsive to client expectations and designed a flexible and adaptable project with key benchmark indicators as triggers. The choice of lending instrument (Adaptable Program of Lending or APL) was appropriate, given the fact that the MOH had strong vision and goals for the health sector. In addition, the APL provided the flexibility that was needed to develop the reform strategy. There was also sufficient client institutional capacity which meant that the client could mould the project design to achieve the short and medium-term objectives of the health reforms, without losing sight of the long-term objectives. The monitoring and evaluation indicators were realistic and consistent with the project development objectives. The only weakness in project design was the absence of a component on communications and building stakeholder support for the health reforms. Given the socio-cultural environment in Slovenia, with strong stakeholder views and general resistance to change when no serious problems in the health sector were visible, a strong communications component would have helped market the reforms among stakeholders and gain their support. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: Achievement of project development objectives is satisfactory. The original project design included three performance indicators to measure the achievement of the project development objectives. Performance Indicator # 1: Refined payment system, in particular, hospital reimbursement model by the HIIS, pilot tested, evaluated and implemented by 2003. The achievement of this PDO is highly satisfactory. The original project design envisaged implementation of revised hospital reimbursement mechanisms in three pilot hospitals. However, project implementation went well beyond this original objective and currently, all 19 acute care hospitals in Slovenia are included in the refined payment system. The system is being incrementally tested, evaluated and rolled out. In 2003, the refined payment system (mainly Diagnostic Related Groups or DRGs for hospital care) covered only 10 percent of the hospitals' total budget. In 2004, a new payment model was introduced with 10 budget pools (e.g., patient care, non-acute care and new tertiary model), but with only 1 percent of budget that could be lost on a yearly basis according to the new model of DRGs. The idea behind phased introduction of DRGs in the determination of the resource allocation formula for hospitals was based on implementation experience, i.e., that more time was needed to refine hospital product costing and the implementation of DRGs could be used, as a first step, to obtain better cost data at the hospital level. Therefore, the first phase objective behind the implementation of DRGs in Slovenia was much more on improving data collection and the use of this data to make purchasing decisions. This contributed to strengthening the capacity of the Health Insurance Fund in purchasing hospital services, and ensuring transparency and accountability in resource allocation decisions between the health insurance fund and providers, while also giving time to providers to understand the DRG system. An evaluation of the - 5 - implementation of DRGs was carried out although it was not able to show any results on hospital throughputs. This was because the implementation was very recent and therefore, the level of impact on resource allocation was still low. Performance Indicator # 2: Health care capacity management method adopted and introduced in participating institutions, based on training, agreements on definitions and standards by 2003. Moderately Satisfactory. By the end of the project, a health care capacity management method was developed and proposed for introduction in two participating institutions (University of Ljubljana and Maribor). Performance Indicator # 3: Health Information Systems Architecture Standards (HISA) introduced and adopted, following guidelines of Technical Committee 251 (TC251) of the European Committee for Standardization (CEN) and the EU data protection, by 2003. Satisfactory. Development of the first version of HISA standard classifications and model has been completed, and a process is in place for establishment of data standards and minimum datasets. Consensus has also been reached among the stakeholders on the functions of a National Health Informatics Institute agency which will be responsible for maintaining a secure environment for effective national health information exchange among the key stakeholders. 4.2 Outputs by components: Component 1: Health Policy Support (Total cost Euro 2.16 million) This component of the project disbursed Euro 1.95 million by the end of the project (90 percent disbursement). One of the consultancy contracts was cancelled, otherwise disbursement would have been in the range of 98-99 percent. The objective of this component was to finance technical assistance and related knowledge-management and training for four thematic working groups consisting of members from participating institutions (IPH, HIIS, UMCL, MOF and other providers). The implementation of some of the sub-components (e.g., provider payment systems and quality of care) was highly satisfactory while others were less successful (e.g., management capacity training). There was good participation of stakeholders in the working groups. Nonetheless, despite the efforts of the MOH and the evidence-base behind the reforms recommended in the White Paper and other policy documents, key institutions such as IPH and HIIS were still resistant to many of the changes suggested by the MOH and did not fully take these ideas on board. This is partly attributable to the lack of a good communications and marketing strategy for the reforms and the fact that the project aimed at strengthening the role of the Ministry of Health (MOH) in policy-making and priority-setting. This was problematic for the HIIS and IPH which were not used to the MOH playing a leadership role in the health sector. Health Policy Formulation: Under the component, support was given to the MOH in sharing international evidence and comparisons in literature for the "White Paper" which defined the vision for health sector development in Slovenia and specific policy directions that would be taken in the coming years to sustain health systems performance in Slovenia. Project staff were involved in drafting the White Paper, especially in the areas of health insurance, DRG-based payments, outpatient classification and payment auditing, and coordinated care. The analytical work undertaken involved intensive review of OECD country experience in improving health systems performance, including specific areas such as provider payment systems, quality of care, and health financing mechanisms. The Project encouraged and facilitated comparisons between Slovenian and other health care systems. In almost all circumstances, the approaches being taken - 6 - elsewhere were considered in a balanced way and were accepted or rejected on the basis of careful analysis. It stimulated a large number of people to become involved in the Project. Most of the changes made during the Project period were consistent with the weight of evidence. It should also be noted that the debate over health reform took place during the first phase of Slovenia's independence. This independence brought opportunities to consider quite different approaches to health care financing and delivery, and relatively few health professionals were experienced in either the technical or the socio-political processes of reform. Relatively uninformed views were strongly held on many complicated and interconnected aspects such as privatisation, governance, and clinical autonomy and control. Revision of payment mechanisms and reimbursement models for primary, secondary and tertiary care: Under the project, technical work was undertaken to identify the most appropriate payments mechanisms, especially for acute hospital care. The project team evaluated provider payments systems implementation in other countries and through a process of discussion with various stakeholders, it was decided that a DRG system would be adopted. Initially, the idea was to only implement it in three hospitals followed by a phased roll-out program covering all acute care hospitals. However, based on broad discussion (hospital managers and other key players in health care system), it was decided to implement DRGs for acute inpatient care, with a completely new model with 10 budget pools for hospitals throughout the country. Under the project, a DRG classification system was developed and applied to hospital budgets (2003). Currently, DRGs are applied in total in the hospital budget together with 10 budget pools, but with incentives that are incorporated in the Special Agreement for hospitals, including the agreement that there will not be a major redistribution of funds during the year. This phased approach is appropriate since it gives time for hospitals to adjust to the new system as well as to build capacity. Mistakes are also more likely in the early stages of implementation and the risk of impacting service delivery is reduced through phased implementation. Continued efforts will be needed to refine the DRG implementation system in hospitals, including the identification of alternative arrangements for coding, since doctors are currently responsible for this and international experience shows that this is not a sustainable arrangement. In some countries, trained coders are responsible for this task, although this does raise the administrative costs of implementing DRGs and has implications for hospital staffing numbers. It may be possible to use appropriate software that would help doctors undertake this task without adding to their administrative burden. In addition to the implementation of DRGs, preliminary technical work was undertaken on developing payment systems that did not just pay by service type (inpatient, outpatient and tertiary services), but across the spectrum of care to guarantee cost-effective use of each level of care. This is critical for sustainable cost containment and ensuring continuous quality improvement. Recommendations were made for about 10 categories of hospital-based services (major service types) and about five other types of care outside hospitals (together with cross-setting categories termed episode management units of EMU). Background technical papers supporting these ideas were completed under the project. For intensive care, the project group developed a study protocol for a pilot project. This was submitted to the MOH for approval and implementation is expected to begin in 2004-05. For outpatient care, the project team worked on defining outpatient clinic services. Under this category, the project team explored and identified favorable technical options for paying for outpatient clinic services according to episode of care and period of care. Through exploring the various ideas, it was determined by the project team that paying on the basis of period of care was more appropriate, since patients with chronic illnesses (or other types of patients that need care from time to time over a long period such as a year) would be better classified under such a system. The project team also conducted technical work on payment mechanisms for palliative care. The team looked at good practices from other countries and determined short-term and long-term changes that would - 7 - be needed to ensure cost-effective delivery of palliative care services. The team concluded that to be most effective, palliative care must be integrated into a program of care that would be defined according to the particular setting. The team prepared a protocol for the piloting of an integrated care model in three hospitals together with respective primary care services. It was proposed that the pilot would be conducted in the last half of 2004 with a view to implementing it nationally in 2005. Technical work was initiated on payment mechanisms for tertiary severe, research and teaching services. These areas are perhaps one of the most complicated to address under payment mechanisms, but satisfactory progress has been made. Definitions have been drafted and a survey of teaching and research costs has been initiated. Data have been collected from the Clinical Center, Golnik Hospital, Maribor General Hospital, the Psychiatric Clinic, and the Institute for Rehabilitation and the Oncology Institute, and analysis of the data was completed in early 2004. Initial work has also been initiated on defining payment mechanisms for rehabilitation, and an expert group has been established and has worked on definitions. In addition, the project team has undertaken technical work on other critical areas such as refinement of methods for budget allocations (needs-based funding), including a draft report on a regional allocation model linking the method of risk-adjusted capitations payments to general practitioners (GPs). Work was also undertaken on discharge planning in acute care hospitals and an informal survey was undertaken. A discussion paper was developed and distributed for comments in September 2003. A position paper on primary health care (PHC) in the context of hospital financing reforms was also developed for discussion by the expert group. Sub-Component 2: Define and Introduce Health Care Management Methodology and Practices in the Service Network: This sub-component was important because it aimed at setting a foundation for continued management training and practices. It was defined in general terms during project design and, as the project evolved, the MOH focused this sub-component to support the ongoing work, especially on provider payments systems and quality. Under this sub-component, proposals for the development of a management education curriculum and educational program based on needs assessment, including an overview of international management standards in health care, were developed. Consultants hired under this sub-component conducted training workshops to test a number of management training modules. Health management training programs were developed and proposed for establishment in the University of Ljubljana (Department of Economics) and in Maribor. Improvement of Efficiency and Effectiveness in the Public Health Networks and the Introduction of Algorithms linked to Evidence-Based Medicine. The implementation of this sub-component was highly satisfactory. Technical work and implementation of improved quality of care practices was supported under the project. A critical element of the quality of care work, as in the case of payment mechanisms, was underlying vision. The MOH and the project team identified quality improvement as a critical medium- to long-term goal of the health system to ensure cost-effectiveness, patient and provider satisfaction as well as fiscal sustainability. Another dimension of the work was the recognition that sustainable implementation of quality improvement requires a multi-pronged and system-wide approach, i.e., the development of clinical guidelines and protocols, and the integration of incentives for quality under provider payment mechanisms, as well as support for the development of quality assurance mechanisms at the facility level. Ultimately, it is important to empower hospitals to work in teams to implement better quality of care. The project provided support on these fronts and helped to integrate the efforts. Under the project, a methodology document (Slovene Guidelines Manual) was produced and subsequently distributed in both Slovene and English. The Manual was presented and discussed in several formal and informal settings and training sessions with respect to methods of guideline development and use. The Slovene Guideline Group, supported under the project, is a part of the Guidelines International Network (GIN). The first clinical - 8 - practice guideline (CPG) was developed for colorectal cancer. The final version of this has already been issued. A second CPG concerning secondary prevention and rehabilitation of coronary heart disease (CHD) has been developed. The main remaining issue is to find a suitable institutionalization process for sustaining the development of evidence-based guidelines on a continuous basis. The current discussions center around the development of the legal and financial aspects of a new entity - a quality network or quality agency - which would support this process on a sustainable basis. The MOH has agreed in principle that such an agency needs to be established. Development of clinical pathways was initiated in three pilot hospitals in 2002. Other hospitals became interested in this work during the following 12 months, and by early 2004, more than half the hospitals in the country had initiated work on clinical pathways. By the end of 2003, Jesenice Hospital was using pathways for laparoscopic cholecystectomy and artificial abortion and had several other pathways in development (including total hip replacement, normal delivery and heart failure). Maribor Hospital had designed and tested two pathways (for atrial fillibration and fractured neck of femur) but only one is in use. Golnik Hospital has pathways for pneumonia and COPD pathways and has conducted an evaluation that shows several positive outcomes, including reduced length of stay. However, there have been difficulties in analyzing the variances and outcomes and therefore a proposal for a computer model to measure these better and allow hospital teams to use this for decision-making has been developed. Other hospitals (Valdoltra Hospital, Psychiatric Hospital in Polje, Postonjna, Murska Sobota and Novo Mesto) have also initiated work on clinical pathways. The Clinical Center has undertaken excellent related work, focusing on the idea of "integrated care path" rather than clinical pathway. The role of the project team has been to provide timely technical support to the hospitals in the implementation of clinical pathways. The project has supported the development of a detailed guide to the costing of clinical pathways. The next step is to get a commitment to using the pathways to pay providers. A detailed discussion paper was prepared in May 2003 that addressed all aspects of auditing of health care from licensing to payment claims processing. This was discussed and its main proposals were accepted. An important recommendation of this paper was the possibility of modification of claims auditing as undertaken by the HIIS. The HIIS has accepted the need for revision of current auditing practices and plans to introduce minor changes starting in 2004. A separate discussion paper was issued in late 2003 to cover accreditation. Various ideas have been discussed and the project team was responsible for developing the idea of a round of small-scale accreditation style reviews that would be included in the 2004 Hospital Agreement, so that each hospital would have practical experiences in the accreditation process as soon as practicable. The MOH agreed with this view and a document outlining the pilot of the clinical audit to be specified in hospital agreements was developed. Component 2: Health Information Standards Formulation (Euro 0.74 million) Disbursements on this component were 66 percent of estimated costs at appraisal. The objective of this component was to establish health information standards, for clinical, economic, statistical, normative and other types of information. This component aimed to support the development of standards that would comply with the Health Information Systems Architecture (HISA) as devised by the Technical Committee of the European Center for Standards. The work under this component was delayed since the Ministry of Health (MOH) wanted to see clear synergies with the technical work completed under Component 1. Nonetheless, once the DRG and clinical quality of care work under Component 1 was underway, there was substantial progress under this component. Under the project, health information standards consistent with HISA were developed and adopted by the Ministry of Health. In addition, work on "harmonization of - 9 - hospital episodes" and a methodological manual on data collection on hospitalizations in 2002 was completed. Component 3: Health Information Systems Implementation (Euro 7.42 million) The component had very low disbursements (only 7 percent of original estimated costs at appraisal). The Borrower decided to cancel the Bank portion of funds for this component since it expected to spend its own funds for this component. However, Borrower spending on this component was also very low. Under the original project design, Component 3 was expected to implement a national clearinghouse. This was fashioned after the concept of an integrated financial management information system (IFMIS) often used by Ministries of Finance to manage public financial management. The implementation of a clearinghouse was expected to streamline the function of data sharing and use. At the design stage, there was agreement among the key players that an integrated information system was needed. However, not enough attention was paid during the design stage to eliciting stakeholder participation as well as identifying the legal changes required to implement an integrated HMIS. The MOH realized soon after the project began implementation that this component could not be implemented as originally designed. Also, as the project evolved, Component I became the most important and the MOH wished to re-design Components 2 and 3 to reflect the evolution of the policy and program framework under Component. Also, it was soon realized that given the existing legal and institutional framework (e.g. IPH has the legal mandate to collect and analyze health outcomes data but other institutions do not have access to this), the implementation of a full clearinghouse concept would not work. Therefore, it was finally decided, in collaboration with the World Bank and other stakeholders that Component 3 would be split into two parts. Part 1 would directly support the work under provider payments systems (DRG implementation) and quality of care and Part II would continue to clarify jointly with stakeholders the original idea of the shape that an integrated HMIS system might take based on the Slovene health system. Finally, in 2003, in the context of a World Bank supervision mission and with the support of a World Bank-financed management information systems specialist, it was agreed that the concept of an information clearinghouse would be implemented only partially. The emphasis would be on developing a "data management center." A particularly important role of the center would be to establish standards and procedures to protect data security and confidentiality, as well as improve accuracy and consistency of data. It was suggested that to emphasize its core functions, this entity might be provisionally named the "National Health Data Management Center" (NHDMC). By establishing efficient, secure and accurate data management mechanisms at the national level, the center would improve access to timely, reliable and secure data for all authorized health institutions, while reducing both the technical and administrative costs of data management and data redundancy within these client institutions. The proposed center would also perform the "clearinghouse" function, as originally intended, through the establishment of a "star network" collection and routing of data, replacing the "point-to-point" network. The center would initially apply this revised institutional design to support the implementation of DRGs and quality of care, specifically, functions such as enabling the Health Insurance Institute to review DRG-based cost data, and to link these cost data to utilization and quality indicators. This improved data management would provide a more informed basis for decisions on funding requirements, resource allocation, and contract negotiations. The functions of this agency were clarified under the project and a legal framework for the establishment of this agency has been developed but not implemented as yet. Component 4: Project Management (Euro 2.67 million) This component disbursed 73 percent of total allocated amounts. Part of the costs under this component - 10 - were for payments to working groups for participating in the design of the health reforms. 4.3 Net Present Value/Economic rate of return: Not applicable. 4.4 Financial rate of return: Not applicable 4.5 Institutional development impact: Modest. The project was conceived as Phase I of a 7-8 year program of health sector development and therefore was expected to focus on identifying key areas for health system improvements based on stakeholder participation and the testing of these improvements through pilots. Phase II was expected to implement the interventions throughout the health system. Therefore, Phase I was not expected to have any substantial institutional development impact. The project contributed to building the capacity of core staff of the Ministry of Health on policy-making, priority-setting, setting purchasing priorities and quality improvement. The project also supported institutional development of providers in the context of the provider payment systems and quality improvement components. The capacity of various stakeholders in the health system was also built through participation in various workshops and working groups. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: There were two external factors that affected the implementation and outcomes of the project. First, timely and full implementation of the management training sub-component of the project was negatively affected by the poor performance of the consulting firm hired for the job. Although the MOH asked the consulting firm to change its scope of work to improve alignment with the overall direction of the project, the consulting firm did not comply and the Government was forced to cancel the contract. The second issue involved delays in project effectiveness. The project was supposed to become effective in June 2000 but this was delayed until October 2000. This was due to national elections in the country. 5.2 Factors generally subject to government control: There was strong and consistent Ministry of Health leadership throughout the life of the project. This allowed the project to be clearly linked to the evolving policy needs and objectives of the Ministry of Health and other stakeholders. This leadership also ensured effective use of project funds. 5.3 Factors generally subject to implementing agency control: In the early years of project implementation, there were changes in project team leadership. Once this problem was resolved, project implementation pace picked up considerably. 5.4 Costs and financing: Final project cost, as well as procurement by categories are reported in Euros instead of US dollars (as reflected in Annex 2). - 11 - At project closure, approximately EURO 4.0 million had been disbursed, or about 45 percent of the original EURO 9.0 million Loan. On May 20, 2004, at the request of the Government, the Bank cancelled EURO 4.7 million from the EURO 9.0 million Loan. These funds were cancelled as a result of savings on technical assistance contracts (e.g, the management training contract was cancelled), and the decision of the Government late in the Project to pay from its own resources for the large information technology (IT) investment package that had been planned under the Project. Disbursements were slow throughout the life of the project and deviated from the planned disbursements at project design. For example, at mid-term (September 2002), the project had disbursed only 16 percent of total funds. In 2003, disbursements reached 30 percent. Disbursement lags were due to several factors: (i) project effectiveness was delayed due to national elections; (ii) approximately 50 percent of total project funds were allocated for Component 3. Disbursements on this component were dependent on progress in Components 1 and 2; and (iii) the design of Component 3 underwent changes during the life of the project with the Government finally delaying the decision to purchase significant amounts of hardware. 6. Sustainability 6.1 Rationale for sustainability rating: Overall, sustainability was rated likely on the basis of several criteria (financial, institutional, technical and political commitment). Fiscal sustainability of investments supported under the project is not a concern; for example, MOH's budget for 2005 includes allocations to cover the operating costs of the three Health Information Institutions established under the project. Technical sustainability is also not a concern since the core group of MOH and IPH staff whose capacity was built under the project are continuing in their respective positions and can continue to provide technical support to the HIIS and health providers. In addition, the new Minister of Health has indicated his intentions to continue the reforms outlined in the White Paper. Institutional sustainability shows a mixed picture, for the following reasons: (i) involvement of key institutions such as the HIIS was not very strong throughout the project, and the continuation and strengthening of provider payment reforms is clearly dependent on the involvement and interest of the HIIS, (ii) a clear plan and institutional framework for sustaining the quality improvement work, especially on clinical guidelines and pathways is lacking, (iii) health providers are clearly interested in the continuation of the reforms, especially quality improvement interventions. Since the beginning of the project, the authorities have been considering appropriate institutional arrangements for sustaining quality of care initiatives. However, decisions had not yet been made at project completion. Health providers are clearly interested in continuing the work on quality of care and supporting changes in hospital payments, however, this requires the continued strong support of the MOH and HIIS. At this point there is no confirmation that the medical chambers will carry forward the work on quality without the financial and institutional support of the MOH. For hospital payment reforms, in order to sustain and build upon the achievements of the project, strong implementation of DRGs, especially application to the calculation of the resource allocation formula is needed. This will require strong HIIS support. 6.2 Transition arrangement to regular operations: The project was originally conceived as a two-phase APL, but Slovenia has graduated from the World Bank and in 2003, the Government indicated that it did not wish to borrow for the second phase, but pay for it out of their own funds. Although the Government has changed and there is a new Minister, there are plans to continue Phase II, and the advisor to the Program has been appointed State Secretary of Health. Under the structure of the MOH, the State Secretary is a key person in reform implementation - 12 - 7. Bank and Borrower Performance Bank 7.1 Lending: Bank Performance (Lending): The Bank's performance during preparation was satisfactory. The Bank was a very important catalyst in encouraging the client to think about ways to operationalize its vision for the health sector. It also played a key role in focusing the attention of the client on fiscal sustainability issues through analytical work completed by the Australian Health Insurance Commission. The Bank consulted with a wide range of institutions, including the Ministry of Finance. The Bank was forward-looking in realizing that the path to achieving the vision of health sector reform in Slovenia was likely to go through some changes, and the client needed flexibility to define the path, within the scope of the agreed project development objectives. The project was also developed quickly which indicated responsiveness to client needs. Nonetheless, the quick preparation time may have negatively affected the design of Component 3, which was quite a complex component. More time would have allowed the Bank to better understand and incorporate into project design an appropriate institutional framework for a clearinghouse. It would have also allowed time for stakeholder consultations. The Bank also did not consider the inclusion of a communications sub-component to assist the MOH in disseminating and implementing health reforms. 7.2 Supervision: Bank supervision is rated satisfactory. The Bank worked closely with the Borrower to support its efforts in successfully implementing the project. Although there were several task managers over the course of the project implementation, the dialogue was maintained and the Bank was responsive and supportive of the directions that certain components of the project were taking. For example, it was realized quite early on in the project that the "clearinghouse" concept was not taking hold and there was a need to embed the health information components (2 and 3) more solidly in the two areas of focus in the project - DRGs and quality improvement. The Bank supported the Borrower in shaping these ideas and proposing ways in which the health information components could be implemented to establish close synergy with other components, as well as to ensure future institutional relevance and sustainability. Health care financing supervision and support was consistently very good. The Bank also played a key role as an objective information broker among the various stakeholders in the health system (particularly the MOH and HIIS). Given that there was a great deal of distrust among the various stakeholders, this role of the Bank was very important in Slovenia. Nonetheless, there was limited Bank supervision input into the quality of care sub-component, which over time, became one of the more important elements of the project. Also, the Bank did not actively try to address a potential sustainability problem generated by the limited participation of the HIIS and IPH in the project. 7.3 Overall Bank performance: Overall, Bank performance is rated satisfactory. The Bank was effective in sustaining good communication and continuing policy dialogue with the Government, in being extremely flexible to the needs of the country, and in carrying out effective supervision throughout most of the life of the project. The fact that the Ministry of Health asked the Bank to continue to support the MOH in an advisory capacity for the second phase of reforms is an important indication of good Bank performance during preparation and supervision. Borrower 7.4 Preparation: - 13 - Borrower performance on preparation is rated satisfactory. The key stakeholders (MOH, HIIS, IPH) were involved during the design stage and the borrower ensured that there was close consistency between the design of the project and its health sector strategy. 7.5 Government implementation performance: Borrower performance on implementation is rated satisfactory. The Ministry of Health played a key role in the implementation of the project and a wide range of beneficiary institutions and stakeholders were involved throughout the process. The MOH helped align project implementation to its evolving vision for health reforms. One of the weaknesses in Government implementation performance, perhaps, was the lack of a well designed communications strategy. This may have helped the MOH bring important stakeholders such as HIIS and IPH on board. 7.6 Implementing Agency: Implementation agency performance is rated highly satisfactory. This was the first Bank-financed project in Slovenia, and the implementing agency - the MOH - needed time to understand Bank rules and procedures. There was a change in the project director soon after project effectiveness, which affected the pace of implementation. Despite these constraints, the MOH managed to achieve significant results based on strong stakeholder participation. It also played a critical role in shaping the direction of the project components in line with the Ministry's vision of health sector development. Without the strong and active performance of the Project Management Unit, there was a risk that the project would have become a problem project and would not have achieved its outcomes. 7.7 Overall Borrower performance: Overall Borrower performance is rated satisfactory. 8. Lessons Learned Seeking Change Through Consensus and Participation: The project has generated an important momentum for change in the Slovene health system. Given that Slovenia was not really facing any serious crisis in the health sector at the time the project was designed and generally there was an impression among stakeholders that the Slovene health system is better than most other countries, the Ministry of Health had a difficult job in implementing any changes. Therefore, the evidence-based approach, which compared Slovenia with other EU and transition countries was needed. There was also a need to hold multiple workshops and discussions with various stakeholders in order to convince these groups that change was needed. Also, in Slovenia, the various stakeholders in the health systems are strong and hold their own ground. In this environment, discussion and deliberation to bring groups on board prior to reform implementation is needed. Seeking change through consensus also contributes to the sustainability of reform efforts. The impact of this approach on sustainability of reforms is clearly visible in provider uptake of hospital payment reforms and quality of care improvements. Nonetheless, the Ministry of Health would have benefitted from technical support on communication and marketing of health reforms. This may have speeded along implementation and helped bring on board difficult stakeholders such as the HIIS and IPH. Implementation of Improved Hospital Payment Systems Requires a Phased Approach: Slovenia is implementing important changes in the hospital payment system (case-based payment based on Diagnostic Related Groups or DRGs). Although there is relatively good institutional capacity in Slovenia (information systems, human capital), Slovenia had adopted a phased approach to implementing DRGs. There are many - 14 - benefits to this phased approach. It has allowed fine-tuning of health information systems and training to meet the needs of DRG implementation, and providers are getting used to the technical details of implementing DRGs. In most of the ECA countries, there are limited data on hospital costs and Slovenia is using the phased implementation of DRGs to obtain better hospital cost data and apply this for improved product costing for the DRGs. Although there are trade-offs in this approach, since it will be a few years before hospitals begin to effectively respond to the efficiency incentives generated by DRGs, this approach is less disruptive for the health care delivery system and allows focus on getting the basics right before full-scale implementation. It allows purchasers and providers to build their capacity in understanding the new systems and contributes to improved transparency and accountability in resource allocation negotiations between providers and payers. Flexibility in Design and Implementation: The implementation of health reforms is a complex and, oftentimes, politically contentious process, and the environment for health reforms can change during the duration of a project. Therefore, a "blueprint" approach to the design and implementation of health reform projects is not likely to result in successful outcomes in every context. The Slovenia Health Sector Management Project was designed as a flexible operation with clear end-points. In partnership with the Bank, the country was able to adapt the investments supported under the project to support key areas of health system improvement that were consistent with the policy priorities of the country and for which there was support among the key stakeholders. Integration of the Project Management and Professional Support Unit into the MOH: The Project Management and Professional Support Unit (PM-PS Unit) was designed as a part of the Ministry of Health. This allowed complete synergy between the work of the Ministry of Health and the PM-PS Unit and enabled project funds to be used effectively. This will also contribute to the sustainability of project interventions since the reforms are endorsed in MOH policy documents and legal frameworks. Design of Health Information Systems Components: Many Bank-financed projects, including those in the ECA Region, are supporting the development of sound information systems. Strong information systems that improve the quality, content and sharing of information can contribute to improved efficiency of the health systems, reduced administrative and transaction costs, and greater transparency and accountability in the health system. Nonetheless, as the Slovenia HSMP shows, the implementation of an integrated information system can be difficult in a context where stakeholders perceive information as a source of power and control and there is no unifying legal framework that supports such an integrated system. For example, in Slovenia, the Institute of Public Health is the only institution that is allowed, by law, to review health outcomes data. This also means that the HIF is not allowed to receive information on the DRG codes which generates additional administrative layers in implementing DRGs. Without IPH's full support on the reform issues supported under the project, including the integrated information systems, progress on this front is not possible. The important lesson to be drawn from this project experience is that, in cases where there are significant information components, more time should be devoted by the Borrower and the Bank to understanding the legal, institutional and political constraints to implementing integrated information systems and then using this information in the design of the information systems component. 9. Partner Comments (a) Borrower/implementing agency: - 15 - The key outputs, results and impact of the project were as follows: 1.1 Health care financing The Project facilitated the consideration of beneficial changes, mainly through its support for the process of consultation surrounding the White Paper. Project staff provided timely, objective, and carefully structured materials to support its drafting through to production of the associated legislation. The Strategy Document, as contained in Deliverable 22 {44}, made an important contribution in this regard. The process has set the agenda for ongoing reforms over the next decade, and ideas included in the many background documents are almost entirely appropriate to Slovenia's needs. The process of public consultation has been beneficial by itself. Health care will always be a contentious issue. The main contribution of the White Paper in this regard was that it presented good ideas that merited (and in fact resulted in) worthwhile public debate. There have been criticisms of both the content and the process of its development. Our view is that the content is admirable, and competing ideas (such as greater consumer choice of health insurance) have not been supported by convincing evidence by the proponents. In contrast, the content as determined by the Ministry have been well founded in evidence and logically developed to suit the Slovenian context. Nor does there seem to be good reason to question the process of development of the ideas for change. One argument has been that the Ministry failed adequately to invite and then seriously consider optional approaches, and that this led to an unnecessary degree of friction and disillusionment among knowledgeable parties in the health sector. This does not seem to be a plausible argument. Most of the criticism came from parties with a clear vested interest, and their willingness and ability to debate the options was generally low. The Ministry's responses were generally serious and fair, and its arguments seldom directly addressed by its critics. There was indeed a high degree of friction, but this was probably only partly related to the proposed changes themselves. Other factors included the degree to which many parties were inexperienced in both the ideas and ways of debating them, and the general political interest in health care in a pre-election period. In total, we believe the process of consultation was sensible, and led to a much wider understanding and support for the changes than might have been anticipated. This said, it might be worth considering minor changes in the process in future to overcome some of the problems that were not adequately anticipated on this occasion. For example, ideas for changes should probably be discussed in a more open way and for a longer period of time than was the case in 2003 and 2004. There should be the opportunity for opponents to develop and present their own ideas. It might eventuate that the views of others are sensible and can be incorporated in the design. Whether they are sensible or not, it is usually wise to allow sufficient time for all concerned parties to understand why. 1.2 The DRG classification This section should be read in association with the next, which concerns diagnosis and procedures classifications. They are strongly interconnected. - 16 - One basis for evaluation of the Slovenian experience is comparison with other countries. It is hard to ensure validity of comparison for many reasons -- such as differences in timing and intended purposes to be served by DRG implementation. For example, Slovenia implemented DRG reporting systems much more rapidly than Portugal. However, Portugal completed implementation more than ten years earlier than Slovenia. Not only was there little experience on which Portugal might draw, but its hospitals lacked the routine discharge abstracting systems that Slovenia had in 2003. The same may be said of Hungary, which also started many years earlier than Slovenia. However, the limited evidence suggests Slovenia was relatively successful. For example, it avoided the superficially attractive option of building a local version. It did not replicate the Australian error of becoming involved in a copyright disputed with a commercial company -- that led to a slow-down in innovation for five years until the ties could be broken. Slovenia was fortunate to have a more cohesive team than was the case in Germany. It therefore avoided some of the consequential delays, such as those resulting from disputes among several agencies that undertook procedure mapping. It seems that having a less cohesive team has been a constraint in Austria as well -- it has had to manage the preferences of each state as well as multiple insurance companies. Slovenia chose not to take a HCFA version simply because it did not overvalue the advantage of HCFA versions being in the public domain. The team was probably correct in concluding that the low initial costs would be more than outweighed by the cost of adaptation (including assignment software development, diagnosis and procedure mapping, and establishment of analytical routines to manage updates). Slovenia's experiences differ little from those of Singapore and New Zealand, which both decided to take Australian DRGs. Slovenia made the decision and implemented it more rapidly, but it had the advantage of learning from what had been done elsewhere. There are some similarities with experiences in Scandinavian countries, where significant advantages resulted from the sharing of development costs. In total, we set in place a process was set in place where people with a mix of experience, clinical intelligence and interest were free to brainstorm and argue. This was the main reason why implementation of DRG data production was completed in about six months at low cost -- and to the satisfaction of most parties. Although the first sets of DRG statistics contain many errors, there is a clear methodology for improvement and there are significant incentives for hospitals to apply it. It cannot be proved that the best variant was chosen. Indeed, it is possible that any of the leading variants would have been suitable: perhaps there are many different starting points that lead to the same endpoint. Moreover, there are many weaknesses that need to be alleviated in terms of coding, estimation of DRG cost relativities, and so on. However, it can be said with confidence that the journey has begun without the delays and friction that have often occurred elsewhere. The only significant implication for the future concerns the need to establish a process of periodic updating of the DRG classification. This is discussed in section 4.16 below. 1.3 The diagnosis and procedure classifications Although the DRG classification was selected and implemented with a high degree of speed and common sense, the same cannot be said of related matters such as implementation of the Slovenian versions of - 17 - ICD-10-AM diagnoses and procedures. There were many unfortunate delays, and in total activation of coding according to the new standards was more than a year behind the original (and apparently reasonable) timetable. Moreover, associated tools (such as a Slovene version of the alphabetic index) were not completed by the end of the Project. It is important to understand the causes, and two seem to have been the most important. First, there was a failure adequately to involve the main parties from the outset (and particularly the clinical associations) and therefore there was an inadequate degree of shared ownership. Second, there was a failure to ensure the commitment of IVZ to ensuring there were adequate coding tools to support the main purpose of the health care system ­ the delivery of good clinical care. IVZ continued to believe that other objectives were more important (such as the maintenance of statistical series). However, these were probably no more than symptoms. This is illustrated by the typical reaction of IVZ to problems of release of DRG data for use in budget setting: that there were legal barriers to making the data available. This was probably true, but if IVZ had been committed to teamwork to achieve the goal of better health care for Slovenians it would have tried to find solutions rather than simply deny the possibility of progress. Perhaps the best illustration of problems of organisational culture concerns the issue of inconsistencies between the new diagnosis classification and the reduced ICD-10 classification required for reporting of causes of death to WHO. IVZ, which has the primary responsibility for data standards, suggested the changeover should therefore be postponed. Project staff suggested that the problems might be easily resolvable. Australia uses the same ICD-10-AM diagnosis and procedure classifications as are proposed for implementation in Slovenia and yet Australia has no problems in meeting WHO's reporting requirements ­ it simply maps from the more complicated ICD-10-AM data to the simpler WHO ICD-10 data. Mapping tables were provided that are used by Australia to translate its detailed coding outputs into the simpler structure for reporting to WHO. Thus it would be necessary only for IVZ to incorporate those mapping tables into its system. In the event, IVZ rejected these and related ideas and consequently delayed implementation for a considerable period of time. The problems were obviously not technical, but rather the inability of concerned parties to share a vision for change and then recognise and address the organisational cultures where they represented barriers. Towards the end of the Project, the situation improved. For example, the Project Team was able to gain access to discharge data to support recalculation of the DRG costs weights. However, the underlying causes of the earlier difficulties have probably not yet been addressed to an adequate extent. It probably helped very little to continue to suggest technical answers. The root problem lies in a cultural unwillingness to want to solve the problems, and we therefore believe it would be useful in the near future for someone to present the Project Team's workshops on organisational culture for staff at IVZ and in other key agencies. We will comment further on this matter in section 4.19 below. 1.4 Budget setting (resource allocation) Slovenia is fortunate that a sensible process for budget determination has been in place for more than a decade, and that most parties are committed to making it work. However, there are weaknesses, and not all of them were resolved by the end of the Project. - 18 - The Project phase during which budget processes were redefined was satisfactory in many respects, in that there was a thorough and systematic review of the options and many clever people were involved in judging their merits. However, it was hard to translate design decisions into attributes of the annual contracts. Agreements that had been reached in principle would sometimes be disputed at a later stage, when vested interests became clear. Data that had been considered adequate were seen to have weaknesses when examined in detail. Disputes arose that were sometimes a reflection of misunderstandings and more often a consequence of mistrust. Having said this, there is no reason to believe the difficulties were any more acute than in similar countries where changes were being made. In total, the years 2002 to 2004 were characterised by rapid improvements based on good ideas and hard work. It would be unwise to assume that there were any simple and obvious ways of improving the process. It might have been sensible to review a wider range of issues (not only the technical issues of budget structure but also the processes of negotiation and decision making) at the start of the Project. However, this might not have achieved any significant gains and it would certainly have disrupted the process of budget finalisation. We suggest, however, that a review be undertaken in the latter half of 2004. We believe it will then be appropriate to reduce the rate of change and to consolidate the gains. If the pressure for innovation is reduced, there might be more likelihood that all parties are willing to turn their attention to process. If such a review is undertaken, we suggest it might address at least four matters. First, it would be worthwhile to reconsider the schedule for the negotiations. Second, there seems to be reason to reconsider the roles of the various parties in the negotiations, partly because the changing structure of the budget agreements has probably had an effect on roles. Third, it might be worth considering a much more formal process of definition of the expectations of each party at the start of the negotiations. Finally, it would be wise to ensure there is a formal evaluation of the negotiation process, and that its findings be made available in the public domain. 1.5 Management systems for other service types It would have helped if all the groups had been commissioned at an earlier stage in the Project. Perhaps more of the Project Team's resources should have been allocated to supporting them. As it was, a considerable amount of the Project Team's resources was diverted to the task at a time when other activities needed to be undertaken. However, in total the work done in this area has been remarkably successful in terms of involving experts from clinical and other health professionals in well-structured, purposeful, enthusiastic, and intelligent design work. In some respects this was one of the best investments made by the Project. In the near future, it would be useful to make available to concerned clinical experts the Project report titled Deliverable 6: classification and payment of services for other major service types {38}. This report starts with summary definitions of all the major service types ­ it is important to examine them as a set, because care is needed to ensure they are exhaustive and mutually exclusive. It then addresses problems and recommendations for solution. Deliverable 6 also covers major service types not yet being addressed adequately, such as general practice and home nursing (home care). - 19 - Over the next two or three years, there should be progressive implementation of ideas generated by the expert groups. The goal should be to ensure expert clinicians continue to play the major role in defining change with a dominant interest in encouraging and rewarding improvements in clinical practice. Expert clinical groups should be given the main responsibility for both continual updating of a long-term strategy and for designing, implementing, and evaluating short-term improvements. 1.6 Primary medical care The Project Team believes that many of the changes in primary medical care made since 1993 were appropriate. However, some significant weaknesses remain including perverse incentives for referrals and for unnecessary consultations to meet targets for contact-related payments. Several changes need to be considered. In particular, the targets for contacts might need to be revised (and probably eliminated for the most part) because the evidence from several countries suggests there are better ways of achieving the stated goals. For example, payments for performance of the style recently introduced in the UK and Croatia are worthy of careful consideration. There are many distinctive features, relative to current methods of payment for target numbers of contacts. For example, the attributes of performance for which additional payments may be made are based on clinical practice guidelines, may encourage either efficiency or outcome improvements, and may encourage more or fewer contacts depending on the circumstances, 1.7 Home care The Project Team's main recommendation is that there should be a shared long-term strategy. Home care services cannot be designed and funded without consideration of other parts of the health care system, and there must be a gradual process of change if undesirable side-effects are to be avoided. In the short term, a new payment method should be developed that takes account of a new way of describing and classifying patients. It should make use of a blend of time-based and per case payment, take account of dependence for activities of daily living (using an international standard ADL index), and contain strong financial incentives to manage patients in the most appropriate setting. 1.8 Cross-setting care: episode management units (EMUs) There have been elements of EMU-style contracting for several years in relation to a few types of health services. The Project had planned to expand the scope of care managed in this way, and to improve the precision and logic of the purchaser-provider contracts. It is unfortunate that these plans were not fully implemented mainly as a consequence of weaknesses in the consultant inputs. This does not mean the Project had no impact. Rather, it provided a greater degree of understanding of why and how EMU-style contracting should be extended ­ at both the policy and the operational levels. The challenge remains. We encourage the Ministry and its Partners to incorporate a plan to introduce additional EMU-style contracting in 2005, without undertaking any kind of piloting that might postpone implementation. We believe the best approach would be to select only one or two problem types (such as COPD or CVA) and make sure the details of the contracts are designed with great care. As in other areas, - 20 - multi-disciplinary teams should take the lead with regard to design. 1.9 Clinical work process control The main objective of refinement of payment methods is to encourage and reward improvements in clinical practice. If such improvements occur on the basis of careful design and evaluation, they will lead to continuing gains in terms of equity and cost-effectiveness. Any approach that fails to stimulate clinicians to improve the ways that they work is bound to have limited benefits at best. The key to continuous improvement in clinical practice is the clinical pathway: it is the practical manifestation of good design and routine control. It ensures that management of clinical work is driven by the way that clinicians behave on a day-to-day basis. Any form of external or periodic audit will be significantly less effective. In total, clinical work process control must remain the central concern in all work related to care provider contracting. This means, inter alia, that all parties need to have a shared view about what constitutes good clinical practice and how it can be promoted. 1.10 Clinical practice guidelines It is not easy to decide the best way to handle the task of development and maintenance of CPGs. Several similar countries have recently established new agencies and given them the responsibilities ­ typically together with related tasks such as clinical audit. However, the Project Team has emphasised the dangers, including the risk that creation of an external agency encourages practising clinicians to believe that quality is not longer their responsibility. In other words, new agencies may undermine basic principles of quality improvement. We believe that any decisions regarding the maintenance of a CPG process must take account of four fundamental ideas. First, the main objective must always be to encourage and reward improvements in clinical practice on the ground that are made as a matter of course by effective clinical teams. Second, there are serious risks of bureaucratisation of a process that needs to be innovative and consultative above all else. Third, the main problem is not a lack of knowledge about good clinical practice, but rather a lack of interest in applying it. The large majority of practising clinicians in Slovenia are capable of obtaining documents on good clinical practice through libraries or the Internet. Fourth, CPGs must always be closely associated with several other aspects of the system including methods of payment and the use of clinical pathways. If these four ideas are constantly borne in mind, several organisational arrangements could be equally effective. Considerations such as whether the agency is new or not, or whether it is government-owned or a private contractor, are secondary matters. 1.11 Clinical pathways In total, the work on pathways has been encouraging. It is always the case, in any health system, that there will be a mix of enthusiasm and opposition because pathway-based care undermines some long-established attitudes to clinical teamwork. One of the two main goals of the Project was simply to ensure it would be possible to point to some practical examples of enthusiastic implementation in Slovenia, and this goal has - 21 - clearly been attained. Progress on the other main goal ­ using pathways as the basis for payment ­ has been less satisfactorily achieved. The principles have been agreed (that the main component of contracts should be payments based on standard costs of pathways for high-volume case types) but this was not put into effect in 2004. This is not intended to be a criticism, but rather an encouragement of action in 2005. Not all of the ideas generated by the Project could be enacted immediately, and the building blocks exist for implementation in due course. Few health systems have made this much progress in such a short time. Clinical practice improvement presents systems problems that require systems solutions. The commitment of many clinical teams to pathway-based work is evident, and this is a major victory. However, this must be supported by financial and political commitments from policymaking and funding agencies to ensure that those care provider agencies that have made the effort obtain some reward. 1.12 Product costing There is no reason to change the strategy that was outlined in the Strategy Document: product costing should become a low-cost and routine activity supported by annual surveys of actual average costs and periodic standard costing exercises of high-volume case types. If this strategy is to be implemented in the near future, however, some practical steps need to be taken. The most important is a national strategy that indicates the kinds of data (including levels of accuracy and detail) and the intended uses. The debates about the best basis for determination of DRG cost weights could have been more productive if there had been a common understanding for the outset of the purposes that would be served. Once a strategy is endorsed, technical issues can be more adequately addressed including acceptance of a national standard method of product costing, the distribution of tools to facilitate its acceptance (such as costing guidelines and easy-to-use cost allocation software), and establishment of a process whereby a national body initiates the costing surveys on a regular basis. One of the important uses will presumably be for the purpose of updating of DRG cost weights. Others will include determination of product cost relativities for each major product type, and for classes of product types such as outpatient services and inpatient rehabilitation. 1.13 Auditing Auditing is a complicated matter in health care, and misunderstandings are common. Few countries have strategies that are well-designed and widely accepted, and this may fairly be said of Slovenia. Some of the essential ideas have been widely accepted, such as the recognition that poor care exists and is largely a consequence of poor clinical teamwork rather than the failures of individuals. The Celje Hospital review demonstrated this clearly to anyone who was prepared to learn. However, shared views on appropriate corrective actions are not yet well established. We firmly believe that the initiative for improvement should largely originate in the knowledge and commitment of clinical teams. For this to happen, there must be sophisticated learning processes whereby clinical teams are able to diagnose and correct problems of teamwork for themselves. There must also be clear messages from external agencies to this effect. For example, the strengthening of external auditing - 22 - gives inappropriate messages (especially if it is viewed to be largely punitive) whereas the provision of financial and other rewards for openness gives the right ones. The correct approach includes the methods used to encourage and reward control of inappropriate admissions in 2002 and 2003 as described in Deliverable 9 {39}, and to encourage and reward open auditing of clinical practice in 2004. There is no need to change direction in any significant way, but rather to strengthen those messages through ensuring that all parties understand there are no other effective ways. It would be unwise to give conflicting messages when the right message is clearly demonstrated by evidence from within Slovenia and around the world. If care providers establish their own internal auditing methods along the lines encouraged by the conditions of contract in 2002-03 (for inappropriate admissions) and 2004 (for clinical practice in general), all other external auditing requirements should be easily met by the use of by-product data. In particular, it should be possible to encourage ZZZS to move away from its current practice of claims auditing that is partly idiosyncratic, a deterrence to open admission of errors, and less than cost-effective. 1.14 Needs-based resource allocation Slovenia is fortunate in that the idea of needs-based funding has long been applied. However, hardly any serious attempts have been made to refine and extend it, and a major review is long overdue. There are some reasonable excuses. Most important, needs-based funding works best if all services are funded from a regional budget. The Ministry of Health correctly concluded that such a redesign should take place, rather than making minor adjustments within the current model (which splits hospital from community services funding at the national level). Fortunately the Ministry accepted in principle the idea of integrated regional health services in early 2004, and we hope that the necessary legislation will emerge before the end of the year. 1.15 Discharge planning Discharge planning processes are weak for the most part, and only a few clinical teams in a few hospitals have made satisfactory progress. Suggestions about methods as contained in materials generated by the Project are adequate for the purpose. What is needed in the immediate future is a set of actions that will encourage more clinical teams to use them. The main barrier is not a lack of suitable documentation that supports audit and improvement of discharge planning processes but rather a lack of good reasons for clinical teams to want to improve. If the motivation were there, clinicians would find their own solutions. The materials from the Project would simply help them to find better ways with a little more convenience. The best way to provide motivation is through actions described elsewhere in this report ­ and in particular through encouraging clinical teams to review the way that they work. Thus we need more workshops on clinical teamwork, more clinical pathways, and more incentives from policymakers and funders. Until there is progress in these respects, discharge planning will remain one more obvious problem with obvious solutions that few people seem able to resolve. - 23 - 1.16 Maintenance and refinement of clinical classifications Slovenia took the right approach in developing classification systems for payment and other purposes: providing technical support and incentives to clinicians so they would be willing and able to take the lead. This idea is particularly relevant to classification. Indeed, it is essential to a successful ongoing process of refinement. The remaining challenge now is to establish the structures and processes for continual review and upgrading of clinical classifications, and our discussion paper is a good place to start. All that is needed now is a collective commitment to make it work. As for other development activities, the key is to begin by talking openly about objectives, attitudes, accountability, and trust before moving on to the technical details of classification refinement. 1.17 Priority lists for non-urgent services (waiting lists) Slovenia has taken many useful steps in the right direction during the period of the Project. Most important has been recognition of the fact that waiting lists are essential features of any well-run health system, and that the challenge is to manage them better rather than claim they can be eliminated. This has led to excellent work on the design of better methods of management led by expert clinicians. No significant review of current directions is required. The challenge over the next few years is to expand the application of strategies and methods that are already being applied, and to ensure that more health professionals understand and actively support them. 1.18 Training in clinical coding The coder training was competently presented, but insufficient to ensure the stated gains were achieved. In part, this reflected the fact that so many changes were being implemented simultaneously with limited resources. However, there was a failure adequately to resolve differences of opinion about the right strategies for improving clinical coding. Some people incorrectly saw it as a largely administrative task (that should be addressed by improving logistics through the employment of specialized clinical coders, acquisition of code finder software, and so on). Only a few clinicians and lay managers have thus far recognized the correct view ­ that it is an essential component of good clinical practice (that should be addressed by improving clinical teamwork). At the time of writing, the conflicting views have not been adequately addressed. We therefore believe it would be wise to establish an ongoing process whereby common views will be formed. As is generally the case, the best guarantee of success will come from ensuring that clinicians have the time and the inclination to find the common ground and make sure it impacts on daily clinical work. 1.19 Training in organizational development and clinical teamwork The Project was successful in identifying and responding to a critical need for learning, and developed excellent materials that were put to good use as described in Deliverable 5 {45}. However, it failed - 24 - adequately to establish an ongoing process for promoting the ideas. Several actions should be taken in the near future to correct this, including ensuring that organisational and especially clinical culture aspects of health care become integral parts of all formal basis and continuing education programs in due course. 1.20 Management training The informal training activities as described in Deliverable 8 {36} were of variable quality. Moreover, they conflicted to some extent with the core strategy of the organizational behavior workshops since they focused on technical rather than social, professional, and organizational aspects. Some members of the Project Team made the elementary (but common) mistake of assuming that training is easy, and failed to make adequate preparations. We believe that ongoing training programs are required. However, we believe it is essential to incorporate them into existing formal programs. The main challenge is to cause the agencies responsible for those programs to adjust their content and method to suit the Ministry's views. For example, it is obvious that the emphasis should be on encouraging and helping health professionals to solve their own problems rather than presenting textbook answers to issues viewed largely as technical. Few of the current training programs fully embrace this idea. We suggest that the Ministry initiates an ongoing dialogue with the agencies responsible for the existing programs. The main goal should be to present to those agencies a cohesive message about their customers' needs ­ and to place a degree of customer pressure on them where it is appropriate. The best teachers will appreciate the Ministry's interest and support its views about program development. 1.21 Purchaser-provider contracting The Project did not result in any major changes in contracting, partly because improvements were already occurring on the ground through the routine processes of the annual budget cycle. It would have been useful to have a wide-ranging review that considered other quite different options, but the Ministry of Health correctly judged it was not a matter meriting a high priority. Inter alia, it would have added unnecessary complexity to the proposals contained in the White Paper. The reality is that models which appear to be quite different on paper can become very similar at the margins, as a consequence of the details. The degree of control exercised by the government over ZZZS is very high, and the `partnership' process of negotiation avoids most of the risks of unnecessary competition. We therefore continue to recommend that current arrangements be strengthened, and that there is no good reason to contemplate organisational changes in the foreseeable future. 1.22 Consumer empowerment This matter has been raised in many contexts within the Project ­ how clinical pathways can help in empowerment, why consumer representatives should be involved on a continuing basis in planning and evaluation groups, how complaints processes can be improved, and so on. In general, these ideas have been widely accepted and actions are under way in many parts of the Slovenian health care system to ensure they are applied. However, it is clear that many practising clinicians have not yet embraced a consumer focus on the ground, and that many consumers still do not understand their rights (or are willing to demand them). We therefore - 25 - recommend that all Parties embrace the idea that consumer empowerment should be a condition of contract in 2005 (as was the case for admission review in 2003 and clinical audit in 2004). The details of the contract should be easily defined. We suggest, however, that consumer representatives be given the leading role. In the longer term, a stable method of involving consumers will be needed. (b) Cofinanciers: (c) Other partners (NGOs/private sector): 10. Additional Information - 26 - Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate Ministry of Health, supported by appropriately Strategy in place for establishment of data Target achieved. The scope of the National processed health information from the standards, data security, minimum dataset Health Information Clearinghouse has been Institute of Public Health and other sources, and information exchange protocol among the reduced to reflect actual needs and use of establishes effective, evidence-based health key stakeholders, through the National Health existing capacities. Establishment of 3 purchasing priorities with the Health Information Clearinghouse. entities: Health Informatics Standardization Insurance Institute Board, and Center for Health Informatics. The participating health providers, including Improve business and case management Establishment of DRG-based payment the University Medical Center, adopt practices introduced and being tested in system in all 26 acute care hospitals in improved business and case management three pilot health care providers, with Slovenia. In 2004, 20% of hospitals using practices, based on improved purchasing preliminary results available for evaluation. DRG-based payment system as basis for incentives, updated practice norms, and hospital budgets. Management training to increased availability of relevant MIS. improve provider performance. Output Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate Introduction of new hospital payment system Pilot implementation of the DRG Target surpassed. DRG system established (DRG) in selected hospitals; preparation of reimbursement system to be completed in in all 19 acute care hospitals; in 20% of these standard clinical guidelines for priority health pilot hospitals, parallel with the legacy hospitals, DRG based system being used as issues; and upgrading of health services payment system. basis for hospital budgeting. management functions in the health providers in the pilot program. Consensus reached on the Health Development of version 1 HISA standard Target surpassed, with additional work on Information Standards Architecture and data classifications and model completed. developing minimum dataset and information standards to be adopted nationally for all standards to meet the DRG reporting system health and health services related activities. also completed. Establishment of a National Health Establishment of an organizational structure 3 national entities now established:Health Information Clearinghouse to support the to manage health informatics. Informatics Council; National Health management and information systems needs Informatics 3 Standardization Board; and of the key institutions in the health sector, Center for Health Informatics. Center still including MOH, IPH, HIIS and health needs to be staffed. providers. 1End of project - 27 - Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) Appraisal Actual/Latest Percentage of Estimate Estimate Appraisal Component US$ million US$ million Component 1: Health Policy Support 2.16 1.96 90 Component 2: Health Information Standards Formulation 0.74 0.49 66 Component 3: Health Information Systems Implementation 7.42 0.52 7 Component 4: Project Management and Professional 2.67 1.95 73 Support Total Baseline Cost 12.99 4.92 Physical Contingencies 0.00 0 Price Contingencies 0.39 0.00 0 Total Project Costs 13.38 4.92 Front-end fee 0.10 0.09 Total Financing Required 13.48 5.01 Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 0.00 0.00 0.15 0.15 (0.00) (0.00) (0.00) (0.00) (0.00) 2. Goods 3.77 0.00 0.73 0.00 4.50 (2.88) (0.00) (0.55) (0.00) (3.43) 3. Services 0.00 0.00 7.12 0.00 7.12 Consultancies (0.00) (0.00) (6.06) (0.00) (6.06) 4. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 5. Front-end Fee 0.00 0.00 0.01 0.00 0.01 (0.00) (0.00) (0.01) (0.00) (0.01) Re-current Cost 0.00 0.00 0.00 1.40 1.40 (0.00) (0.00) (0.00) (0.00) (0.00) Total 3.77 0.00 7.86 1.55 13.18 (2.88) (0.00) (6.62) (0.00) (9.50) Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 2. Goods 0.00 0.00 0.07 0.00 0.07 (0.00) (0.00) (0.05) (0.00) (0.05) - 28 - 3. Services 0.00 0.00 4.85 0.00 4.85 Consultancies (0.00) (0.00) (4.15) (0.00) (4.15) 4. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 5. Front-end Fee 0.00 0.00 0.09 0.00 0.09 (0.00) (0.00) (0.09) (0.00) (0.09) Re-current Cost 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) Total 0.00 0.00 5.01 0.00 5.01 (0.00) (0.00) (4.29) (0.00) (4.29) 1/Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. 2/Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Component (in US$ million equivalent) Percentage of Appraisal Component Appraisal Estimate Actual/Latest Estimate Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF. Component 1: Health 2.15 0.83 0.00 1.72 0.23 80.0 27.7 0.0 Policy Support Component 2: Health 0.46 0.27 0.00 0.40 0.09 87.0 33.3 0.0 Information Standards Formulation Component 3: Health 4.02 3.39 0.00 0.45 0.08 11.2 2.4 0.0 Information Systems Implementation Component 4: Project 2.24 0.43 0.00 1.63 0.32 72.8 74.4 0.0 Management and Professional Support Front-end Fee 0.09 - 29 - Annex 3. Economic Costs and Benefits - 30 - Annex 4. Bank Inputs (a) Missions: Stage of Project Cycle No. of Persons and Specialty Performance Rating (e.g. 2 Economists, 1 FMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation 05/1999 Appraisal/Negotiation 09/1999 Supervision 10/03/2000 1 OPERATIONS ANALYST (1) S S 10/03/2000 3 LEAD HEALTH SPECIALIST S S (1); HEALTH MIS (1); PROCUREMENT SPECIALIST (1) 02/14/2002 4 TEAM LEADER (1); S S PROCUREMENT SPECIALIST (1); HEALTH SPECIALIST (1); FMS (1) 02/14/2002 3 PROGRAM TEAM LEADER S S (1); PROCUREMENT SPECIALIST (1); HEALTH MNGT INFOR SPEC (1) 10/11/2002 5 TEAM LEADER (1); S S OPERATIONS OFFICER (1); PROCUREMENT OFFICER (1); HEALTH MIS CONSULTANT (1); FINANCIAL MANAGEMENT (1) 10/11/2003 2 TTL (1); CONSULTANT (1) S S 10/11/2003 3 OPERATIONS SPECIALIST S S (1); SECTOR MANAGER (1); CONSULTANT (1) ICR 1 1 TTL S HS (b) Staff: Stage of Project Cycle Actual/Latest Estimate No. Staff weeks US$ ('000) Identification/Preparation 43 146 Appraisal/Negotiation 29 121 Supervision 54 421 ICR 6 40 Total 132 728 - 31 - Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating Macro policies H SU M N NA Sector Policies H SU M N NA Physical H SU M N NA Financial H SU M N NA Institutional Development H SU M N NA Environmental H SU M N NA Social Poverty Reduction H SU M N NA Gender H SU M N NA Other (Please specify) H SU M N NA Private sector development H SU M N NA Public sector management H SU M N NA Other (Please specify) H SU M N NA - 32 - Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bank performance Rating Lending HS S U HU Supervision HS S U HU Overall HS S U HU 6.2 Borrower performance Rating Preparation HS S U HU Government implementation performance HS S U HU Implementation agency performance HS S U HU Overall HS S U HU - 33 - Annex 7. List of Supporting Documents - 34 - Annex 8. Beneficiary Survey Results The beneficiary survey included visits and in-depth interviews with three hospitals that had participated in the project. The hospitals were: (i) Jessenice, (ii) Golnik and (iii) Novo Mesto. Two out of the three hospitals are acute care while Golnik is a specialty hospital. The results of the discussions and visits are included below: Jessenice Hospital. Jessenice is a 287 bed acute care hospital based in the city of Jessenice, about an hour and a half drive from Ljubljana. It is a full service hospital consisting of four main inpatient departments (surgical, internal medicine, pediatric, gynecology and obstetrics). The surgical department is important and covers the entire Gorenjska region. The pediatric department also covers most of the Gorensjska region (except Skofja Loka). The departments of internal medicine, gynecology and obstetrics cover the northern part of Gorenjska. There is an intensive care unit at the hospital. Other departments at the hospital include transfusion medicine, day care hospital, dialysis, central sterilization unit and extended care section. There are 40 specialists' or sub specialists' clinics, organized in the outpatient department, as a part of the hospital. 11,500 patients are admitted to the hospital annually and the average duration of hospitalization was 6.4 days in the year 2002. Approximately, 3500 major surgical procedures, 800 major ambulatory procedures and 4000 small ambulatory procedures are performed yearly at the hospital. Altogether 91000 ambulatory examinations are done, 4600 different endoscopic procedures, 4000 ultrasound examinations and 4000 CT-scans. 6200 dialysis are performed annually, 6 patients have peritoneal dialysis. All the above mentioned work is undertaken by 530 employees; 64 doctors, 7 pharmacists, 73 certified nurses, 9 physiotherapists, 10 radiological engineers and 186 medical technicians. The hospital is one of the top hospitals in Slovenia in the fields of non-invasive cardiology, gastroenterology, traumatology and orthopedics. Beside the highly ranked expertise we want to be patients' friendly hospital as well. Jessenice hospital participated in the implementation of hospital reimbursement reforms, the management capacity-building sub-component and implementation of clinical pathways. Overall, all staff interviewed in the hospital felt that the reforms supported under the project were needed and had the potential to improve quality and efficiency of care. The hospital staff were particularly enthusiastic about the implementation of clinical pathways, since the staff felt that in the medium-term this had very good potential in improving morale and motivation of clinical teams and strengthen quality assurance at the hospital level. On DRGs, hospital staff felt they understood the rationale for DRGs and this would, in the medium-term, lead to better resource allocation of resources across hospitals since under the current system, there were few incentives for hospitals to compete on output of services. However, the staff felt that implementation of DRGs needed to be balanced with good quality control and improvement mechanisms. On the management training sub-component, hospital staff recognized the importance of this - 35 - sub-component but felt that the project could have done more to support management training for hospital staff. Staff mentioned that even after the project closed, there was commitment to continuing the reforms supported under the project. University Clinic of Respiratory and Allergic Diseases Golnik Golnik is a village situated in the foothills of Kriska gora mountain in the Gorenjsko region. The University Clinic of Respiratory and Allergic Diseases Golnik was founded in 1921 and during the first 29 years serviced mainly as center for treating tuberculosis. In the 1950s, the hospital began to specialize in pulmonary diseases. In the 1980s, it extended the specialty to allergic diseases. Currently, the University Clinic of Respiratory and Allergic Diseases, Golnik is a clinical, research and pedagogic insitution. It is a fully equipped clinic providing medical care for pulmonary and allergic diseases for Slovenia. It introduces new clinical developments and is a teaching center for medical students of The Faculty of Medicine of the Ljubljana University. It is is also a teaching institute for postgraduate education, mostly for doctors of internal medicine but also doctors who specialize in other areas of medicine, general practitioners, GPs, anaesthetists and surgeons, as well as for medical technicians and other laboratory workers. The clinic has 220 beds; it has 7000­8000 clinical admissions and 46000 outpatients' consultations a year. The average duration of the treatment stands at 10 days per admission. Top referral care of the clinic is closely related to its themes: asthma, COPD, lung cancer, tuberculosis, interstitial lung diseases, asbestosis, and other rare pulmonary diseases. The University Clinic units involved in clinical activities are as follows: · Specialised out-patients' departments that deal with TB, lung cancer, asthma, allergic and immunologic diseases. · Physiotherapy ward which mostly deals with respiratory physiotherapy and rehabilitation of the patients suffering from pulmonary diseases. · Respiratory bacteriology laboratory which ascertains the presence and researches sensitivity of bacteria to antibiotics. · Microbacteria laboratory which is a national micro bacteria laboratory. It deals with the identification of M. tuberculosis bacilli as well as other mycobacterium and researches their sensitivity to antituberculatics. Apart from that, the laboratory also researches the genotypes of individual strains of M. tuberculosis for epidemiological purposes. · Along with everyday work, laboratory for the research in the field of clinical biochemistry and hematology also researches individual elements of proteases system. · Cytology and pathology laboratory is the most advanced laboratory in the field of pulmonary diseases in Slovenia. The laboratory is in close cooperation with the Pathology Institute of The Faculty of Medicine in Ljubljana. · Recently developed immunology laboratory deals with diagnostics and research in the field of allergic and immunologic diseases. · The department for the research of pulmonary functions is the oldest laboratory, which, besides everyday work, deals with research in the field of diffusion and individual products in the exhaled air. It also examines different bronchially responsive stimuli. - 36 - · Laboratory for the examination of sleeping disorders. · Highly advanced CT technology in the radiology ward will enable more accurate diagnostics of respiratory diseases. Everyday work includes lungs X-raying, ultrasound research and diagnostic operations on respiratory and digestive organs. · Endoscopy ward is a highly advanced diagnostic and therapeutic department in the field of respiratory system endoscopy. It is a teaching place for all field specialists at home and abroad. The ward researches early detection of cancer with the help of fluoroscent bronchoscopy and brochial ultrasound. Its everyday work includes entire endoscopy of digestive organs. · Cardiology diagnostic ward deals with ultrasound diagnostics, cardiac catheterization and hindrance tests. · Epidemiology ward is a national tuberculosis register. Apart from that, it also deals with research in the field of respiratory diseases epidemiology. · The library is an up-to-date unit which assures strong support to clinical, research and pedagogic work. The clinic employs 45 doctors and 14 other medical workers with higher education. 20 of them are registered as researchers. Many of them have academic titles. Four of them have a part-time employment at a university and give lectures mostly at The Faculty of Medicine of the Ljubljana University. Specialists publish their studies and research in periodicals at home and abroad. Great emphasis is also given to nationwide medical awareness in the form of lectures in the media and popular articles in newspapers and magazines. The number of all published articles in the last five years goes well beyond 900 and includes every single and different sort of article published. Clinical, research and pedagogic activities are some of the conditions required in order to acquire (and preserve) the title of a clinic. Golnik, as in the case of the other hospitals, was involved in implementing hospital reimbursement reforms supported under the project as well as clinical pathways. It has successfully implemented a clinical pathway for COPD (Chronic Pulmonary Diseases), and this has already impacted the length of stay for COPD which is on the decline. Hospital staff noted that the next challenge is to implement clinical pathways that involved ambulatory care, so that patients received quality care at all levels of the health system. This hospital has also introduced DRGs for acute inpatient care, and there were no problems in the technical implementation of DRGs. The hospital staff mentioned, however, that one of the main objectives of DRGs is to improve resource allocation, i.e., move to an output-based payment system where hospitals have greater incentives to perform. Currently, since DRGs were applied to only 1 percent of the hospital budget, this was not the case. It was noted that there is some level of frustration among hospital staff that implementation of DRGs had not resulted in improved resource allocation. Hospital staff also mentioned that they only had limited control over the organization and management of hospital services, and therefore, greater attention was needed on how to support autonomy of hospitals to get the best out of the health reforms supported under the project. Overall, support from MOH on various aspects of reform implementation was highly rated. General Hospital Novo Mesto: Novo Mesto is a general hospital serving the population of Dolenska and Bela Krajina - 37 - (approximately 110,000 people). It has 385 beds and 10 beds for one-day treatment. The average length of stay for acute care is 6.23 days. It has 858 employees of which 126 are doctors and 403 nurses. It has approximately 19, 233 hospitalized patients a year. Novo Mesto has been mainly involved in the project in terms of implementation of DRGs. They did not experience any technical difficulties in implementation and hospital staff are currently able to do the coding quite effectively. The hospital staff felt, however, that the promise of improved resource allocation under DRGs had not happened and they felt that they were paid less than other hospitals. Hospital staff did not seem to have a good understanding of the fact that resource allocation based on DRGs was only being incrementally applied and that only 1 percent of their budget was affected by DRGs. - 38 - Annex 9. Stakeholder Workshop Results The results of the project were presented at a stakeholder workshop organized by the Ministry of Health and the European Observatory on Health Systems. The workshop included about fifty participants representing the following institutions: (i) Ministry of Health, (ii) HIIS, (iii) IPH, (iv) public and private health providers, (v) representatives from the medical chambers. The participants also presented the work that was carried out in their own institutions, either under the rubric of the project, or generally under the health reform program of the Government. Overall, there was consensus about the results of the project. However, some participants expressed concerns about ideas expressed in the White Paper such as clarification of the role of the primary health care provider in the Slovene health system versus specialists such as pediatricians and gynecologists. There was a lot of discussion about the levels managerial autonomy for health providers and implications for getting results under output-based provider payment systems. This resulted in more in-depth discussion of private provision of health services and the need for a clear policy regarding public and private facilities since there was no level playing field for these two types of providers. There was also a discussion of provider payment systems and the stakeholders generally endorsed the approach of the White Paper to move to payment systems that support continuity of care across different levels of care as well as provides incentives for providers to deliver goo quality health care services. - 39 - - 40 -