Document of The World Bank Report No: ICR3479 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-40470 IDA-49120) ON A CREDIT IN THE AMOUNT OF SDR 11.2 MILLION (US$17.0 MILLION EQUIVALENT) AND ADDITIONAL FINANCING OF SDR 6.5 MILLION (US$9.5 MILLION EQUIVALENT) TO THE REPUBLIC OF BOSNIA AND HERZEGOVINA FOR A HEALTH SECTOR ENHANCEMENT PROJECT June 23, 2015 Health, Nutrition and Population (GHNDR) South East Europe Country Unit Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 2015) Currency Unit = Konvertible Mark (KM) SDR 1.0 = US$ 1.3912 US$ 1.0 = KM 1.793 US$ 1.0 = Euro 0.91 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS AF Additional Financing IPH Institute of Public Health BHP Basic Health Project M&E Monitoring & Evaluation CAE Country Assistance Evaluation MoH Ministry of Health CAS Country Assistance Strategy OED Operations Evaluation Department CEB Council of Europe Development Bank OHR Office of the High Representative CIDA Canadian International Development Agency PAS Procurement Accredited Staff CPPR Country Portfolio Performance Review PCU Project Coordination Unit DPA Dayton Peace Agreement PHRD Policy and Human Resources Development ECA Europe and Central Asia PHC Primary Health Care EU European Union PRSP Poverty Reduction Strategy paper FBiH Federation of Bosnia and Herzegovina QER Quality Enhancement Review FM Family Medicine RF Results Framework FMR Financial Management Report RS Republika Srpska GDP Gross Domestic Product SWAp Sector Wide Approach GP General Practitioners TA Technical Assistance HIS Health Information System WB World Bank HSEP Health System Enhancement Project WHO World Health Organization IDA International Development Association Senior Global Practice Director: Timothy Grant Evans Country Director: Ellen A. Goldstein Practice Manager: Daniel Dulitzky Project Team Leader: Feng Zhao ICR Author: Lorena Kostallari BOSNIA AND HERZEGOVINA Health Sector Enhancement Project CONTENTS Data Sheet.................................................................................................. ...........................i A. Basic Information.............................................................................. ..............................i B. Key Dates........................................................................ .................................................i C. Ratings Summary............................................................ .................................................i D. Sector and Theme Codes.................................................................................................ii E. Bank Staff........................................................................................................................ii F. Results Framework Analysis...........................................................................................ii G. Ratings of Project Performance in ISRs.................... ..................................................xiii H. Restructuring................................................................................................................xiii I. Disbursement Graph.....................................................................................................xiv 1. Project Context, Development Objectives and Design. .................................................. 1 2. Key Factors Affecting Implementation and Outcomes .................................................. 7 3. Assessment of Outcomes .............................................................................................. 13 4. Assessment of Risk to Development Outcome ............................................................. 22 5. Assessment of Bank and Borrower Performance ......................................................... 22 6. Lessons Learned............................................................................................................ 25 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............... 26 Annex 1. Project Costs and Financing .............................................................................. 27 Annex 2. Outputs by Component...................................................................................... 29 Annex 3. Economic and Financial Analysis ..................................................................... 32 Annex 4. Bank Lending and Implementation Support/Supervision Processes................. 38 Annex 5. Beneficiary Survey Results ............................................................................... 40 Annex 6. Stakeholder Workshop Report and Results....................................................... 50 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 51 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 62 Annex 9. List of Supporting Documents .......................................................................... 63 MAP IBRD 33375R1 Data Sheet A. Basic Information Health Sector Country: Bosnia and Herzegovina Project Name: Enhancement Project Project ID: P088663 L/C/TF Number(s): IDA-40470, IDA-49120 ICR Date: 06/23/2015 ICR Type: Core ICR Lending Instrument: SIL Borrower: GOVERNMENT Original Total XDR 11.20M Disbursed Amount: XDR 17.39M Commitment: Revised Amount: XDR 17.63M Environmental Category: C Implementing Agencies: Ministry of Health in Federation of Bosnia and Herzegovina (FBiH) and Republika Srpska (RS) Cofinanciers and Other External Partners: Council of Europe Development Bank (CEB) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) 04/04/2006 (RS) Concept Review: 09/02/2004 Effectiveness: 12/04/2005 09/07/2006 (FBiH) Appraisal: 01/24/2005 Restructuring(s): 11/18/2010 Approval: 03/31/2005 Mid-term Review: 09/15/2008 09/15/2008 Closing: 12/15/2010 12/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Supervision No None time (Yes/No): (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 20 20 Health 55 55 Other social services 5 5 Tertiary education 20 20 Theme Code (as % of total Bank financing) Health system performance 100 100 E. Bank Staff Positions At ICR At Approval Vice President: Laura Tuck Shigeo Katsu Country Director: Ellen A. Goldstein Orsalia Kalantzopoulos Practice Manager/Manager: Daniel Dulitzky Armin H. Fidler Project Team Leader: Feng Zhao Betty Hanan ICR Team Leader: Lorena Kostallari ICR Primary Author: Lorena Kostallari F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The overall development objective of the project is to improve overall results for non- communicable diseases as measured by proxy indicators and: (i) enhance health system efficiency through restructuring and strengthening of primary health care along the family medicine model; and (ii) strengthen the policy making process through the development and implementation of a system for monitoring and evaluating health sector performance. ii Revised Project Development Objectives (as approved by original approving authority) The PDO was simplified at the time of the Additional Financing as follows: (i) expand and enhance the family medicine model of primary health care; (ii) build management capacity in the sector; and (iii) strengthen the policy making process through the development and implementation of a system for monitoring and evaluating sector performance. (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Health Sector Strategy prepared by respective Ministry of Health and ratified by the Indicator 1 : respective governments. The RS Primary Health Care Strategy and the The Federation and Value Federation RS operational plans quantitative or No strategy Strategy in place Primary Health are updated and Qualitative) Care Strategy adopted. operational plans are updated. Date achieved 03/21/2005 12/05/2010 12/30/2014 12/30/2014 Comments Target achieved. This outcome indicator was kept and further developed during the (incl. % revision of project RF and AF. achievement) Indicator 2 : Coverage of population through Family Medicine (FM teams). The coverage at 70% of national level Value population: reached 80.2% quantitative or FBiH:130.659; RS:86.270 NA Federation (95.5% for the RS Qualitative) 1500.000; RS and 69% for the 1250.000 FBiH). Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments Target achieved (surpassed). At project closure (December 2014) the coverage at (incl. % national level reached 80.2% (95.5% for the RS and 69% for the FBiH). achievement) Indicator 3 : Number of FM teams and Health Centers in the RS meeting accreditation standards. 260 FM teams meeting 200 FM teams and Value accreditation Accreditation process not 35% of centers quantitative or NA standards, 35% initiated. meeting the Qualitative) of Health standards. Centers meeting standards. Date achieved 03/20/2011 12/30/2010 12/30/2014 12/30/2014 Comments Target partially achieved. The indicator was introduced at project AF. (incl. % iii achievement) Systemic implementation of selected preventive services (for example, hypertension, Indicator 4 : breast cancer, prostate enlargement, smoking, etc.) in at least 2 major FBiH cantons. Systemic Active selected implementation of preventive selected preventive services Value No standard set for services in four implemented quantitative or preventive services, except NA cantons. FBiH systemically by Qualitative) for special annual programs. implemented result- all FM teams in based payment, based at least two on performance of major cantons. preventive services. Date achieved 03/20/2011 03/20/2011 12/30/2014 12/30/2014 Comments (incl. % Target achieved. The indicator was introduced at project AF. achievement) Indicator 5 : Health Management Center operational with annual budget and regular work plan. Centers for Value Centers of health Center of health Centers for health health quantitative or management in management fully management not initiated. management Qualitative) place. operational. operational. Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments Target achieved. The health management center in the Federation developed as a unit (incl. % within the Public Health Institute. The RS center is under the MoH. Both Centers are achievement) fully operational. Improved efficiency and equity in the planning, financing and regulations of the health Indicator 6 : service delivery system. Value quantitative or NA NA NA NA Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments The indicator is presented as per the PAD. No baseline or target value were define. (incl. % The indicator was dropped before project MTR. achievement) Indicator 7 : Improved access to quality primary health care throughout the country. Value quantitative or NA NA NA NA Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments The indicator is presented as per the PAD. Baseline value was not define. A survey (incl. % was proposed at project completion, but target value was also missing. The indicator achievement) was dropped before project MTR. Indicator 8 : Decreased number of unnecessary referrals to higher levels of care. Value quantitative or NA decrease of 10% NA NA Qualitative) Date achieved 03/20/2005 12/15/2010 12/30/2014 12/30/2014 iv Comments The indicator is presented as per the PAD. While no baseline value was defined the (incl. % target value was proposed as 10% decrease. The indicator was dropped before project achievement) MTR. Target not achieved. Indicator 9 : Number of primary care providers financed under new provider payment mechanisms. Value quantitative or NA NA NA NA Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments The indicator is presented as per the PAD. Baseline and target values were not define. (incl. % The indicator was dropped before project MTR. achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Indicator 1 : The number of Health Insurance Fund (HIF) staff trained in contracting. Value (quantitative 0 FBiH: 16; RS: 5 NA FBiH: 16; RS: 5 or Qualitative) Date achieved 03/30/2006 12/30/2008 12/30/2010 12/30/2008 Comments Target achieved. The indicator was introduced prior to project MTR, and was (incl. % monitored until project AF. achievement) Indicator 2 : Percentage of population registered with FM teams. Value (quantitative 22% of population 70% of population NA 70% registered or Qualitative) Date achieved 05/30/2004 06/30/2008 12/30/2010 06/30/2008 Comments (incl. % Target achieved. achievement) Government using feedback from stakeholders, including non-government sector for Indicator 3 : priority setting and resource allocation. Value (quantitative No report Report on time NA Report on time or Qualitative) Date achieved 03/20/2005 12/30/2007 12/30/2010 12/30/2007 Comments Target achieved. The indicator is presented as per PAD and monitored until 2007. (incl. % Target is considered achieved since MTEF preparation and reports on budgetary report achievement) allocations were produced on time (involving stakeholders’ consultations). Indicator 4 : Number of university teaching staff with Masters or PhD level in FM. Number of FM Value educators (quantitative 0 5 people FBiH 34; RS 6 (Cathedra or Qualitative) Faculty or v education center) with Master or PhD in family medicine. Target FBiH 30; RS 10. Date achieved 03/20/2005 12/30/2008 12/30/2014 12/30/2014 Comments Target partially achieved. The indicator was originally defined in PAD and revised (incl. % during project AF. achievement) Indicator 5 : Provision of preventive services increased by providers. Development of programs and monitoring tools for targeted non- Program monitoring Value 50% of population communicable tools for targeted (quantitative NA for 10 priority disease NCD preventive or Qualitative) diseases. preventive programs completed. programs (FBiH) and target programs completed. Date achieved 03/20/2005 12/30/2008 12/30/2014 12/30/2014 Comments Target achieved. The indicator, originally defined as per PAD, was revised at project (incl. % AF. achievement) Indicator 6 : The number of doctors and specialists who have completed PAT. Number of FM doctors operating in FM Value teams who (quantitative FBiH: 330; RS: 69 FBiH: 740; RS: 342 FBiH: 986; RS: 148 completed or Qualitative) PAT. Target FBiH: 940; RS: 342. Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments Target partially achieved. The indicator was originally defined prior project MTR, and (incl. % slightly revised during AF. Target value is surpassed in the Federation, but not fully achievement) achieved in the RS. Indicator 7 : The number of doctors who have completed FM specialization. Value FBiH: 523; RS: (quantitative FBiH: NA; RS: 83 FBiH: 234; RS: 116 FBiH: 436; RS: 227 250 or Qualitative) Date achieved 06/30/2005 12/30/2008 12/30/2014 12/30/2014 Comments Target partially achieved. The indicator was introduced before project MTR and (incl. % monitored also during project AF. achievement) vi Indicator 8 : The number of teams operating in family medicine. Value FBiH: 900; RS: (quantitative FBiH: 107; RS: 138 FBiH: 652; RS: 649 FBiH: 793; RS: 685 650 or Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments Target achieved. The indicator was introduced before project MTR and monitored also (incl. % during project AF. achievement) Indicator 9 : Number of nurses in operating FM teams that completed PAT. Value FBiH: 1437; RS: FBiH: 1900; FBiH: 2070; RS: (quantitative FBiH: 536; RS: 141 1031 RS: 1300 1273 or Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. achievement) Indicator 10 : Total number of health personal receiving training. Value FBiH: 3853; RS: FBiH: 4000; FBiH: 4587; RS: (quantitative 0 2055 RS: 3196 3425 or Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved (slightly surpassed). achievement) Indicator 11 : Heath buildings (physical sites) reconstructed or renovated. Value FBiH: 129; RS: (quantitative 0 FBiH: 79; RS: 56 FBiH: 385; RS: 409 110 or Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved (surpassed). achievement) Indicator 12 : Family Medicine ambulantas reconstructed or renovated. Value FBIH: 385; RS: (quantitative 0 FBIH: 335; RS: 146 FBiH: 455; RS: 213 275 or Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target partially achieved, (surpassed in FBiH but not achieved in the RS). achievement) Family Medicine Offices/ambulantas equipped with medical equipment and/or Indicator 13 : furniture. Value FBiH: 426; (quantitative 0 FBiH: 281; RS: 251 FBiH: 488; RS: 409 RS :400 or Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments Target achieved. (incl. % vii achievement) Indicator 14 : Family Medicine Offices/ambulantas equipped with IT. FBiH: Sarajevo and Mostar completed FBiH: Value FBiH: 3 (300 offices); RS: ambulantas of 3 (quantitative 0 municipalities; RS: 157 (by the project) municipalities; or Qualitative) 500 and 510 (financed by RS: 510 synergy projects including HSEP). Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target partially achieved. achievement) Performance based contracts signed with FM teams (directly or through municipalities) Indicator 15 : in the Federation. Value FBiH: 100 contracts (quantitative 0 75% FBiH: 100 signed and or Qualitative) implemented. Date achieved 03/20/2005 12/30/2008 12/30/2014 12/30/2014 Comments Target achieved. The indicator was originally phrased as: "performance based contracts (incl. % with health providers signed". The indicator was monitored during the original project achievement) and revised during AF preparation. Indicator 16 : Development of Federation level rules for health information standards. Value FBiH rules No Federation level rules FBiH rules in (quantitative NA completed and for HIS. place for HIS. or Qualitative) implemented. Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. This indicator was added at project AF. achievement) Support the implementation of PHC Information Systems (IS) in about 3 municipalities Indicator 17 : (Federation). Value No municipalities have 3 municipalities 18 municipalities in (quantitative NA functioning PHC/IS. in FBiH. FBiH. or Qualitative) Date achieved 12/30/2010 12/05/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved (surpassed). This indicator was added during project AF. achievement) Number of clinic guidelines developed and disseminated according to needs of FM Indicator 18 : practitioners (RS). Value (quantitative 31 NA 51 31 or Qualitative) Date achieved 03/22/2011 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target not achieved. achievement) viii Indicator 19 : Number of management staff that have been trained in relevant managerial techniques. Value FBiH: 131; RS: (quantitative FBiH: 25; RS: 20 FBiH: 131; RS: 50 FBiH: 211; RS: 397 300 or Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments Target achieved (surpassed). The indicator was introduced before project MTR and (incl. % monitored until project completion. achievement) Number of management staff in municipalities that completed all training modules Indicator 20 : offered by Health Management Center (RS). Value (quantitative FBiH: NA; RS: 0 NA RS: 50 RS: 71 or Qualitative) Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. The indicator was introduced during project AF. achievement) Number of management staff in municipalities that completed one or more training Indicator 21 : modules offered by the Health Management Center (RS). Value (quantitative FBiH: NA; RS: 0 NA RS: 250 RS: 349 or Qualitative) Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. achievement) Indicator 22 : Number of FM teams trained in changed management. Value FBiH: 600; RS: (quantitative 0 FBiH: 600; RS: 300 FBiH: 600; RS: 385 650 or Qualitative) Date achieved 06/30/2006 12/30/2010 12/30/2014 12/30/2014 Comments Target partially achieved (FBiH fully achieved the target value, while RS could not (incl. % achieve it). The indicator was introduced prior to project MTR and was monitored until achievement) project completion. Indicator 23 : Number of new Masters and PhDs in health management. Value FBiH: 1; RS: FBiH: 1; RS: 2PhD FBiH: 1; RS: 1PhD (quantitative 0 6PhD and 7 and 8 masters and 7 masters or Qualitative) masters completed Date achieved 06/30/2006 12/30/2010 12/30/2014 12/30/2014 Comments Target partially achieved. The indicator was introduced prior to project MTR and was (incl. % monitored until project completion. The original target was revised during project AF. achievement) Indicator 24 : Prepare program for specialist in health management (Federation). Develop A program for Value program for Continuous Program No specialization in health (quantitative NA health Education on health management program. or Qualitative) management management and specialization, rulebook developed. ix one cohort Training started in enrolled. October 2012 (199 applicants completed the first level, 132 the second level and 75 the third level). Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. The indicator was introduced during project AF. achievement) Indicator 25 : The number of Health Centers that have implemented the FM module. Value (quantitative 20% 80% NA NA or Qualitative) Date achieved 03/20/2005 12/30/2010 12/30/2014 12/30/2014 Comments The indicator is presented as per PAD. There is no evidence to prove its achievement, (incl. % no clear target value. The indicator was dropped before project AF. achievement) Indicator 26 : System for sector wide annual review of health status indicator established. Value (quantitative NA Report on time NA NA or Qualitative) Date achieved 03/20/2005 12/30/2007 12/30/2010 12/30/2010 Comments Target achieved. The indicator was introduced in PAD and monitored only until end of (incl. % 2007. The report for health status indicators was produced on time. The indicator was achievement) dropped before MTR. Indicator 27 : System to monitor and analyze impact of reform measures, effectively implemented. Value (quantitative NA Report on time NA NA or Qualitative) Date achieved 03/20/2005 12/30/2007 12/30/2010 12/30/2010 Comments Target achieved. The indicator was introduced in PAD and monitored until 2007. The (incl. % report on indicators analyzed also the reform impact. achievement) Closer collaboration among governmental and non-governmental sector and donors Indicator 28 : established. Agreements reached Value with donors and (quantitative NA NA 5 agreements non-government or Qualitative) sector. Date achieved 03/20/2005 12/30/2007 12/30/2010 12/30/2007 Comments Target not achieved. The indicator is presented as per PAD. Although efforts were (incl. % made to monitor it until end 2007, there is no clear evidence to justify its achievement. achievement) Indicator 29 : Number of private-public collaboration agreements (innovation grants in place). Value (quantitative FBiH: 0; RS: 2 FBiH: 10; RS: 8 NA FBiH: 41; RS: 9 or Qualitative) x Date achieved 03/20/2005 12/30/2010 12/30/2010 12/30/2010 Comments Target achieved. Grants (competitively awarded) signed with NGOs and local service (incl. % delivery providers. Not a PPP in real sense. achievement) Indicator 30 : Public health grants for Innovative Service Delivery (Federation). Value FBiH: 55; RS: (quantitative FBiH: 40; RS: NA NA FBiH: 57; RS: NA NA or Qualitative) Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. The indicator was introduced during project AF. achievement) Indicator 31 : Utilization rates for preventing services increased. Value (quantitative NA 60% NA NA or Qualitative) Date achieved 03/20/2005 12/30/2008 12/30/2010 12/30/2010 Comments The indicator is presented as per PAD. No baseline value was defined and the indicator (incl. % was dropped before project MTR. achievement) Indicator 32 : National health summits. 2 (Health Summit "All policies Health in South-East Europe: common Value goal and (quantitative 0 Yearly 2 responsibility." or Qualitative) Banja Luka October 2011; 90 year anniversary of HP in BiH October 2013. Date achieved 03/20/2005 12/30/2007 12/30/2014 12/30/2014 Comments Target achieved. The indicator was introduced before project MTR and monitored also (incl. % during the AF. Although few summits were organized during original project, they achievement) were not considered as proper policy discussion events. Health status and risk factors indicators based on household survey completed and Indicator 33 : publicly available. Completed analysis based Value Survey completed on survey (quantitative Survey ongoing in the RS. NA and results published results and or Qualitative) in both FBiH and RS. publicly available. Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. The indicator was introduced at project AF. achievement) Indicator 34 : FBiH prepare health M&E report. xi Value Complete M&E Most of indicators FBiH: report (quantitative NA report on health developed. completed. or Qualitative) sector. Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. The indicator was introduced at project AF. achievement) Indicator 35 : RS PHC program evaluation completed. Value Completed RS: survey (quantitative Initial analyses completed. NA review of PHC completed. or Qualitative) reform. Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target partially achieved. The indicator was introduced at project AF. achievement) Number of FM Educators (cathedra faculty or education center trainers) with a higher Indicator 36 : education (Masters or PhD) in FM. Value FBiH: 35; RS: (quantitative FBiH: 24; RS: 5 NA FBiH: 34; RS: 6 10 or Qualitative) Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target partially achieved. The indicator was introduced at project AF. achievement) Development of programs and monitoring tools for targeted NCD preventive program Indicator 37 : (Federation). Program monitoring Program monitoring tools Value tools for for targeted NCD FBiH: completed in 4 (quantitative NA targeted NCD preventive programs does cantons; RS: NA. or Qualitative) preventive not exist. programs completed. Date achieved 12/30/2010 12/30/2010 12/30/2014 12/30/2014 Comments (incl. % Target achieved. The indicator was introduced at project AF. achievement) Indicator 38 : Ratio between FM doctors and non-FM doctors in primary health centers. Value FBiH: 1:1.1; RS: FBiH: 1:1.1; RS: (quantitative FBiH: NA; RS: 1:1.56 NA 1:0.77 1:0.77 or Qualitative) Date achieved 12/30/2005 12/30/2010 12/30/2010 Comments Target achieved. The indicator was introduced before project MTR and was monitored (incl. % only until AF. achievement) Indicator 39 : Number of Cantons who have established a process of contracting for PHC. Value FBiH: 2; RS: NA NA NA FBiH: 3; RS: NA (quantitative xii or Qualitative) Date achieved 12/30/2005 12/05/2010 12/30/2010 12/30/2010 Comments Target not achieved. The indicator was introduced before project MTR. Since no (incl. % target values were defined the indicator cannot be considered achieved. achievement) G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 05/05/2005 Satisfactory Satisfactory 0.00 2 11/11/2005 Satisfactory Satisfactory 0.00 3 12/21/2006 Moderately Satisfactory Moderately Satisfactory 2.31 4 03/14/2007 Moderately Satisfactory Moderately Satisfactory 2.76 5 12/09/2007 Moderately Satisfactory Moderately Satisfactory 4.71 6 05/28/2008 Moderately Satisfactory Satisfactory 6.68 7 06/11/2009 Moderately Satisfactory Satisfactory 12.61 8 12/29/2009 Moderately Satisfactory Satisfactory 14.82 9 06/29/2010 Moderately Satisfactory Satisfactory 15.36 10 02/28/2011 Moderately Satisfactory Satisfactory 16.14 11 10/30/2011 Moderately Satisfactory Satisfactory 16.97 12 01/16/2012 Moderately Satisfactory Satisfactory 17.27 13 04/24/2012 Moderately Satisfactory Satisfactory 17.20 14 12/21/2012 Satisfactory Satisfactory 19.50 15 04/27/2013 Satisfactory Satisfactory 20.98 16 07/13/2013 Satisfactory Satisfactory 21.64 17 12/01/2013 Satisfactory Satisfactory 22.12 18 03/30/2014 Satisfactory Satisfactory 23.00 19 10/18/2014 Satisfactory Satisfactory 25.84 20 12/15/2014 Satisfactory Satisfactory 26.07 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions A level II restructuring took place to: (i) simplify the PDO language based on DCA wording, and revise the RF; (ii) reallocate Credit 11/18/2010 MS S 15.36 proceeds; (iii) change the environmental safeguards category from “C” to “B”; and (iv) extend the project closing date. xiii I. Disbursement Profile xiv 1. Project Context, Development Objectives and Design 1. The Bosnia and Herzegovina (BiH) Health Sector Enhancement Project (HSEP) was approved on March 31, 2005. The Development Credit Agreement (DCA), in the amount of SDR11.2 million (US$17.0 million equivalent), was signed on September 5, 2005 and the project became effective on April 3, 2006. The original Closing Date of December 15, 2010 was first extended to June 15, 2011, to allow processing of a project Additional Financing (AF). Project AF was approved on March 22, 2011, in an amount of SDR 6.5 million (US$9.5 million equivalent). As a result, a second extension of the project Closing Date was granted, until December 31, 2014. The additional funds were to allow scaling up of key original Project activities as well as supporting the implementation of certain new activities. 2. The Council of Europe Development Bank (CEB) co-financed the original project in the amount of US$14.0 million. In addition, CEB reached an agreement with the government to co- finance the project AF in the amount of US$10.0 million. Ratification of the CEB Legal Agreement by the government was completed on July 2014. Hence, the Closing Date of the CEB AF has been extended to December 31, 2015. 1.1 Context at Appraisal 3. Country and Sector background. At project appraisal in 2005, BiH had become a member of the Council of Europe and had started the long road toward European Union (EU) accession and membership in the North Atlantic Treaty Organization’s (NATO) Partnership for Peace Program (PFP). The country saw enhanced internal harmonization of the relatively autonomous sub-national Entities, while strengthening the coordination functions of BiH’s central State. With strong support from the international community, BiH made impressive progress in post-war reconstruction and in economic, social, and political integration. However, BiH continued to face a difficult economic situation, with GDP still only three-quarters of its pre-war level and growth not sufficiently strong to make a dent in the high unemployment. As most other sectors, health sector faced a number of challenges. The burden of disease was high and the epidemiological profile largely dominated by non-communicable diseases (NCD). In addition, the existing organization of the health sector required significant adjustments to ensure that the system would be able to cope with impeding demographic and epidemiological changes, as well as growing population expectations. 4. Main Health Sector Issues. Challenges faced by the health system in BiH were related to key areas, such as: (i) financial sustainability, (ii) inefficient organization model and service delivery; (iii) limited institutional capacity and institutional fragmentation; and (iv) unequal access to health care. The country epidemiological profile was largely dominated by non-communicable diseases (NCD), with 50 percent of deaths being attributed to cardio-vascular disorders and about 20 percent to cancers. BiH ischemic heart disease rate (160 per 100,000) was the highest in the Southeast Europe (SEE) region. Road accidents, injuries (including from landmines) and suicides were at high levels and on the rise. With a rapidly ageing population, increasing mental health issues and levels of tobacco and alcohol consumption, pollution and road accidents, the burden of disease was expected to further increase. 1 5. The 2003 Poverty Reduction Strategy Paper (PRSP) provided an overall vision for the health sector and its main reform goals. Major health gains, improvements in service delivery and financial savings were deemed to be achievable through working on three main axes: (i) reconfiguring the health care delivery system, reducing inefficiencies and improving quality in the delivery of care; (ii) reforming health financing; and (iii) building institutional capacity. The project was conceptualized based on key pillars of the reform, and thus focused on supporting the government efforts in two main areas: (i) the expansion of already tested family medicine pilots; and (ii) promotion of a number of innovative features, including: (a) development and implementation of a comprehensive reform program for PHC through family medicine expansion, including rationalization of health centers; (b) development of structures and processes for discussing strategic and management issues and reviewing sector performance; (c) strengthening of management and implementation capacities for future use of country systems and fiduciary processes; and (d) addressing political economy issues through support for consultation, communications, and public relations campaigns to ensure that the public and policy makers are informed about changes in the delivery system and the overall reform. 6. Country Partnership Strategy and Rationale for Bank involvement. The Bank was the main supporter for the health system reform in the country. Under the third pillar of the Country Assistance Strategy (CAS, 2004): “investing in key social and economic infrastructure”, one of the key objectives and expected outcomes was: improved health sector management and better access to quality health care. The project was fully consistent with this CAS goal. The Bank pursued an integrated approach for the assistance to health sector. It acquired considerable cross-sector knowledge of the country through implementation of adjustment and investment operations and economic sector work. The Bank was a partner of the government since the Dayton Peace Agreement and has been regarded by the government as an important partner not just for its direct support to the sector but also to help leverage additional funds from other bilateral and international agencies. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 7. The overall development objective of the project (as per PAD) was to improve overall results for non-communicable diseases as measured by proxy indicators and: (i) enhance health system efficiency through restructuring and strengthening of primary health care along the family medicine model; and (ii) strengthen the policy making process through the development and implementation of a system for monitoring and evaluating health sector performance. 8. The wording of the Project Development Objective (PDO) was consistent throughout the main text and Annex 3 of the Project Appraisal Document (PAD). However, there is a difference between the PDO’s wording in the PAD compared to the DCA. The PDO in the DCA is formulated as: “enhance health system efficiency by expanding and enhancing the family medicine model of primary health care, building management capacity in the sector and strengthening the policy making process through the development and implementation of a system for monitoring and evaluating sector performance and addressing inefficiencies of the sector”. The PDO wording in the PAD includes “improving overall results for non-communicable diseases”, which is not part of the PDO in the DCA. Meanwhile, the PDO in the DCA states: “building management capacity in the sector” as part of the PDO, which is not included in the PDO stated in the PAD. 2 The PDO indicators were: • Improved efficiency and equity in the planning, financing and regulations of the health service delivery system, • Improved access to quality primary health care throughout the country, • Decreased number of unnecessary referrals to higher levels of care, and • Number of primary care providers financed under new provider payment mechanisms. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 9. When the Additional Financing was processed, the project was restructured to address the need to: (i) simplify the PDO language; and (ii) revise the project results framework. The PDO wording referred to the original PDO in the DCA (since this was considered the “legal” language 1) and was stated as: “(i) expand and enhance the family medicine model of primary health care; (ii) build management capacity in the sector; and (iii) strengthen the policy making process through the development and implementation of a system for monitoring and evaluating sector performance”. Meanwhile, the project RF was revised, aiming at fine-tuning, based on available information, and taking into consideration (in time and scope) the new activities under the AF (details in the datasheet). 1.4 Main Beneficiaries 10. Entire population of the BiH was targeted to be the main beneficiary of a better access, utilization and quality of services, offered by a restructured primary health care. Most of Family Medicine (FM) teams, including FM doctors and nurses, were to benefit from project support to strengthen their capacity and adopt to the PHC reform. In addition, the PHC services were to benefit from the project through significant upgrading of most centers (physical rehabilitation, furnishing and supplying with medical equipment), providing the FM doctors and nurses with necessary tools to adequately carry out their tasks. Hospital and PHC managers were to benefit from capacity building in management area, with regard to quality improvements, regulations, resource management, etc. Also, the MoHs and key health institutions in both entities were to benefit from project support in strengthening the capacity for further sector reforming, including strengthening the M&E systems. 1.5 Original Components 11. The Project included three components: Component 1 - Primary Health Care Restructuring: The objective of this component was to support the government to restructure primary health care, addressing sectorial inefficiencies and ready the sector to deal in a cost effective manner with core health issues such as the growing burden of NCD. The component would support the restructuring of the traditional narrow specialist based primary care system into family medicine units that would ultimately have the 1 Project restructuring slightly simplified the language of the PDO as per project DCA, without revising it substantially. 3 capacity to address 80 percent of the country’s health needs, thus significantly reducing the unnecessary high rate of cases treated at hospitals. The component would support: (i) implementation of training programs in FM for doctors and nurses, including family medicine specialization and the Program for Additional Training (PAT); (ii) training of faculty of the FM Cathedra in the universities; (iii) limited refurbishing and equipping of FM facilities to provide retrained family physicians with the minimum tools necessary to adequately carry out their job; and (iv) communication activities to inform policy makers, providers and the public about the reform. The curriculum for FM specialization and for the PAT would be adjusted to address more deeply preventive services to allow FM teams to expand their scope of work. Component 2 - Improvement of Health Sector Management Capacity: The overall aim of the component was to support ongoing efforts in the sector to build capacity to manage and implement essential changes in service delivery at different levels of the health system. The component aimed to build on development and implementation of health management training initiated under Basic Health Project (BHP). The training and professional development programs would include: (i) training in management and oversight for hospital managers and board members, and health centers’ managers; (ii) change management training for FM teams leading to the development of restructuring plans for specific PHC facilities; (iii) academic degree training (Master’s and PhD programs) for faculty of health management; and (iv) development of purchasing and contracting skills for staff of the health insurance funds to support the expansion of health finance reforms to be piloted under the Bank financed Social Insurance Technical Assistance Project (SITAP). In addition, the component would provide technical assistance to carry out an accounting and internal control review of the Health Insurance Funds (Entity and a sample of cantonal HIFs) and a sample of the largest (in terms of expenditures) providers in each Entity with the objective to clearly identify control gaps and devise steps to improve transparency and accountability in the use of funds. Component 3 – Health Policy Formulation and Project Support: The aim of this component was to strengthen the capacity for data collection and analysis in support of sector policy formulation and to guide priority setting and resource allocation. The component would help introduce regular sector performance reviews. Monitoring and Evaluation (M&E) activities would support the data and study needs of MoHs to develop reports/information required for policy and strategic purposes. To this end, the component would support the development and implementation of a system for monitoring and evaluating sector performance and improving the dialogue among governments, local stakeholders and external partners. The component would support: (i) the definition of a list of sector wide indicators and the organization of surveys and studies on key issues, (ii) the preparation and organization of sector performance reviews; and (iii) the organization of "health summits" at which the results of the performance reviews would be discussed. The reviews would be conducted during the first part of each year to ensure that recommendations from the “health reviews" can be taken into account during the preparation of annual work programs and budgets for the succeeding year. The component would also support health innovation funds through which grants would be made available on a competitive basis to support: (i) jump-starting of hospital sector restructuring following the completion of hospital rationalization plans under SITAP; (ii) improvements in the continuum of care; and (iii) NGOs and youth organizations to develop and implement initiatives related to innovative public health interventions. 4 1.6 Revised Components See section 1.7. 1.7 Other significant changes 12. Project Additional Financing and Restructuring. An AF was approved in an amount of SDR 6.5 million (US$ 9.5 million equivalent) on March 22, 2011, to scale-up activities of the original HSEP and enhance its impact and development effectiveness. The additional funds were mainly to support expansion of certain original activities as well as a number of new activities. The AF was considered the best tool to complete the primary health care restructuring, without any significant disruption in on-going activities. The AF was fully supported by the BiH government, as clearly shown in the Country Partnership Strategy Progress Report (FY08 – FY11). At the same time, the HSEP went through a level II restructuring, to: • Simplify the language of the PDO as per the DCA, and revise the RF. The PDO wording was defined as: “ (i) expand and enhance the family medicine model of primary health care; (ii) build management capacity in the sector; and (iii) strengthen the policy making process through the development and implementation of a system for monitoring and evaluating sector performance” (as indicated under Section 1.3). Meanwhile, the project RF was revised, aiming at adjusting key indicators towards outcomes that were measurable and more closely aligned with Project activities. The following outcome indicators were dropped, namely: (i) improved efficiency and equity in the planning, financing and regulation of the health service delivery system; (ii) improved access to quality primary health care throughout the country; (iii) decreased number of unnecessary referrals to higher levels of care; and (iv) number of primary care providers financed under new provider payment mechanisms. A number of intermediate indicators also were dropped. New outcome and intermediate indicators were included and new target values were defined, as per the revised Project Closing Date. • Reallocate original Credit proceeds (Table 1). The reallocation was needed to: (i) allocate funds from the Unallocated category towards Goods and Works that have been included in the primary health care restructuring; (ii) reallocate 7 percent of Credit funds from the Training to Consultants Services category (to correct previous financing of training from consultancy funds, during original project); and (iii) reallocate 8 percent of the Credit funds to the Goods category from the Innovation Grants and incremental costs, resulting from the reduction in the number of Innovation Grants awarded. • Change the environmental safeguards category from C to B. The scaled-up Project triggered Bank’s Environmental Safeguards Policy 4.01. Due to the nature of the works planned to be supported under the AF, including significant rehabilitation and reconstruction of PHC facilities, the Environmental Rating was upgraded from category C to B. • Extend the Project Closing Date. The first extension of the project Closing Date was done for six months (from December 15, 2010 to June 15, 2011), granted prior the AF, aiming at allowing enough time for the preparation and approval of the Project AF. With the AF, a second extension of the Closing Date was granted, adjusting it to December 31, 2014. 5 Table 1: Revised Allocation of the Credit Proceeds by Category of Expenditures Original Amount of Amount after % of the Credit Allocated Reallocation Expenditures Category Expressed in SDR Expressed in SDR to be Financed (1) Works 330,000 945,620 90% (2) Goods, including 2,230,000 2,798,700 100% of foreign publications and expenditures; 100% of materials local expenditures (ex- factory and 85% of local expenditures for other items procured locally) (3) Consultants’ 5,040,000 6,262,320 100% of foreign services, including consultants firm, audit foreign individual consultants and 80% of local consultant firms and local individual consultants; 100% for surveys and studies (4) Training 1,300,000 478,340 100% (5) Innovation Grants 920,000 358,136 100% of withdrawn amount (6) Operating costs 590,000 356,884 90% until December 31, 2006; 75% until December 31, 2007 and 50% thereafter (6) Unallocated 790,000 0 TOTAL 11,200,000 11,200,000 13. The AF supported the following project components/activities: Component 1 – Primary Health Care Restructuring. The AF would support scaling up of activities related to: training programs in FM for doctors and nurses, including specialization in the PAT; Master’s and PhD programs in FM (only for the FBiH); development of additional clinical guidelines according to needs identified by FM practitioners (in the RS); increasing the number of rehabilitated, equipped and furnished FM offices and ambulantas and scaling up communication activities focused on the provision of the preventive programs (only for the FBiH). In addition, a number of new activities were approved, including: support to the Institute of Public Health in the FBiH to develop programs and monitoring tools for targeted non-communicable disease preventive programs; piloting incentive based payments for provision of preventive services in the FBiH; supporting the community nurses in the FBiH through training and provision 6 of minimum equipment; developing the rules for the health information standards in the FBiH; and supporting accreditation process of FM teams through the Agency for Accreditation and Quality Improvement. Component 2 – Improvement of Health Sector Management Capacity: The AF would provide further technical assistance for scaling-up training for hospital managers and health centers’ managers in the RS; provide change management training for the FM teams in the RS; and scale- up academic training (Master’s and PhD programs) for faculty of health management in the RS. Also, providing support for the institutionalization of the Health Management Centers was newly included (short-term courses for health managers, refurbishing of the teaching rooms and premises, equipment and technical assistance). Component 3 - Health Policy Formulation and Project Support: The AF would continue to support further development and implementation of a system for monitoring and evaluating health sector performance and improving health policy dialogue among stakeholders, in both RS and FBiH (including organization of Health Summits). The AF would also support the functioning of the PCUs in both entities, including incremental operating costs, consultant services, training and workshops. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry The ICR team rates design and quality at entry as Moderately Satisfactory, based on the following: 14. Financing Instrument. The project was financed from an original IDA Credit of SDR 11.2 million (US$17.0 million equivalent) and an Additional Financing IDA Credit of SDR 6.5 million (US$9.5 million equivalent), using a Specific Investment Loan (SIL) instrument. In addition, the project benefited from a co-financing from the CEB, in the amount of US$14.0 million. A SIL was considered the most appropriate instrument, based on the fact that the designed activities (mainly infrastructure upgrading and equipping the PHC facilities, as well as extensive capacity building) would be most effectively accomplished through investment lending instrument. 15. Soundness of background analyses. The project was prepared by a large team of experts. This was the fourth in a planned series of four operations in support of BiH’s health sector recovery and development program. Project preparation benefited significantly from the lessons learned during the implementation of the Basic Health Project (the third operation in support of health sector, closed in June 30, 2004). In addition, the team carefully considered key analyses and priorities extracted from the Poverty Reduction Strategy Paper (PRSP) (prepared by the government in 2003). Also, results of a Bank’s Operations Evaluation Department (OED) in-depth study on four completed health projects in ECA (published a year prior to project preparation), were carefully considered by the team 2. 2 Key lessons learned include the following: (i) great attention needs to be paid to political economy of the reform in the health sector; (ii) simple projects with relatively uncomplicated implementation arrangements are more likely to succeed; (iii) health sector reform is a lengthy, politicized process and expectations for the reform process in 7 16. Assessment of Project Design. A number of alternatives were considered during project preparation. The first alternative was having a Sector Wide Approach (SWAp). The feasibility of adapting a SWAp approach was discussed at length, including during the Quality Enhancement Review (QER). However, due to issues related to: (i) institutional fragmentation and weak capacity; (ii) lack of a single-operational and budget framework; and (iii) insufficient knowledge donors had on the sector financial management and procurement arrangements; it was decided not to move away from the traditional project approach and engage with other partners in a SWAp. A second alternative was proceeding with a narrow project, focusing only on expanding family medicine. Although this was the preferred option for the FBiH government at first, during the design process it became apparent that the project should also address the need to deepen the reform and integrate various elements into a comprehensive approach to sector development. Finally, expanding the scope of the project was also considered, including the areas of public health and accreditation, previously supported by the Basic Health Project. Yet, this alternative was not opted by the government, since the reforms in these areas were considered sustained, due to support provided under an EU-financed project. 17. Based on the existing government strategy for development of the health sector (under the 2003 PRSP), a wide range of issues were identified and discussed by the team during the design phase. However, the project was designed to focus on the most critical issues, with a long-term outlook towards implementing the health sector strategy. 18. Project design was the outcome of a thorough assessment of health sector needs and a cautious preparation. The team was successful in designing a project focused on key sector priorities. Yet, the design failed to put in place an appropriate monitoring and evaluation system for the project. This was a significant shortcoming, which negatively impacted the achievement of project main outcomes and outputs. In addition, although critical risks and related mitigation measures were clearly assessed during the design phase, the team overvalued the existing local capacity and neglected the risk posed by the highly fragmented nature of the health services, especially in the FBiH. 2.2 Implementation 19. The ICR team rates overall implementation as Moderately Satisfactory. The project was successful in achieving most of its expected outcomes and delivering the outputs specified under the three components. Project start up suffered from delays. While the project was approved on March 31, 2005, followed by signature of the Legal Agreement on September 5, 2005, the effectiveness date was extended twice from its original deadline of December 4, 2005, to April 4, 2006 for the Republika Srpska (RS) and September 7, 2006 for the Federation. Nonetheless, the initial phase of project implementation was rated as Satisfactory for both Development Objective (DO) and Implementation Progress (IP). previous projects have been too optimistic; (iv) institutional aspects of reform are as important as technically proficient strategies; and (v) close coordination among institutions is key to the success of a project. 8 20. However, only few months after effectiveness the project DO and IP were downgraded to Moderately Satisfactory. The downgrade was largely based on slower than expected progress in project implementation and stated project development objectives. 21. Implementation progress at this stage was negatively influenced by the following key factors: (i) considerable delays in mainstreaming of functions of the Project Coordination Units (PCU) in their respective MoHs and the necessary capacity building within the institutions; and (ii) highly fragmented nature of the health systems, especially in the Federation, requiring significant efforts for coordination and consensus-building among cantons. In the Federation of BiH (FBiH) the responsibility for health services has been delegated to the cantons, so the FBiH health sector includes the FBiH MoH, the 10 cantonal MoHs, the Federal Health Insurance Fund (HIF), the 10 cantonal HIFs and 11 Institutes of Public Health. While the Republic of Srpska (RS) health system has been (and remains) centralized at the entity level, so responsibility is shared only between the RS MoH, a single HIF and a single Institute of Public Health. 22. The situation improved, especially close to the project MTR (September 2008). The MTR noted that the PDO as well as overall project design remained valid. With more than 40 percent of funds disbursed and almost 80percent contracted by the time of MTR, project implementation was upgraded to Satisfactory. Meanwhile, the achievement of project development objectives was kept at Moderately Satisfactory, due to: (i) slow progress in the registration of the population by the Family Medicine, especially in the FBiH; and (ii) lack of proper evidence in the RS on improved efficiency with information on referral trends not being still available. Following the project MTR, two main issues were addressed, which positively influenced in further improving project implementation: (i) dropping financing of grants to NGOs for public health related activities (re-focusing attention on more important activities); and (ii) completion of required information for the revised project RF. As requested by the Bank management and agreed with the government prior to the project MTR, the task team significantly revised key performance indicators in the project RF and started to collect all required baseline data and agree on target values. 23. The project continued to perform satisfactorily until the original Closing Date, December 15, 2010. Six months before the original closing date, disbursement had reached 89 per cent of the total Credit allocation, while the remaining Credit funds were committed for activities that were under implementation. At this stage, an Additional Financing (IDA Credit) was approved in March 22, 2011 in the amount of US$9.5 million (following a six months project extension for allowing the preparation of the AF). The AF was already discussed and agreed with the government and reflected in the Country Partnership Strategy Progress Report (FY08-FY11), in order to scale-up activities of the HSEP and enhance its impact and development effectiveness. The AF, which had very strong support of the BiH government, was considered as a best mechanism to maximize development impact and results than a repeater project, a new operation, or non-lending instruments. In parallel, the project went also through a level II restructuring (as detailed in Section 1.7). 24. The overall performance of the project AF was satisfactory. Only a year after AF approval, the rating for the achievement of the DO was upgraded to Satisfactory, while the implementation performance remained at satisfactory. AF implementation faced some challenges, including: (i) delays in implementation of the IT activities in the FBiH, mainly due to lack of proper capacity. Following the hiring of technical assistance (firm) to support the IT implementation, the activity 9 was successfully completed; and (ii) significant delay in signing the agreement with the CEB, for the AF. In order to bridge this shortcoming, the Bank had to modify procurement plans to allow main activities to be financed mainly by IDA (while originally planned to be co-financed); and (iii) uncertain financial situation, especially for the RS, due to accumulated hospital arrears, could risk the required support to project implementation. 3 However, despite issues and delays during implementation, the performance of the AF remained satisfactory. 25. Key positive factors that influenced the overall project performance, included: (i) Strong government commitment to reform the health sector. Despite relatively weak capacity in BiH, fragmented nature and diffusion of authority of the health systems in the FBiH, the government has continuously displayed a strong commitment towards reforming the existing system and a clear strategic vision in the sector. (ii) Continuous coordination with other international organizations. The project kept close coordination with other donors and international organizations. Apart from the support offered by the CEB, a long partner of the Bank and BiH government has been the Canadian International Development Agency (CIDA). Following the project MTR, an assessment of the improvement towards quality and continuity of primary health care in the RS was carried out, with the support of the Bank as well as CIDA funding. In addition, the RS, with CIDA support, carried out an evaluation of the relative efficiency of the PHC providers. 2.3 Monitoring and Evaluation (M&E) Design, Implementation, and Utilization 26. Design. The RF in the PAD proved to be ambitious, especially for the project key performance indicators, in terms of monitoring and achieving them. The original RF missed the baseline data and most target values for key performance indicators and some intermediate indicators, with the assumption that required information would be collected within the first year of project implementation. The project key performance indicators were mostly general and not thoroughly adjusted towards project outcomes. In addition, a number of intermediate indicators were not closely aligned with project activities. An M&E plan was developed and agreed with the client, including specific reports, data collection instruments and responsible institutions. 27. Implementation. At the beginning of project implementation, M&E was relatively neglected. It was only after a year from project implementation start that the team was asked by Bank management to carefully revise the RF and work with both entities to ensure baseline and define missing target values. Lack of proper framework negatively influenced the ability to correctly monitor project achievements. 28. A year prior to project MTR the team started the work to intensively revise the results framework, to promote a more effective monitoring of progress towards the attainment of PDO, and outcome and output indicators were refined. Despite the changes made, the intended outcomes of the project remained the same. In addition, agreement was reached with both entities on 3 The situation was managed by the Government and did not affect project implementation. 10 collecting the appropriate information related to monitoring of performance indicators 4. Although the revised RF was used and closely monitored by the teams, the formal approval of the revisions was completed only at the time of project restructuring (finalized in parallel with project AF). Key factors which influenced the delayed approval included: (i) specific complexity working with two Entities and highly decentralized institutions in the FBiH, which brought difficulties in collecting the appropriate information; and (ii) frequent changes of project TTLs during that period, resulting in delayed processing of the formal project restructuring. 29. During the project AF (March 22, 2011), a second enhancement of the RF took place, aiming at reflecting the newly introduced activities. The RF was fine-tuned, taking into account the proposed new as well as scaled-up activities, both in time and scope. In addition, a household survey of the health status of the population of the FBiH and public opinion survey was carried out by the Institute of Public Health (IPH) in 2012, with a comparison to the survey conducted in 2002. Meanwhile, in the RS a few surveys were organized, with the last one completed in November 2014, aiming at analyzing the impact of training on the PHC staff and managers performance. 30. While at the beginning, project implementation experienced the lack of a proper M&E and weak ownership, the RF and monitoring and reporting started to improve prior to project’s MTR and continued to further improve during the AF. Overall, the RF was adequate for the remaining project implementation period. 31. Utilization. The surveys organized in both entities as well as the M&E report produced in the FBiH collected data not just related to the project’s needs, but also included important information used by the government in strategic documents (including the Systematic Country Diagnosis - SCD) and in policy decision-making. The most thorough survey was carried out in the FBiH by the IPH (completed in 2012) on the health status of the population, health care services utilization and health related needs. The survey results proved to be very important in defining priorities for adopting new strategies and health care programs and creating prevention and promotion programs. 2.4 Safeguard and Fiduciary Compliance 32. The environmental category of the original project at appraisal was “C”. Physical rehabilitation of existing facilities through remodeling and refurbishing did not include any major structural changes and therefore only minor quantities of debris were generated. The implementing agencies were guided by the Project Implementation Manual on workers safety, dust and noise pollution, proper handling transportation and disposal of construction waste materials. The Project Implementation Manual had specific provisions for and guidance on the safeguards for preventing health impacts from direct and indirect exposure of humans to health care waste. In addition, reconstruction activities were carried out in full compliance with national environmental requirements and based on construction permits and urban planning documentation. 4 In the case of FBiH data collection was part of the regular health statistics, through the Institute of Public Health (IPH) of FBiH and through cantonal IPHs. Other information needed for monitoring was obtained through cantonal Ministries of Health and from the database of the Federal Ministry of Health. While in the case of RS, information was obtained through the Ministry of Health. 11 The project followed up on good measures practiced in BiH for safe management of syringes, blades, etc. 33. During project MTR a special environmental supervision was conducted. The supervision confirmed that: (i) majority of implemented civil works (completed in 24 out of 33 buildings) were considered corresponding to category “C”; and (ii) activities in the remaining 9 buildings included leveling down the old buildings and construction of new ones. However, the assessment carried out by the Bank team in collaboration with local government, concluded that there was no need for any remedial measures as there were no environmental issues related to that. In addition, there were no involuntary resettlement issues associated with the newly constructed centers. 34. It was only at the time of the AF when the upgrade of the environmental category to “B” was introduced, based on: (i) the nature of the works to be supported by the project (including significant physical rehabilitation and reconstruction of existing building, as well as few new construction); and (ii) the changing requirements of the Bank’s safeguards policy. All new construction was done on the available municipal lands that were not being used in any matter, and thus did not imply any of involuntary resettlement issues. Also, Environmental Management Plans (EMPs) of both entities were developed, disseminated in country and disclosed. The EMPs included also a monitoring plan with environmental indicators and monitoring and reporting procedures. Overall, all the works have been carried out according to standards and environmental guidelines and safeguards compliance has been satisfactory, throughout the original project and the AF. This was also confirmed during the visit of the ICR team to several rehabilitated facilities. 35. Procurement. Procurement Management (PM) functions have been carried out by the respective Project Coordination Units (PCU) in both entities. While in the FBiH the PCU became part of the MoH staff soon after project start up, the PCU at RS remained as a Unit outside the MoH and was integrated only during the last year of implementation. Project implementation faced a number of procurement issues, including: (i) lack of proper capacity within the PCU in the FBiH. Most of the time procurement was handled by part-time consultant, which, at a certain degree, hampered the overall performance of the Unit and negatively influenced in the overall contract administration and physical inspections (which were not always carried in a fully satisfactory manner) and timely updating of the procurement plans; and (ii) lack of experience in the local private sector in preparation of bids in accordance with the WB procedures, which resulted in delays of the procurement processes. The Bank’s teams have continuously recommended strengthening of the procurement capacity in the implementing agencies as well as offering training to the local private sector on WB procurement procedures, in order to avoid these issues. In addition, there were isolated cases of potential collusion between bidders, from which two local firms were debarred. Despite these issues, the overall performance of the procurement has been rated as Moderately Satisfactory or Satisfactory throughout the original project and the AF. 36. Financial Management. The PCUs in both entities carried financial management responsibilities in a satisfactory manner. Quarterly financial reports were prepared and submitted on time, providing reliable financial information. The financial statements were regularly audited by independent auditors, resulting in unqualified opinions. Despite certain delays, caused by PCU transfers and staff changes, overall both PCUs had appropriate skills and ability to manage the project financial management and disbursement issues. 12 2.5 Post-completion Operation/Next Phase 37. Transition Arrangements. A number of positive developments have taken place in restructuring the PHC and strengthening the capacity building in the sector. The government is committed to sustaining project achievements and making sector performance improvements. Investing in reforming the PHC services is expected to result in reduced unnecessary expenditures in hospital care. Also, a significant area of project support has been the improvement of human resource capital (health workers in PHC). Although there exist the risk of having a number of FM doctors and nurses trying to leave the primary care, most of project investment in the area are expected to remain in PHC for a long time. Meanwhile, investments made in works and goods would need regular maintenance, hence required financial resources need to be considered by the government. This is an important issue, considering that the current financial sustainability of the health system is threatened by poor collection of insurance contributions as well as inefficiencies from the fragmentation of service delivery at secondary and tertiary levels of care. These are issues to be considered during discussions between the government and the Bank on future potential engagement. 38. Follow-up project. The government, during the ICR mission, has expressed its strong interest in a follow-up project that will further support the on-going reforms. The following are among key priority areas under discussion, during the preparation process of the Country Partnership Framework (CPF): (i) further deepening the FM reform model, which is considered as the single largest reform implemented by the BiH government in the health sector; (ii) strengthening governance and management capacity in the sector, including for hospitals. The existing highly fragmented nature of the health system in BiH has resulted in limited sector performance. Hence, further improvements in stewardship and governance would increase efficiency and improve service delivery and health outcomes; and (iii) improving the financial sustainability of the system, which is currently threatened by inefficiencies resulting from the fragmentation of service delivery and insurance systems. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 39. The relevance of objectives is rated Substantial. Reforming the health system, with specific focus on improvement of primary health care services through the FM model and the management capacity in the sector, was a high priority for the BiH government (FBiH and RS) before project start and remains so at completion. Project objectives were fully relevant to the priorities of the 2003 PRSP. In this strategy, the government clearly recognized the urgent need to improve service delivery, reduce inefficiencies and build institutional capacity. The project has firmly supported these goals, which have remained as priority of the current RS and FBiH sector strategies. Also, the project has served to address key challenges in the health sector reform, which clearly falls under one of main pillars of the Systematic Country Diagnosis (SCD), currently being finalized, as well as the Country Partnership Framework (CPF), under preparation. 40. The relevance of project design is rated Modest. The SIL design was the right approach to support activities mainly related to infrastructure upgrading, equipment and capacity building. Planned activities were relevant to the achievement of the project’s objectives. However, 13 weakness in M&E design, including lack of baseline and target values, clearly undermined the relevance of the project design. 41. The overall rating for relevance is Modest, based on Substantial relevance of objectives and Modest relevance of project design. 3.2 Achievement of Project Development Objectives 42. The achievement of the PDO is rated as Substantial. This evaluation is based on the achievement of key outcome indicators, on the analyses of the accomplishment of project intermediate indicators and the contribution of project activities to the achievement of the PDO (two out of three PDO parts are rated as substantial, with one part rated as modest). In addition, the results of the survey on the Health Status in the FBiH 5 confirm the significant improvement of satisfaction with FM services, as a result of PHC reforming. An average of 89.4 percent of the respondents expressed their satisfaction with the approach of FM teams, as well as with the improvement in health care quality following the introduction of FM. Also, the survey shows improvement in a number of selected health indicators (details in Annex 5). 43. Overall, the investment in the FM model has been the single largest reform implemented by the government during last decade. In addition, the project has achieved significant results in strengthening the institutional capacity in the health sector. These achievements have positively affected the reform in other key areas, including: (i) health financing, where a new payment system is being introduced to create incentives for providers to improve sector-wide results. Under this, diagnosis-related group system (DRG) is introduced in hospitals. While in primary care, the unified methodology for contracting health services is being used, primarily with FM teams 6; and (ii) adjustment of several policy and legal frameworks, to increase the efficacy and efficiency of the sector, to strengthen financial sustainability, and/or to improve the quality of health care 7. These policy and legal improvements are paving the way for additional reforms in the areas of health financing, quality improvements, service delivery, and governance; to improve health outcomes and increase financial protection. 44. Achievement of PDO part 1: Expand and enhance the family medicine model of primary health care (Rating: Substantial). With project support, the family medicine (FM) model – implemented over recent years – has been the single largest reform implemented by the 5 Survey of the Health Status of the population of the Federation of BiH in 2102, with a comparison to the survey conducted in 2002. 6 With project support, incentive payments for family medicine teams were initiated, to provide the standardized set of preventive and promotional services. 7 RS adopted the Strategy for Primary Health Care in 2006, the Strategy for Secondary and Tertiary Health Care in 2007, the Law on Health Care in 2009 and the law on health insurance with associated bylaws. In addition, a Strategic Plan for Strengthening of the Health System until 2020 is being prepared. The FBiH has also developed a new Strategy for Primary Health Care and a Strategy for Health System Reform and has amended the Health Care law. 14 government in the health sector, aiming at addressing sectorial inefficiencies and ready the sector to deal in a cost effective manner with core health issues (such as growing burden of NCD). The project led to successful achievement of the three key outcome indicators, associated with this project objective. In addition, the ICR evaluation is also based on the analyses on how project activities contributed to the overall achievement of this objective. 45. Coverage of population through Family Medicine (FM) teams reaching a national average of 70 percent. With the support of the project, a national average of 80.2 percent coverage was achieved at project closure (95.5 percent for the RS and 69 percent for the FBiH). The project supported key activities (capacity building for FM teams, physical upgrade of FM facilities and supply with IT systems), which positively affected improvement of coverage of the population by FM services. 46. The project supported the establishment and implementation of the training programs in FM for doctors and nurses, including family medicine specialization and the PAT. A total number of 793 FM teams in FBiH and 685 FM teams in the RS were operating at project closure, not only achieving but also exceeding the agreed target values. Capacity building of the FM teams, FM educators and health personnel in general was significantly supported by the project, mostly achieving the expected results. A total of 436 medical doctors in FBiH (target 523) and 227 medical doctors in the RS (target 250) successfully completed the required specialization. In addition, 986 FM doctors in the FBiH and 148 FM doctors in the RS completed the PAT, by project closure. Aiming at sustaining the results achieved under the capacity building support, 40 FM educators from the faculty of the FM departments in the respective universities (both in FBiH and RS) completed the trainings for Masters or PhD in FM. All capacity building activities were supported by the original project and scaled up by the AF. 47. All training activities significantly expanded on the successful experiences under the previous Basic Health Project, while ensuring that the institutional capacity within universities is strengthened to guarantee sustainability of all training programs. However, although the capacity building activities clearly achieved the expected final results, evidence collected during the ICR mission indicates the risk of specialized FM doctors lacking proper motivation and seeking specialization in other medical specialties, mainly moving from primary to hospital care. This signals the importance of considering a set of incentives, in both entities, aiming at preventing future losses of trained human resources. 48. With project assistance, both entities have carried out physical rehabilitation, furnishing and equipping (with basic medical equipment) of a considerable number of the FM facilities. At project closure, the FBiH has achieved and even surpassed the agreed target value (with 455 FM offices/ambulances reconstructed or renovated, while target value is 385 FM offices/ambulances). The same goes for the FM offices/ambulances equipped with medical equipment and/or furniture. The FBiH has again achieved and surpassed the target value, with 488 offices/ambulances equipped, while the target was only 426. This was supported by some savings achieved from combining procurement processes and reducing the operating costs, especially during the project AF. The savings were also used to solve some of the emergency needs generated by the flooding of the summer 2014 (repair ambulances, replace small equipment, etc). Meanwhile, the process of rehabilitating the FM offices/ambulances in the RS suffered certain delays in contracting, especially the last four contracts. However, the end target values were mostly achieved even in 15 the RS, with 213 FM offices/ambulances renovated (out of an end target of 275). Meanwhile target value for furnishing with medical equipment and furniture was fully achieved. 49. In addition, the project successfully supported IT activities for the FBiH primary health care, with the objective to implement a state-of-art and nationally certified information system in the community health care centers. Support included: (i) preparing and enacting the Rulebook on definition of PHC health Information System Architecture, to ensure technological and functional requirements for implementation of Health Information System (HIS) for PHC, as well as to stipulate procedures for certification of software solutions to be used in the PHC; (ii) conducting situational analysis to assess both quality and quantity of IT capabilities at community health care centers in the FBiH; (iii) purchasing software applications, to improve the management of health services and implemented performance based payments, with special focus on preventive services; and (iv) purchasing needed IT equipment (hardware and network equipment), LAN and WAN equipment. Due to lack of staff in the health institutions, with proper IT knowledge, certain shortcomings were faced at the situational analysis, though without any significant impact on final results. At project completion, agreed target values for the FM offices/ambulances equipped with IT were achieved successfully. Although the original plan was to purchase and implement the same Electronic Health Record across the entire FBiH, the system covered all cantons with the exception of two cantons which did not approve the purchasing of the software applications. Meanwhile, the IT in the RS was supported mostly through the MoH funding. Currently, the HIS is widely being used to support decision making at FM facility level and at entity level. 50. In order to inform policy makers, providers of primary health care and the public about the reform in primary health care, the project provided assistance for carrying out a number of communication activities (including preparation and dissemination of materials, media spots, press conferences, brochures and posters, radio and TV shows). Project AF further elaborated the use of communication activities, scaling it up to focus also on provision of the preventive care programs for the FBiH (while no communication activity was scaled up for the RS, following the agreement with the MoH to continue through regular communication channels). Although no specific indicator is related to this activity, evidence collected during the ICR mission clearly indicates the significant impact communication activities had in raising awareness of the public as well as providers and policy makers on restructuring the primary health care, in both entities. The results were quite positive even for the RS where, during project AF, the communication was taken over by the government itself. 51. Number of FM teams and Health Centers in RS meeting rigorous accreditation standards. The project supported (via provision of technical assistance, capacity building as well as installing a specific software which allows access to real-time information) the FM teams and ambulantas in the RS to strengthen their overall standards, in order to complete all requirements of the accreditation process. The support was provided only during the AF, for FM practitioners and health centers to be accredited through the Agency for Accreditation and Quality Improvement. 52. At project closure, this specific outcome indicator was only partly achieved: 200 FM teams were accredited, out of the agreed target value of 260 FM teams. Two were main reasons for not fully achieving the agreed target (i) lack of motivation for some FM teams to continue performing as FM doctors and seek specialization in other specialties; and (ii) stripping off the original 5 per cent incentive for accredited teams (shortly after establishing it as an important incentive), due to 16 the overall economic crises. It is strongly suggested to both entities to consider a new set of incentives to avoid additional losses of trained FM teams. 53. Systemic implementation of selected preventive services (i.e. hypertension, breast cancer, prostate enlargement, smoking, etc.) in at least two major FBiH cantons. This indicator was introduced at project AF stage. Four pilots (cantons of Sarajevo, Zenica, Tuzla, and Mostar) to promote preventive services for NCD by introducing pay-for-performance were implemented and successfully completed before project closure. A total of 100 FM teams participated in this piloting, divided in two groups: experimental and control group. Also, contract models were developed between community health care centers and their FM teams and between the FBiH MoH and community health care centers. A final report was produced at the completion of the pilots’ implementation, indicating a significant greater performance of the 50 FM teams receiving the payment incentives. In addition, the reporting system developed for the four pilots has been adopted as standard reporting system in the FBiH. This new reporting system (including the software) was very well accepted by the FM teams, because it allows them to analyze their own performance and also to better report their activities. Also, during the mission, the ICR team observed that one of the cantonal health insurance fund was considering to include a similar incentive in the new contract with the FM teams; an action which positively speaks for the sustainability of this outcome. 54. Achievement of PDO part 2: Building Management Capacity in the sector (Rating: Substantial). Project RF includes only one outcome indicator linked to this PDO, added during project AF/restructuring. This is because the PAD did not specifically include this part of the overall PDO (which was clearly included in the PDO of the original DCA as well as the project AF). Nonetheless, the PAD included a number of activities and output indicators, which have clearly contributed to the achievement of this specific PDO. In addition, the responses received from the PHC managers and staff during the survey conducted in the RS (completed in November, 2014) indicate that training activities significantly helped them in the process of establishing a quality management system and preparation and completion of the PHC accreditation process. 55. Health Management Center operational with annual budget and regular work plan. The indicator was fully achieved by project closure. The Health Management Center in the FBiH was developed as a unit within the Federation Public Health Institute (FPHI). A contract was signed between the FPHI and the School of Economics in Sarajevo to cover all training required. Meanwhile, in the RS, training was covered by the Public Health Institute’s Health Management Center. Project support through technical assistance, purchasing of required IT equipment, as well as the software application for the e-learning, is considered instrumental in establishing the Centers. 56. By project closure, most of related output indicators in support of PDO part 2 were achieved. With project assistance, the following training programs were developed and implemented: • Training in management and oversight for hospitals and PHC’s managers. Target values for this indicator are achieved in both entities. The FBiH reached a total of 211 managers, trained in relevant managerial skills (while target value was only 131 managers). Meanwhile in the RS the total number of trained managers reached to 379 (against a target value of 300). This training program supported managers to better align with the on-going 17 reform, implementing required changes in their institutions with regard to quality improvements, regulations, referrals, resource management, etc. • Training in change management for FM teams. The program included support for the transition of service delivery to the new primary care model, enabling the teams to implement required changes in management concepts and skills, staff planning, patients flow and management. The FBiH has achieved the target, with 600 FM teams completing this training program within project closure. While the RS only partly achieved it, with 385 FM teams trained, out of agreed 650 teams. Although the RS Health Management Center was established in time, there were delays as well as lack of incentives for some FM teams to complete the training. • Training modules offered by the Health Management Center in the RS. This output concerns only the project support to the RS. 349 management staff completed one or more training modules offered by the Center, surpassing the target of 250 staff (the activity falls only under the project AF). • Academic degree training (MAs and PhDs) in health management. This training program has produced a cadre of teachers of health management, able to teach Masters and other courses offered by the centers of health management. A total number of 3 PhDs and 8 master degrees were completed, partly achieving the target for this output indicator. • Training program for Continuous Education (CPE) only in the FBiH, during project AF. The program for CPE on health management and related rulebook was successfully developed, and the first cohort was enrolled. By project closure, 199 applicants completed the first level of training, 132 applicants the second level and 75 the third/final level; ensuring the successful achievement of this indicator. 57. Achievement of PDO part 3: Strengthening the policy making process through the development and implementation of a system for monitoring and evaluating the health sector performance (Rating: Modest). With project support, policy making process based on use of evidence, has significantly improved, although with some shortcomings. 58. The project has led to important achievements, as evidenced by the following key associated outcome and output indicators: 59. Primary health care strategies, prepared by relevant ministries of health and ratified by the relevant governments. This indicator was defined as outcome indicator in the original project and was successfully achieved before the AF was approved (target met in 2010). While in the RS the strategy was prepared by the MoH, in the FBiH the MoH was supported also by the WHO to complete the strategy in time. 60. The AF scaled up this outcome indicator, with the following wording: “The RS Primary Health Care Strategy and the Federation Primary Health Care Strategy operational plans are updated”. Both entities updated the operational plans in time and implemented (with slight delay in the RS due to government changes during 2013). Hence, the indicator is considered fully achieved. 18 61. The project successfully met most of the targets outlined in the RF, in reference to output indicators, as follows: • FBiH prepares the Health M&E Report. By closure of the original project, the FBiH MoH had developed the required set of indicators and the initial data were collected. The work continued during project AF and the M&E report was completed and publicly available by project closure. Project assistance was instrumental in the successful completion of the report, with support provided for: (i) technical assistance, contracted for strengthening the MoH monitoring and evaluation system; and (ii) establishing the website of the FBiH MoH and making it operational. • Health status and risk factors indicators based on household survey completed and publicly available. The surveys have been completed successfully in both entities and the results have been made public. In the FBiH, project support played an important role in the preparatory work and required equipment for the Public Health Institute to carry out the survey on the population’s health status and risk factor for the NCDs. • National Health Summits held with the participation of the relevant stakeholders. Although few summits were organized during the original project at the entity level, they were not based on results of the M&E systems, hence did not have the expected impact on policy discussions. As result, the anticipated annual review process was not fully put in place. The project supported this activity during the AF too, ensuring the achievement of the related indicator. Two health summits were held: one in Banja Luka (RS) titled “Health in South-East Europe common goals and responsibilities”, and the second one in Sarajevo (FBiH) “90 year anniversary of the primary health care in BiH”, in October 2013. Both summits were considered significant events in health policy discussions in the country. • Program Evaluation for the Primary Health care in the RS. During the original project, a number of analyses were carried out to analyze and evaluate the progress of the reform in PHC in the RS. A complete survey at PHC level was organized towards original project closure. The survey was completed during project AF and served for a thorough review of the PHC reform in the RS. • Public health grants for Innovative services: In the FBiH a number of innovative grants were offered, with the objective to improve general public awareness on prevention of risk factors for most common and mass NCDs, through different innovative prevention approaches, strengthening of prevention through FM teams and better understanding and acceptance of FM model in general public as a system of primary prevention. The grants were implemented in three phases, reaching a total of 40 grants successfully implemented by project original closure. An additional 17 grants were implemented during the project AF, focusing on prevention of risk factors in the most common mass noncontiguous diseases. These projects effectively demonstrated existing weaknesses in the area of organized prevention and health care for vulnerable groups. • Finally, the reporting system developed for the four pilots, to promote preventive services for NCDs by introducing pay for performance, has been adopted as standard reporting system in the FBiH (as detailed under PDO part 1). This is an important achievement for 19 improving the M&E system. Also, progress achieved in establishing HIS and FM monitoring has clearly contributed to decision making process based on evidence use. 3.3 Efficiency 62. Efficiency is rated as Substantial. The BiH Health Sector Enhancement project had three components: restructure the PHC system; improve health sector management; and promote health policy formulation and project management support. Although all three project components generated benefits, most direct benefits of the project are from the first component, while the second and third components generated indirect benefits by strengthening the healthcare system, health sector management and policy development. 63. Economic analysis (Annex 3) used the actual project costs and estimated flow of benefits to compute the measures of economic efficiency. The analysis assumes that the project benefits continue for 20 years. The measures of economic efficiency under the base case scenario and sensitivity analysis under two alternative scenarios are presented in Table 1. Table 1: Measures of economic efficiency under base case and alternative scenarios Base Sensitivity analysis under two alternative case scenarios 8 Scenario 1 Scenario 2 Net present value (million 28.7 18.7 15.1 $) Benefit/Cost ratio 2.01 1.76 1.3 Economic rate of return (%) 19.13 14.7 9.2 64. Under the base case scenario, the project yielded 19.13 percent Economic Rate of Return. The sensitivity analysis showed that even under conservative assumptions the discounted project benefits covered costs resulting in modest returns to investment. 65. The operation and maintenance costs of the project are not significantly high when compared with the public expenditures in the health sector in the country, as high as 9.6 percent of the GDP. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 66. The project’s overall outcome rating is based on the Substantial achievement of project development objectives, as discussed under section 3.2 above, its Modest relevance, and Substantial efficiency. 8 The first scenario assumes a 20 percent reduction in benefits compared to the baseline case. The second scenario assumes a 30 percent increase in recurrent costs. 20 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 67. The project has clearly contributed to improvements in the overall functioning of the health system, with significant impact in the primary health care reforming. Project achievements have impacted the overall population of BiH, including vulnerable and poor groups. The project RF did not include any indicator specifically related to poor groups of the population. However, it positively impacted the standardization, quality and access of the population to the primary health care services. By doing that, the project clearly has encouraged equity in health service delivery. (b) Institutional Change/Strengthening 68. The project had an important impact on institutional development and strengthening capacity in the sector. A significant proportion of project investments was addressed to improve the capacity of human resources in the PHC, in a number of key areas, such as: (i) capacity building of the FM teams, FM educators and health personnel, with a total of 436 medical doctors in FBiH and 227 medical doctors in the RS successfully completing the required specialization; (ii) training on management for hospitals and PHC’s managers, with 211 managers trained in relevant managerial skills in the FBiH and 379 managers trained in the RS; (iii) training for continuous education, in the FBiH; (iv) academic degree training (Masters and PhDs) in health management field; and (v) training of the FM teams in change management. Establishing the Health Management Centers and having them operational with annual budgets and regular work program, is considered an important step for further expanding and sustaining the capacity building in the sector (more responsibilities were transferred to and properly handled by the local institutions). In addition, study tours and most recent Health Summits organized at national level, had a positive impact on improving institutional capacity at policy level. The evidence collected and results of the interviews during the ICR mission clearly indicate that project support was instrumental in strengthening institutional capacities and enabling them to further continue with significant reforms. (c) Other Unintended Outcomes and Impacts (positive or negative) 69. During project AF phase, some savings were created from combining procurement packages and reducing the operating costs. These savings were used to solve certain emergency needs, generated by the flooding of the summer 2014. 9 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 70. See Annex 5 9 Such as repairing ambulantas, replacing small equipment, etc. 21 4. Assessment of Risk to Development Outcome Rating: Moderate 71. The risk that project development outcomes will not be maintained is Moderate. The government (in both entities) has clearly demonstrated strong commitment in considering health as a priority sector (despite periods of economic crises) and continuously supported the health care reforms. In addition, it has shown commitment throughout project lifetime, aiming at achieving and maintaining main project outcomes. This was clearly demonstrated during project implementation, when there were delays in ensuring CEB funding for the PHC reconstruction works and provision of medical equipment. The government took over part of the financing, ensuring timely completion of these main project activities. In addition, it also provided financial support to a number of cantons, which were having difficulties in providing their own part of the financing. Also, the significant reform of the FM in primary health care is preserved in a concrete legal framework, which clearly ensures the use of the primary care as a gatekeeper. 72. However, there are still a few issues to be noted, which can have a negative influence on the government commitment towards sector reform: (i) the frequent political changes may bring high level of staff turnover in the MoHs and related institutions (in both entities); (ii) FM specialized doctors may look for other employment opportunities, mainly in the hospital sector, negatively affecting the capacity built in the PHC; and (iii) despite its strong commitment, the government may not have the adequate capacity to maintain and deepen sector reform, hence a follow-up support by the Bank and/or involvement of other donors is of significant importance. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 73. Preparation of the project was carried out by a large team of experts, providing needed technical skills to produce a proper design. The project design built upon key lessons learned from analytical work, Bank-financed health projects in the BiH and other countries in the region as well as international best practices. A number of alternatives and main risk factors were considered by the team during the preparation process, prior to finalizing the project design. In addition, the team ensured that project design was in line with the government strategy (PRSP 2003) and paid particular attention to the political economy of the proposed reforms. The ICR team noted satisfactory feedback from the client on the working relations the Bank team kept with all involved stakeholders, at design stage. 74. However, the project was prepared quite fast, with less than nine months from concept review to Board approval. Hence, the project design suffered from certain shortcomings. Most importantly, project design did not put in place a proper M&E framework for monitoring the implementation of project activities and assessing the achievements of its objectives. Most indicators in the original RF were missing the baseline data. Although the design talked about surveys to be forecasted, nothing was organized during the first couple of years of project implementation. Despite the fact that this shortcoming was addressed during project 22 implementation, monitoring of project progress during implementation and final assessment of project achievements were made more challenging. (b) Quality of Supervision Rating: Moderately Satisfactory 75. The Bank's performance during the implementation of the project was Moderately Satisfactory. Sufficient budget and staff resources were allocated, and the project was adequately supervised and monitored. Bank supervision took place on regular basis and provided appropriate and well-targeted advice and observations. The aide memoires provided evidence of regular supervision and professional advice given by the Bank’s experts throughout the project’s lifetime. The Implementation Status Reports (ISRs) realistically rated the performance of the project both in terms of achievement of development objectives and project implementation. In addition, the feedback received by the ICR mission during the interviews with stakeholders, clearly show the government’s appreciation of the technical skills and advice provided by the Bank’s experts, not just on project but also on health sector related issues. 76. The Bank team has been committed not just in performing close supervision of project implementation but also trying to overcome any shortcoming from project design stage. Hence, prior to project MTR, the team was flexible in trying to adjust the M&E framework. Direct involvement and leadership by the CMU and sector management side at this point was instrumental in ensuring the required adjustments. However, the issues with the project RF remained present until AF phase. Although the RF was revised and closely monitored by the teams during implementation of the original project, the formal approval of the revisions was completed only at the time of project restructuring (finalized in parallel with the AF). At preparation and implementation of the AF the team was commendably proactive in further refining the project RF and identifying any implementation bottlenecks and measures to overcome delays. 77. During the entire project life (original and AF) there were in total five task team leaders. Although there were frequent changes of TTLs, especially during the original project, they could ensure smooth transition, which guaranteed continuity. 78. Fiduciary and safeguards policies were well managed and reported. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 79. The overall performance of Bank’s teams is rated as Moderately Satisfactory. The quality at project preparation was less than satisfactory. Also, Bank’s performance during project supervision was moderately satisfactory. Although the performance improved during project supervision, the shortcomings from the design phase (mainly lack of proper M&E) could not be fully overcome. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 23 80. The project design was strongly supported by the government (in both entities), recognizing the need to decisively pursue restructuring of primary health care, to ensure more effective resource allocation. While being aware of the institutional fragmentation and capacity constraints at almost all levels, the government was able to be selective, paying particular attention to the political economy of sector reforms. 81. Despite political transitions during project implementation (resulting also in staff changes), the government kept strong commitment to project and sector reform implementation. Also, the government demonstrated a long - term strategic vision in the sector over the project’s lifetime. Government officials worked closely and fully cooperated with the Bank’s project team, on a continual basis. The project MTR particularly demonstrated the commitment and involvement of both MoHs in project implementation. Both MoHs played a key role in project monitoring, analyzing implementation bottlenecks and proposing solutions for ensuring successful completion of project activities. Government involvement remained the same also during project AF, its commitment was one of key reasons for proceeding with the additional financing. 82. The project did not experience any counterpart funding problems. While delays happened in ensuring CEB funding, the government also supplemented Bank financing, aiming at timely completion of main project activities (i.e. physical rehabilitation and medical equipment for PHCs). In addition, the government took over funding of a number of activities, originally planned to be supported by the cantons in the FBiH (due to a lack of local government financial resources). 83. Overall, government performance in ensuring political and institutional commitment was satisfactory. The political transitions in the government, the fact that project was implemented in two entities, as well as strong decentralization in the FBiH negatively affected project implementation. However, this negative impact was mostly fixed, due to continuous commitment. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory. 84. The MoHs in both entities were responsible for overall project implementation. The MoHs were fully involved and responsible for the project preparation, while during the initial phase of implementation, technical support from local consultants was required. Following Bank’s recommendation, the MoH in the FBiH included the project unit (composed of a number of local consultants) within the ministry, with most of the staff being MoH employees. Meanwhile, in the RS, the project was implemented by a PCU financed by the project. 85. During project implementation (most of the original project and whole AF), the Federation MoH continued to implement the project through a department of the Ministry, led by an Assistant Minister and using contractual experts on procurement and financial management. This helped in streamlining technical capacity within the MoH, and therefore, strengthen its longterm capacity. However, during the first year of implementation, the FBiH MoH used the team of consultants working for SITAP (another Bank supported project). The fact that these consultants did not work full time on the project hampered the quality of work. In addition, there were significant delays in hiring the experts of the PCU as civil servants. Even when the PCU was within the MoH, it lacked a full-time procurement specialist, which has negatively affected the Unit performance. 24 86. In the case of RS, the project was implemented by an external PCU. The experts of the PCU were contracted as consultants and working, at the same time, on other projects. Although it was recommended to have the Unit incorporated within the RS MoH, aiming at strengthening the MoH capacity, this happened only in January 2014 (during project AF). Since then, the PCU staff were hired as civil servants and financed by the MoH. According to the interviews during the ICR mission, this transition influenced in a poorer performance of the Unit (main reason would have been the change in benefits for the PCU staff, due to transition). (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 87. The satisfactory performance of the government and moderately satisfactory of the Implementing Agency averages to a rating of Moderately Satisfactory for overall Borrower performance. 6. Lessons Learned 88. Flexibility from the Bank side, to adjust to client specific institutional arrangements, is essential for a successful implementation. The complex political environment (working with two entities), frequent political transitions, and highly decentralized governance in the FBiH, brought up some difficulties during project implementation. Working in such a specific complex environment, required certain flexibility from the Bank teams, which were successful in adjusting to the specific client arrangements and needs. 89. The Bank and the government need to give careful consideration to the project implementation arrangements, to ensure a smooth implementation of project activities. Fully staffed PCUs, responsible for overall project implementation, are crucial for successful completion of project activities. While the FBiH/PCU was streamlined in the MoH during project implementation, yet its performance was negatively affected by certain delays in hiring its experts as civil servants, as well as the lack of a fulltime procurement specialist. Meanwhile, the transition of the RS/PCU (streamlined within the MoH only during last year of project implementation) resulted in less efficient functioning and poorer performance of the Unit. Hence, it is recommended that during the preparation of a new operation, careful consideration be given to defining the PCU arrangements, based on analyses of the impact that streamlining in the MoHs had during project implementation. 90. Sustaining the achievements of the reform in PHC, through Family Medicine model, is of significant importance. The FM model – implemented over recent years – is considered the single largest reform implemented by the government in the health sector. Currently, there exist the risk of specialized FM doctors and nurses, lacking proper motivation and seeking specialization in other medical specialties, mainly trying to move from primary to hospital care. This signals the importance of considering a set of incentives to prevent future losses of trained human resources, as a crucial step for sustaining the achievements of this large reform. 91. M&E is a critical part of the project design and it requires strong attention during project preparation and implementation. The lack of a proper framework negatively influences the ability to correctly monitor project achievements. The RF for this project was, to some extent, neglected at design stage and its ownership was weak. 25 92. A quick project preparation may negatively affect the quality of project at entry. The project was prepared quite rapidly, with less than nine months from concept note review to Board approval. Despite team efforts for preparing a thorough project, the quick preparation did not provide the required time to: (i) produce a proper M&E framework and collect required information; and (ii) carefully consider specific needs and situations of the country, with focus on human capacity. As a result, project implementation endured certain bottlenecks during its first years, which were mostly sorted out during project life. 93. Close coordination with development partners is very important for a successful project implementation. Coordination with key donors involved in the sector, such as: CIDA and, most importantly, CEB (who was the co-financier for the project), was very important for a successful project implementation. The project experienced delays in ensuring funding from the CEB (due to late signatories of the agreement between the Government and CEB, for the original project as well as the AF). However, because of the strong coordination between the government, the Bank and CEB, project implementation could overcome issues, resulting in successful completion of planned project activities. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies See Annex 7 for Borrower’s contribution to the ICR. No comments were provided by the Borrower on the ICR, following the Bank letter of May 29, 2015. (b) Co-financiers No comments were provided by the CEB on the ICR, following the Bank letter of May 29, 2015. c) Other partners and stakeholders N/A 26 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ Million equivalent) 10 Components Appraisal Restructur Actual/Latest Percentage Appraisal Actual/Latest Percentage Estimate ed Estimate of Estimate Estimate AF of AF (US$ milli Estimate (US$ millions) Restructure AF (US$ millions) Appraisal ons) IDA d Estimate (US$ milli (US$ milli ons) ons) 1.Primary Health 10.86 12.15 10.79 88.80 5.49 6.78 123.50 11 Care Restructuring 2.Improvment of Health Sector 1.81 1.65 1.39 84.30 1.14 0.72 63.20 12 Management Capacity 3.Health Policy Formulation and 4.33 3.20 2.29 71.60 2.87 1.46 51.00 13 Project Support Total Financing 17.00 17.00 14.47 14 85.10 9.50 8.96 95.00 10 The table represents only IDA financing (since the project implementation is still continuing, financed by CEB and Government funds). 11 Due to use of cost savings from other components. The savings were mainly used to solve certain emergency needs, generated by the flooding of the summer 2014. 12 Due to: (i) cost savings from combining procurement packages, and (ii) transferring more responsibilities on capacity building activities to local institutions (mainly during project AF). 13 Due to reduction in the number of Grants implemented as well as operating costs (resulting from streamlining the PCU experts to the MoHs). 14 Overall, decrease in the actual project costs reflects the exchange rate fluctuations in the Euro to $ US. 27 (b) Financing Appraisal Actual/Latest Percentage Source of Funds Estimate Estimated(USD of Appraisal (USD millions) millions) Borrower 8.90 5.57 62.6 Council of Europe Development Bank 14.00 n.a n/a (CEB) Council of Europe Development Bank 10.00 n/a 15 n/a (CEB) Additional Financing International Development Association 17.00 14.47 85.10 (IDA) International Development Association 9.50 8.96 95.00 (IDA) Additional Financing Total Financing 59.40 n/a n/a 15 Project implementation is still on going, financed by CEB and Government funds. 28 Annex 2. Outputs by Component Component Planned outputs at Appraisal Actual outputs/outcomes at ICR 1 - Primary Health Care The component included the following This component includes the Restructuring planned outputs: following outputs/outcomes: (i) Capacity building for the FM teams (i) Implementation of training through implementing training programs programs in family medicine in family medicines for doctors, and provided for doctors and nurses, nurses, including family medicine including family medicine specialization and the Program for specialization and the Program of Additional Training (PAT). Additional Training (PAT). A total of 436 medical doctors in FBiH and 227 (ii) Capacity building through training in RS completed the required for faculty of Family Medicine specialization in addition 986 FM Cathedra’s in the universities. doctors in FBiH and 148 FM doctors in the RS completed the PAT. (iii) Improving FM facilities infrastructure through limited (ii) Training of faculty of FM refurbishing and equipping, to provide departments in the universities. The retrained family physicians with training produced a cadre of teachers minimum tools necessary to carry out of health management, able to teach their job, masters and other courses. In total 3 PHDs and 8 Master degrees were (iv) Communication activities to inform completed. policy makers, providers and the public on the reform. (iii) The FM facilities in both Entities have carried out physical rehabilitation, furniture and equipping (with basic medical equipment). In FBiH 455 FM offices/ambulances were reconstructed, while 488 were equipped with medical equipment and/or furniture. Meanwhile 213 FM offices/ambulances were renovated in the RS. (iv) IT system was supported in the FBiH PHC. This included purchasing of software applications, required IT equipment, LAN and WAN equipment. Meanwhile unified Electronic Health Record across FBiH was established (with exception of two cantons). (v) A number of communication activities were carried out, including preparation and dissemination of materials, media spots, press 29 Component Planned outputs at Appraisal Actual outputs/outcomes at ICR conferences, brochures, radio and TV shows. (vi) TA and IT software to support the RS FM teams and ambulances to complete requirement of accreditation process. 200 FM teams were accredited by project closure. (vii) TA support and software system for 100 FM teams in the FBiH to promote preventive survives for NCD by introducing pay-for-performance. 50 FM teams received payment incentive. 2 - Improvement of Health The component included the following This component includes the Sector Management planed outputs: following outputs/outcomes: Capacity (i) Strengthening capacity through (i) Completed training in training in management and oversight management and oversight for for hospital managers and board hospital managers and board members, and health centers’ managers. members, and health centers’ managers. A total of 211 managers in (ii) Capacity building through change FBiH and 379 managers in RS were management training for FM teams trained. leading to the development of restructuring plans for specific PHC (ii) Training in change management facilities. in FM teams completed by 600 teams in FBiH and 385 in RS. (iii) Development of purchasing and contracted skills for staff of the Health (iii) Health Management Centers Insurance Fund. (HMC) were established and operational in both entities by project closure. (iv) The RS HMC trained 49 management staff. (v) The training program for Continuous Education on health management and related rulebook successfully completed in the FBiH. 3 - Health Policy The component included the following This component includes the Formulation and Project planed outputs: following outputs/outcomes: Support (i) Definition of a list of sector wide (i) Set of indicators were defined and indicator and the organization of the health M&E report was produced by surveys on key issues. project closure in the FBiH. 30 Component Planned outputs at Appraisal Actual outputs/outcomes at ICR (ii) The preparation and organization of (ii) With project support the IPH in sector performance review. the FBiH completed the survey on the population health status and risk (iii) Organization of “health summits” at factors of the NCDs. which the results of performance would be discussed. (iii) Few national health summits have been organized to discuss on (iv) Innovation funds through which sector performance, with two main grants would be made available on a ones organized during project AF. competitive basis. (iv) Public health grants for innovative services (mainly in the FBiH) were implemented in three phases, reaching a total of 57 grants by project closure. 31 Annex 3. Economic and Financial Analysis 1. Introduction Health sector in Bosnia-Herzegovina is characterized by high burden of disease with an epidemiological profile dominated largely by cardiovascular diseases and cancers. The sector has high public health expenditures, an inefficient organization model and service delivery, limited institutional capacity, institutional fragmentation and unequal access to health care. The motivation for the project was the financial unsustainability and inefficiency in the health sector. The project supported the government strategies to improve the healthcare delivery system by reducing inefficiencies and improving quality in the delivery of care. This was achieved through advancing primary care and public health, reforming health financing and building institutional capacity in the sector. The project helped the Government of BiH to reform of the health sector and improving efficiency and development of the primary health care so as to attain financial sustainability of the health system in long run. The project had three components: a. Primary Health Care Restructuring. This component was to restructure the primary health care delivery system and to ensure that it is equipped to adequately address core health sector issues such as the growing burden of non-communicable diseases (NCD) and enable development of an effective interface between outpatient care delivered through family medicine (FM) and hospital care. This component supported restructuring of the narrow primary care system into family medicine units that have the capacity to address 80% of the country’s health needs. This has significantly reduced the unnecessary high rate of cases treated at hospitals. b. Improvement of Health Sector Management Capacity to strengthen the capacity of provider managers and governing bodies to implement essential changes in service delivery at different levels of the health system. This component provided training and technical assistance for hospital managers, board members and managers of health centers. c. The third component was to promote Health Policy Formulation and Project Support to improve data collection and analysis for sectorial benchmarking and to improve use of evidence for priority setting and resource allocation, and help modernize the role of the Public Health Institutes. These activities promoted health sector policy formulation. 2. Recent developments in health sector The Health Sector Enhancement Project sought to address the inefficiencies in the health sector like inefficient treatment protocols, duplication of services, and a neglect of primary healthcare, which contributed to inadequate provision of services to the poor and uninsured. The impacts of the reforms resulted in progress in health status and health outcomes in general (Table 1) over the period since 2005, an indirect measure of the impact of the project. The health condition of the population in BIH was poor in the early 2000s with about 16 percent of the population in poor health (LSMS, 2004). The poor health condition was by and large due to illness, disability due to the deterioration in access to health care and inefficiencies in healthcare management. However, in recent years there have been improvements in the indicators of health status and health care. Infant mortality rates increased from 6.7 in 1990 to 8 in 2000 and then fell 32 to 6 in 2013. The life expectancy has also been gradually increasing since 2000. Table 1 shows gradual improvements in health care indicators as well. Table 1: indicators of health status and health care in BIH Health status 2000 2005 2011 2013 Life expectancy at birth 75 75 77 77 Infant mortality rate (per 1000 live births) 8 7 6 6 Under-5 Mortality rate (per 1000 live births) 9 8 7 6 SDR, Diseases of the circulatory system per 100000 427 Tuberculosis incidence (per 100,000 people) 94 73 54 46 Health care indicators Physicians (per 1000 people) 1.4 1.4 1.7 1.9 Average length of stay in hospital 9.6 11.1 Hospital beds per 1000 people 3.2 3 3.5 Child immunization rates, Measles (% of children) 80 90 89 94 Child immunization rates, DPT (% of children) 85 87 88 92 Total health expenditure as % of GDP 7.1 8.7 9.6 Inpatient hospital admissions (average number of days) 8.2 Total health expenditure per person 603 Source: World Health Organization and World Bank Although there has been improvement in the above group of indicators, the population is getting older and the incidence of diseases of the circulatory system and other diseases like cancer are increasing. The percentage of population in poor health and affected by serious physical limitations due to illness is around 16 percent (LSMS, 2003 results). Mortality rates per 1000 live births increased from 6.7 per 1000 in 1990 to 8 in 2002 due mainly to the effects of the war on health of the population and deterioration in accessibility and standard of healthcare. Male life expectancy increased gradually from a fairly low level at 68 years in 2002 to 73 in 2010 and 74 in 2013. Hospital discharges for diseases of circulatory system were 861 per 100, 000 in 2006. Although there have been improvements in health status, lifestyle related illness continue to impose high burden disease and the population quality of life. 2.1 Health expenses The health services in BIH continue to be costly. The percentages of population using health care services vary by the availability of health insurance and health status. The poor do not have access to insurance services and they continue to use public health care institutions. Private sector healthcare is not covered by the health insurance. However, a significant proportion of the population continues to use private sector health care at higher costs. This is because private sector continues to offer better services, better quality and shorter wait times. Although BiH has a network of primary care facilities, at the time of project development only about 60 percent of the cases consulted were solved at the primary level. The rest 40 percent are 33 referred to the specialized hospitals at the secondary and tertiary care level at higher costs for the services rendered. Health expenditure in BIH continues to be high and payment for health services constitute a significant share of the household expenses. More than 80 percent of the patients incur out of pocket expenditures, out of pocket expenditures on the average account for more than 90 percent of the total health expenditures. Although health expenditures per capita have been increasing in the early 2000’s it has begun to fall in recent years 16. Table 2: Health expenditures in BIH 2000 2005 2011 2013 Out of pocket health expenditures as % of private 97 100 96.9 96.9 expenditures on health Health expenditure (Per capita) 103 246 471 449 Public Health expenditure % government 11.4 16.6 expenditure Total health expenditure as % of GDP 7.1 8.7 9.6 NA However out of pocket expenditures continue to be high, accounting for more than 90 percent of the private expenditures on health. The high health care costs have been due mainly to the inefficiencies in health sector even when health infrastructure has been successfully rebuilt. In this context, the project contributed to five health policy goals namely strengthened primary health care, improved health sector management, increased utilization of quality health care, cost containment and more efficient use of scarce resources. The primary health care model (PHC) resulted in a significant percent of the population being covered through family medicine, leading to lower out-of-pocket expenditures for households. The project facilitated access to family medicine for the majority of the population resulting in improved access to primary health care, less severe case-mix among patients, lower treatment costs and out of pocket payments. Use of preventive care and improved access to healthcare also improved productivity through avoiding lost productivity due to preventable illnesses and related premature deaths. Further, better health and fewer days lost due to illness will also improve overall productivity. The low-income groups and people without insurance also benefited the most from the expanded family medicine network. 3 Methodology, data and assumptions In order to conduct the economic analysis the actual project costs and the estimated benefits are used. 3.1 Project costs The actual project costs for the three components by years are presented in Table 3. Primary healthcare restructuring component accounted for 88 percent of the project costs followed by third component at 8.3 percent. 16 OECD STAT-Extracts; http://stats.oecd.org/index.aspxDataSetCode=HEALTH_STAT 34 Table 3: Project costs by components and by years (US $ 1000) Primary health Improvement of health Health policy Total care sector management formulation and restructuring capacity project support 2006 532.5 39.5 572.0 2007 3,911.4 17.2 126.3 4,055.0 2008 5,234.5 355.3 373.2 5,963.1 2009 6,003.6 472.1 541.8 7,017.5 2010 2,835.6 66.7 289.3 3,191.6 2011 762.5 154.3 417.4 1,334.3 2012 94.5 14.4 129.3 238.1 2013 1,428.8 115.2 464.3 2,008.3 2014 6,936.2 225.8 7,162.0 Total 27,739.9 (87.9) 1195.2 (3.7) 2,606.8 (8.3) 31541.9 (100) (Figures in parentheses are percentages of total) The project costs show that the primary emphasis of the project was in restructuring of the primary healthcare services delivery system for improvements in efficiency and to ensure that it is equipped to adequately to address core health sector issues such as the growing burden of non- communicable diseases (NCD) and enable development of an effective interface between outpatient care delivered through family medicine (FM) and hospital care. 3.2. Project benefits The project scaled-up the PHC to provide improved access to care in family medicine practices to about 80 percent of the population. The second and third components improved financial sustainability and institutional capacity in the health sector and policies to increase efficiency in healthcare delivery. Among the above three project components, most benefits accrued from the first component that incurred about 88 percent of the total project costs. The second and third components contributed to the overall project benefits. The direct and indirect benefits from the project are outlined below. a. Reduction in unnecessary hospital admissions and bed days as a result of the increase in quality of family medicine and primary care. The project facilitated improved family and primary care treatments that reduced the number unnecessary referrals to specialist and hospital admissions. As a result of this intervention specialist hospital admissions dropped overall, due in part to the decline in unnecessary self-referrals at specialized hospitals. This is because the trained providers in PHC clinics could provide primary care, early diagnosis and treatment plans that reduced the need for specialist care and admissions resulting in significant drop hospital admissions and about 30 percent of those savings could be attributed to the project. We do not have data on actual cost of inpatient care to estimate the above savings from bed days avoided. Hence cost of inpatient care is estimated from the total health spending, recurrent budget for inpatient care and total number of hospital days. It is further assumed that about 75 percent of the total health spending goes to hospital care, resulting in total cost per inpatient day at US$ 184. In addition the recurrent costs, including salaries and drugs are estimated at a 50 percent above the total estimate at US$ 92. 35 b. Reduction in disease incidence in days and thus averted losses in productivity due to diseases and lost workdays. Averted productivity and lost man-days arise from fewer disease incidence, reduced inpatient admissions and bed days from inpatient admissions. Productivity is valued based on the average daily wage per person at a labor participation rate of 45 percent. The reduced productivity losses are estimated to be 0.5 days per referral. Similarly, fewer admissions and fewer bed days resulted in savings in productivity. d. Reduction in travel costs to hospitals and specialist clinics and out of pocket expenditures. Out of pocket expenses and travel costs to the hospitals are in general higher than for visits to support primary care centers. Hence a reduction in specialist consultation will result in savings in travel costs and out-of-pocket expenses. Based on LSMS findings out of pocket expenditures for travel to PHC is estimated at $1.8 per visit. For hospital stays it is estimated at around US $ 3.8. The above expenses are then adjusted for inflation. e. Reduction in-patient waiting times at primary health centers. Streamlining the appointment process reduced the patient waiting time at the care centers. The average waiting time has been reduced by about 5 percent. The resulting savings in waiting time is valued at the daily wage rate. f. Savings in potential life years as a result of reduced mortality from diseases. Better quality of treatment and preventive measures averted chronic diseases, mortality and the burden of diseases. According to the World Health Organization’s Atlas of Health in Europe improved monitoring and treatment of respiratory and circulatory system diseases as well as early detection of diseases of digestive system reduces mortality rates from these diseases. Based on available evidence from surveys and evidence from other published works, early detection through higher utilization of PHC services and better quality of care reduced the mortality reduced by about 2.9 percent over ten years. In order to compute the gains from reduced mortality it is assumed that each death averted results in nine life years saved. The monetary value of life years saved was then estimated from the per capita incomes over nine years. The project supervision documents and the beneficiary surveys showed that the benefits started to materialize from the second year after the project was implemented. 3. 3 Measures of economic efficiency As outlined in the previous section, the project benefits include the savings from reduction in hospital inpatient care and admissions, reduction in disease incidence and savings and averted losses in productivity, savings in potential life years as a result of reduced mortality and reduction in travel costs to hospitals and specialist care centers. Under the base case scenario, most project benefits started to accrue in the third year and are expected to continue until 20 years. The costs and estimates of benefits were discounted at 5 percent to compute the measures of economic efficiency (Table 4) Table 4: Base case scenario: Measures of efficiency (5 % discount rate) Net present value (Million $) 28.7 Benefit/Cost ratio 2.01 Economic rate of return (%) 19.13 36 All three measures of economic efficiency show modest returns to the investment and economic efficiency. The estimated measures of economic efficiency are very close to the expected results at project initiation as presented in the Project Appraisal Document. 3.4. Sensitivity analysis The project benefits and measures of economic efficiency are sensitive to alternative scenarios. Reduction in one or more of the benefits or an escalation in recurrent costs could affect the net present values and rates of return. The sensitivities of the project results to the following two scenarios are considered. i) Scenario 1: Reduction in benefits by 20 percent compared to the baseline situation. ii) Scenario 2: There is an increase in recurrent costs by about 30 percent. Table 5: Sensitivity analysis under the two alternative scenarios (5 % discount rate) Scenario 1 Scenario 2 Net present value (Million $) 18.7 15.1 Benefit/Cost ratio 1.76 1.3 Economic rate of return 14.7 9.2 The sensitivity analysis shows that the returns from the project are positive even under the above two scenarios. The effect of an increase in recurrent costs seems to have higher impact on the returns from the project. On the whole these results under the base case scenario and the alternative cases are comparable to the ex-ante results presented in the Project Appraisal Document. 4. Conclusion The analysis showed that the project generated positive benefits with modest rates of return on the investment. The increase in access to primary health care, particularly for the poor with no health insurance, benefited the poor and low-income groups who may not have a formal employment. The positive rates of return and the facilitation of primary health care to all groups, particularly the poor and lower income groups are notable features of this project. The operation and maintenance costs of the project are not significantly high when compared with the public expenditures in the health sector in the country as high as 9.6 percent of the GDP. Compared with such high share of public health expenditure, the recurrent cost impacts of this project on the state budget are not significant. 37 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending Francois Decaillet Lead Public Health Specialist ECSH1 - HIS Betty Hanan Senior Operations Officer ECSHD - HIS Monika Huppi Adviser IEGPS Nedim Jaganjac Senior Health Specialist GHNDR Mirjana Karahasanovic Operations Officer GENDR Carmen F. Laurente Senior Program Assistant GEDDR Imelda Mueller Operations Analyst ECSH2 - HIS Augustina M. Nikolova Operations Analyst GHNDR Jesus Renzoli Senior Procurement Specialist ECSO2 - HIS Sanjay N. Vani Lead Financial Management Spec OPSOR Mark Walker Chief Counsel LEGSO David Webber Consultant GGODR Supervision/ICR Francois Decaillet Lead Public Health Specialist ECSH1 - HIS Michele Gragnolati Program Leader LCC7C Dominic S. Haazen Lead Health Policy Specialist GHNDR Betty Hanan Senior Operations Officer ECSHD - HIS Nedim Jaganjac Senior Health Specialist GHNDR Mirjana Karahasanovic Operations Officer GENDR Nikola Kerleta Procurement Specialist GGODR Carmen F. Laurente Senior Program Assistant GEDDR Zorica Lesic Operations Officer ECSH2 - HIS Lamija Marijanovic Financial Management Specialist GGODR Imelda Mueller Operations Analyst ECSH2 - HIS Vedad Ramljak E T Consultant ECSHD - HIS Pia Helene Schneider Lead Evaluation Officer IEGPS Sanjay N. Vani Lead Financial Management Spec OPSOR Ethan Yeh Economist ECSH1 - HIS 38 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY05 71.47 305.87 FY06 3.15 1.97 Total: 74.62 307.84 Supervision/ICR FY06 32.44 115.07 FY07 33.57 98.14 FY08 32.13 111.40 FY09 25.53 72.23 FY10 25.40 73.84 FY11 29.04 91.21 FY12 19.97 100.05 FY13 20.84 82.85 FY14 19.35 111.36 FY15 10.60 58.57 Total: 248.87 914.72 39 Annex 5. Beneficiary Survey Results FOREWORD This documents is a research study into health status of population of the Federation of Bosnia and Herzegovina (FBIH) in 2012. This study was jointly carried out by the Federation of BiH Ministry of Health and the Federation of BiH Public Health Institute under the Health Sector Enhancement Project (HSEP) financed by the World Bank's IDA credit funds. This extensive population research was carried out on a representative sample of adult population of the FBiH and it also represent a study that effectively monitored major health determinants such as risk factor prevalence in development of chronic diseases, availability of health care, utilization of health care providers and population satisfaction with health care provided, since the baseline study was carried out in 2002. Using almost identical methodology that relies on international recommendations, the 2012. Federation of Bosnia and Herzegovina Adult Population Health Status Study provides good basis to evaluate effects of health reforms and health care system, evaluate health care and also promotion of health in the past decade. When compared to earlier baseline study carried out in 2002, data obtained now will allow for evaluation of adopted policies and implemented programs, identification of priority health issues and issues faced by health sector. At the same time, this study also allows monitoring of population's health condition and risk factor prevalence trends, as well as a careful look at real health needs in order to define immediate operational goals in further improvement of health system. The analysis showed that we are yet to face ever increasing challenges caused by modern society and social and economic transition in the Federation of Bosnia and Herzegovina, which altogether has substantial implications on health condition of population. Federation of BiH Ministry of Health is dedicated to continue to work to increase health care for population, in particular primary health care and health prevention services, to increase general risk factors awareness and awareness on self-responsibility for health in population, which would in turn result in sustained improvement of health outcomes. Primary goal is to enhance health which in long-term should involve activities aimed at strengthening skills and abilities of individuals and changes to social and economic conditions which should mitigate their (potentially negative) effects on health of individuals and community and also provide more efficient and quality health care for longer life and better life quality of people. Minister Prof Rusmir Mesihović, PhD, MD 40 METHODOLOGY OF THE SURVEY The survey has been conducted as a cross sectional study on the area of the ten cantons in the Federation of BiH in 2012. Sample Target population Members of the population of the Federation of BiH aged 18 and over and present in the country for at least a full year have been selected as the target population. Collective households such as student homes, children's homes, homes for the elderly, infirm, retired, prisons, monasteries and the like are not included in the target population. Sample selection frame The sample selection frame is a master sample of enumeration districts and households from 2009, established and updated by the Federal Institute for Statistics. Households and individuals aged 18 and over are the units of observation, whereby the sample was designed to be representative on the level of individuals aged 18 and over, as well as on the level of households. Family medicine utilization More than two-thirds of respondents in the FBiH have their family doctor (68,9%), more so respondents living in urban (72,6%) than those living in rural areas (66,4%). 72.6 68.9 66.4 FB&H Urban Rural Graph 1 - Respondents that have their family doctor, by type of settlement they live in the FBiH, % 41 Territorial availability of family medicine Almost half of the respondents in the FBiH reside less than 1501m away from the nearest health care facility providing family medicine services (48.4%), which is more often the case among respondents living in urban (69,9%) than those living in rural areas (33,9%). Over a third of respondents in the FBiH (35,7%) who receive family medicine services reside more than 2000m away from the nearest health care facility, which is true for almost half of the respondents in rural areas (48,1%) and a significantly lower percentage of respondents living in urban areas (17,1%). 38.3 31.6 27.8 24.3 24.1 22.5 20.3 19.1 17.9 15.9 14.8 14.6 13.0 13.2 2.6 Up to 700 m From 701 to 1500 m From 1501 to 2000 m From 2001 to 5000 m More than 5000 m FB&H Urban Rural Graph 2 - Respondents by distance from nearest health care facility providing family medicine services in the FBiH, % Visits to a family doctor One third of respondents have visited their family doctor in the past 4 weeks (33,7%), while a similar percentage of respondents visited their family doctor in the past 12 months (32,1%), respondents in both urban and rural areas have done so in almost equal percentages. More than a quarter of respondents in the FBiH have visited their family doctor more than a year ago (28,1%). On the other hand, 6,1% of respondents have never visited their family doctor. 42 35.1 33.7 33.0 32.6 32.1 31.5 29.3 28.1 26.4 6.1 6.6 5.4 During the past 4 weeks During the past 12 months More than a year ago Never visited FB&H Urban Rural Graph 3 - Respondents by when they last visited their family doctor in the FBiH, % Among the respondents that visited their family doctor in the past 4 weeks most frequent were those aged 65 and over (58,5%), while the highest percentage of respondents visiting their family doctor in the past 12 months can be found in the 18-24 age group (43,8%). 58.5 45.4 43.8 40.7 37.7 36.5 36.2 34.0 31.6 31.9 29.9 28.3 25.3 22.1 18.7 16.5 13.9 14.0 10.5 9.0 5.3 5.1 4.1 1.1 During the past 4 weeks During the past 12 months More than a year ago Never visited 18-24 25-34 35-44 45-54 55-64 65+ Graph 4 - Respondents by age and time passed since their last visit to a family doctor in the FBiH, % Respondents have visited their family doctor on average 1,7 times during the past 4 weeks, respondents in rural areas more often (2,0 times on average) than those in urban areas (1,2 times on average). 43 The average number of visits during the past 4 weeks was higher among male (2,0) then female (1,4). Respondents have visited their family doctor on average 2,9 times during the past 12 months, again this number was higher among respondents in rural (3,0 times on average) than among those in urban areas (2,8 times on average). The average number of visits during the past 12 months was higher among female (3,2) then male (2,6). It was highest among respondents in the 65 and over age group (3,7) and lowest among respondents in the 18-24 age group (2,0). 2.8 2.0 2.0 2.0 1.7 1.6 1.4 1.5 1.4 1.3 1.2 FB&H Urban Rural Female Male 18-24 25-34 35-44 45-54 55-64 65+ Graph 5 - Average number of visits to a family doctor during the past 12 months The main reason for the last visit to a family doctor The highest percentage of respondents in the FBiH, over a third, indicated illness as the main reason for their last visit to their family doctor (36,9%), this did not differ significantly between respondents in different types of settlements. Over one fifth of respondents in the FBiH indicated a health check-up as the main reason of their last visit to a family doctor (22,5%), which was more often the case among respondents living in urban (24,5%) than among those living in rural areas (19,8%). One fifth of respondents in the FBiH indicated drug prescription as the main reason for their last visit (22,1%), which was again more often the case among respondents in urban (24,9%) than those in rural areas (20,1%). Other reported reasons for the last visit to a family doctor in the FBiH were: referral (7,4%), full physical examination (4,2%), doctor's note (1,3%), and other reasons (1,4%). 44 Doctor's note Other reasons Full physical examination Referral (medical 1.3 1.4 4.2 specialist, laboratory) Injury 7.4 4.1 Health check-up 22.5 Drug prescription 22.1 Illness 36.9 Graph 6 - Main reason for the last visit to a family doctor in the FBiH, % Slightly more female (24,2%) than male (20,6%) reported a health check-up as the reason for the last visit to their family doctor. Twice as many female (9,7%) as male (4,9%) reported referral as the reason for the last visit to their family doctor. On the other hand, a physical check-up was more often the reason for the last visit among male (6,3%) then female (2,2%), as well as an injury (6,7% of male and 1,8% of female). In terms of the respondents' age, illness was the most frequent reason for the last visit among respondents in the 18-24 age group (54%) and the least frequent reason among respondents in the 65 and over age group (21,6%). On the other hand, drug prescription as the main reason for their last visit to a family doctor increases in line with age, whereby it was the main reason among 50% of respondents in the 65 and over age group, and among only 2,8% of respondents in the 18-24 age group. 45 Time spent waiting during the last visit to a family doctor 65+ 20.1 0.0 21.6 1.5 50.0 5.5 0.7 55-64 24.6 1.3 25.9 2.5 37.4 6.3 0.2 45-54 23.7 4.1 37.2 4.6 21.9 7.4 0.0 35-44 28.5 5.9 43.3 4.3 8.3 7.7 0.2 25-34 20.6 8.1 46.0 5.6 3.7 10.8 3.5 18-24 15.5 6.7 54.0 7.6 2.8 7.2 4.5 Health check-up Full physical examination Illness Injury % Drug prescription Referral (medical specialist, laboratory) Doctor's note Graph 7 –Main reason for last visit to a family doctor, by age in the FBiH, % More than a quarter of respondents have been seen by the doctor on the same day when they last visited their family doctor (79,9%), this was more often the case among respondents in rural (84,7%) than among those in urban areas (73,5%). 13,0% of respondents waited 2-3 days for an appointment with the doctor. Lower percentages of respondents waited 4-5 (4,0%) and more than 5 days (3,1%) to be seen by the doctor, which occurred more often in rural then in urban areas. 79.9 66.1 63.2 15.5 13.0 13.3 4.0 3.1 0.7 1.0 0.1 0.5 I have been seen by the doctor Waited 2-3 days Waited 4-5 days Waited more than 5 days right away FB&H Urban Rural Graph 8 - Length of time spent waiting, in days, to be seen by their family doctor in the FBiH, % 46 Satisfaction with health care services Satisfaction with services provided by a doctor of medicine in the public and private sectors Respondents that have visited a doctor of medicine in the public sector, have mostly been satisfied with the services provided by a family doctor (89,4%), more so respondents in rural (91,3%) than those in urban areas (86,8%). More than four-fifths of respondents have been very satisfied or satisfied with the services provided by a doctor of general medicine (86,2%), a doctor of occupational medicine (82,5%) and a medical specialist (87,0%) in the public sector, without significant difference between respondents living in different settlement types. 91,4% of respondents have been satisfied or very satisfied with the services provided by a medical specialist in the private sector. 89.4 91.3 89.4 89.4 91.4 91.9 91.0 86.8 86.2 85.6 86.6 87.0 89.4 89.4 82.5 Doctor of family Doctor of medicine Doctor of Specialists in the Doctors in the private medicine occupational public sector sector medicine FB&H Urban Rural Graph 108 - Respondents satisfied with the services provided by a doctor of medicine in the public and private sectors during the past 12 months in the FBiH, % 47 Comparison of selected indicators in FBiH for 2002 and 2012 Respondents aged 25-64 Indicator 2002.* 2012.** Chronic diseases Total 2,1% 1,6% Respondents that have been diagnosed with a Female 1,9% 0,9% myocardial infarction at least once in their lifetime Male 2,3% 2,4% Total 1,7% 1,5% Respondents that have been diagnosed with a Female 1,5% 1,5% stroke at least once in their lifetime Male 2,0% 1,5% Total 5,4% 5,7% Respondents that have been diagnosed with Female 5,8% 5,4% diabetes at any time during their life Male 4,8% 6,0% Total 14,7% 17,1% Respondents that are currently undergoing Female 18,0% 18,6% antihypertensive therapy Male 10,0% 15,5% Respondents with a chronic disease that have had their blood pressure measured in the past 12 Total 63,4% 82,0% months Respondents with a chronic disease that have had Total 41,9% 69,2% their blood sugar measured in the past 12 months Respondents with potential hypertension (systolic Total 41,0% 39,8% pressure> 140, diastolic> 90) and/or are Female 44,8% 35,8% undergoing antihypertensive treatment Male 35,5% 44,0% Physical activity Respondents that are physically active for at least Total 15,1% 26,5% 30 minutes, to the extent of sweating and an Female 12,0% 22,1% increased breathing rate, 2-3 times a week Male 19,6% 31.0% HEALTH CARE UTILIZATION Respondents that have visited a doctor of medicine Total 54,1% 61,6% in the past 12 months Female 60,9% 64,5% Male 47,1% 57,4% MEASURED VALUES Respondents with BMI values <25 Total 36,8% 37,6% Female 38,2% 44,9% 48 Respondents aged 25-64 Indicator 2002.* 2012.** Male 34,8% 30,0% Respondents with BMI values 25-29 Total 41,0% 39,9% Female 35,9% 31,3% Male 48,4% 48,9% Respondents with BMI values >30 Total 21,5% 22,5% Female 25,0% 23,8% Male 16,5% 21,1% The Survey on risk factors of non-contagious diseases in the Federation of BiH in 2002 was conducted on a sample of adult population members aged 25-64 years. For purposes of comparison, the same age group has been observed within the Survey of the Health status of the Population of the Federation of BiH 2012. RECOMMENDATIONS The results of this survey point to priorities and activities necessary for the advancement of the health of the population of the Federation of BiH. It is necessary to increase the availability of preventive care and promotional health care services through the implementation of the existing strategic commitments: • Risk factor monitoring in family medicine teams needs to be further strengthened (anthropometric measurements, blood fat and sugar elves, blood pressure control, addictive diseases) • The scope of advising done by health workers needs to be increased (health impact of nutrition, promotion of physical activity prevention of obesity, prevention of addictive diseases, prevention of heart and blood vessel diseases, mental health and oral health) • Addictive disease rehabilitation services have to become more prominent in the public sphere (with a focus on family medicine teams and mental health centers) • Physical and regular check up and risk group immunization coverage has to be increased • Risk group screening coverage has to be increased: early detection of breast, cervical and colon cancer. The active participation of all the relevant sectors in the promotion of a healthy lifestyle is very important: • Education and information activities have to be carried out with the aim of changing habits (proper nutrition, physical activity, addictive disease prevention, mental health) • Inter-sector programs have to be implemented(healthy schools, healthy workplaces, healthy communities) • Thematic campaigns of health promotion in the community have to be organized 49 Annex 6. Stakeholder Workshop Report and Results Not Applicable 50 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR (ICR input from the FBiH only. The full report is available upon request) Course of Implementation of the Project Component 1 – Primary Health Care Restructuring Objective Objective of this component is to support PHC strengthening through development and expansion of FM and restructuring of community health care centers in order to reduce inequality in access to health care, allow health care continuum between different levels of health system based on local community needs and cost-effective way in dealing with issues such as large-scale noncontiguous diseases. This component includes the following activities: • Training in family medicine provided though the Program of Additional Training for doctors and nurses and two-year and three-year family medicine specialization program; • Equipping of Family Medicine Ambulantas with a standard set of equipment and furniture, as well as refurbishment of interior of Family Medicine Ambulantas phased out in three stages; • Equipping of family medicine training centers with IT equipment, professional literature, anatomical training dummies needed for practical classes, including missing furniture and medical equipment • Equipping of all community health care centers with network and IT equipment (hardware and software for Primary Health Care with focus on Family Medicine). Also, performance-based incentive payments were made to Family Medicine Teams for provided health promotion services in four community health care centers in Sarajevo, Mostar, Tuzla, and Zenica. Subcomponent 1.1 – Family Medicine Training Programs 1.1.2 Training for Family Medicine teams Objective Primary objective of this assignment was to enable, through program of additional training, Family Medicine teams to acquire skills needed for practicing of family medicine by PHC specialists in areas other than family medicine, doctors without specialization and nurses. Strategic direction that Federation of BiH Ministry of Health pursued was to use the one-year training program and speed up the process of including the sufficient number of Family Medicine teams into the primary health care system based on the number of population in each catchment area. 51 Another key task was to develop and improve the Family Medicine Specialization Program including academic development of faculty with the Family Medicine Departments at medical schools and with Family Medicine Training Centers in community health care centers of Sarajevo, Mostar, Tuzla and Zenica. Results achieved In order to ensure continuity of training family medicine training process during the period between the end of the Basic Health Project and the beginning of the Health Sector Enhancement Project, we continued to work with the Queens University from Canada by signing the contract in November 2005. Based on the agreed project activities and timeline, we continued with the on- the-job training for doctors and nurses (PAT program) and two-year and three-year family medicine specialization program without any gap between the academic years. During this assignment, a number of activities aimed at improving the existing family medicine training programs (both PAT and the specialization training programs) were implemented after being meticulously revised to reflect principles of modern procedures and evidence-based medicine. Continued support was also provided in the process set up and operation of the professional association, i.e. the Association of Family Physicians of the Federation of Bosnia and Herzegovina and its induction into the European and World family medicine association (WONCA). In this way, a significant number of family medicine teams was able to actively participate in WONCA's conventions and conferences and was able to exchange experience in the areas of operation and development of family medicine with their colleagues from the region and beyond. Difficulties in implementation and lessons learned Although the training was provided on-the-job and both physician and nurses were trained as a single Family Medicine, a number of Family Medicine teams have been unfortunately lost and they do not practice Family Medicine. One of the main reasons for this is a huge turnover of staff, in particular in PHC, followed by lack of staff, in particular young physicians in small towns and rural areas, as they do not stay long in PHC and basically once they go through PAT they tend to pursue other specializations within PHC or specializations offered by hospitals. On the other hand, there is no legislation to prevent such drain in PHC/Family Medicine and the Federation of BiH Ministry of Health has no jurisdiction to impose any such legislation. The key problems are the fact that there is no staff planning for Primary Health Care and the sluggish process of health sector financing reform, and the systemic approach to human resource planning at the cantonal and municipal level is unsatisfactory and it lacks comprehensiveness. Subcomponent 1.2 – Support for Family Medicine Training Centers Results achieved Since the training in family medicine (practical classes under the Family Medicine Specialization Training Program and the entire Program of Additional Training) is provided in the Family Medicine Training Centers with community health care centers (Sarajevo, Tuzla, Mostar and satellite centers located in Zenica and Bihać), it was necessary to improve capacities of these 52 centers by providing adequate equipment and learning aids. Thus the Project procured for this purpose several anatomical training dummies, professional literature, missing furniture, commuter equipment and video link equipment so that training centers can effectively communicate with each other and with other centers in the region. Also, the training centers’ premises were refurbished and this substantially improved training, both theoretical and practical. The Project also supported academic training for the centers’ staff by providing financial support to a number of employees of the training centers to complete their Master’s or PhD degrees. This effectively created necessary prerequisites to ensure that the centers can now autonomously provide Family Medicine training and CPE trainings for health professionals. Subcomponent 1.3 – Limited refurbishment and procurement of equipment for Family Medicine Ambulantas Results achieved Outstanding results have been achieved in the area of improvements to the infrastructure. Refurbishment of interior of Family Medicine Ambulantas and procurement of equipment for those Family Medicine Ambulantas was carried out in five phases, which resulted in total of 455 refurbished and 488 equipped Family Medicine Ambulantas. During this process, SPI coordinated with cantonal ministries of health and visited the sites and met with directors of local community health care centers and their staff. At the same time SPI intensively worked on activities aimed at developing tenders for purchase of medical equipment and furniture. Based on the situational needs analysis for each individual Family Medicine Ambulantas, technical specifications of the equipment and furniture were revised at each stage of the procurement process and improvement in technical terms. Difficulties in implementation and lessons learned Number of cantons, or rather their municipalities demonstrated significant willingness to try and ensure co-financing from local communities and some other sources in order to provide, built, and prepare proposed PHC sites to the extent that would allow effective support to be provided by the Project. On the other hand, support to number of originally proposed was canceled at some point due to construction and structural issues of Family Medicine Ambulantas, unresolved ownership issued, or because it was simply not cost-effective to invest in such structures. Use of medical equipment has been satisfactory, and there were no major complains to quality of the equipment procured under the Project. Designs for FM Ambulantas under Refurbishment Phases 4 and 5 While the Project was waiting for the funds for Additional Financing to become effective, the decision was made to go ahead and conduct adequate preparations regarding the continuation of refurbishment of Family Medicine Ambulantas nominated by the cantonal ministries of health under remaining phases of refurbishment and provision of equipment. As a result, the Federation of BiH Ministry of Health signed a contract with a selected design company which visited FM Ambulantas, prepared, and developed project documentation for 60 Family Medicine Ambulantas 53 proposed for refurbishment under the Additional Financing of the health Sector Enhancement Project. Difficulties in implementation and lessons learned Due to delay in effectiveness of funds from the Council of Europe Development Bank (CEB) in relation to already effective IDA loan on the beginning of the Project, refurbishment of Family Medicine Ambulantas included under Phase 1 faced a number of difficulties, including delayed payments to contractors once the works completed, and altogether led to slow down of the entire refurbishment process and significant delay in civil works. In addition to these delays, procedure to purchase medical equipment and furniture was also delayed and that resulted in delayed delivery to the Family Medicine Ambulantas that by that time were already refurbished and ready to receive the medical equipment and furniture. Preparation and implementation of other phases of refurbishment and procurement of equipment for Family Medicine Ambulantas were realized for the most parts in accordance with the timeframe as provided under the contracts with contractors/suppliers and without major difficulties. Purchase of IT equipment – Co-financing of Computerization Project in the Sarajevo Canton and selected sites in Hercegovina-Neretva Canton In the final year of the Project, the plan was to purchase ultrasound imaging devices for community health care centers in the Federation of BiH using the remaining co-financing funds provided by the Government of the Federation of BiH. SPI formally advertised the purchase of the devices in accordance with the World Bank’s procedures. However, even though received bids were evaluated, the Federation of BiH Ministry of Health decided to cancel the purchase because the Government of the Federation of BiH decided that health care sector computerization is priority. Subcomponent –Primary Health Care Informatization Support to improved information system for Primary Health Care was provided through the Project’s Additional Financing. Subcomponent under which the PHC Informatization was implemented was carried out in two stages – preparation and implementation. Under the preparation stage this subcomponent’s project plan was defined, and the focus was placed on definition, finalization, and enactment of required legislature in support of development of Health Information System for PHC. Rulebook on Definition of Primary Health Care health Information System Architecture (Federation of BiH Official Gazette No. 82/13) was enacted to ensure technological and functional requirements for implementation of Health Information System for PHC. This Rulebook also stipulated procedure for certification of software solutions to be used in the Primary Health Care. Beyond this, a situational analysis was conducted to assess both quality and quantity of IT capabilities at community health care centers in the Federation of BiH.. Also, WAN’s were installed at remote sites in Sarajevo Canton, city of Mostar, and Tuzla Community Health Care Center under continued efforts to procure IT equipment in accordance with the Decision of the Federation of BiH Government. 54 Under the implementation stage of this subcomponent SPI started procedure to purchase missing IT equipment on the basis of the situational analysis. In addition, the Rulebook on Technological and Functional Requirements for Introduction of Integrated Information System (hereinafter referred to as the Rulebook) was formally enacted and this Rulebook was used as a basis to formally certify software solution in Primary Health Care by the Expert Committee appointed by the Federation of BiH Ministry of Health in accordance with the Rulebook. Difficulties in implementation and lessons learned Due to immense workload and frequent changes in the field, findings of the situational analyses had to be revised several times by IT supervision from the time the analysis complete until such time as the purchase of network and IT equipment was carried out. In principle health institutions do not have any staff with IT knowledge and skills needed to support the situational analysis. This resulted in certain gaps and shortcomings in recording actual IT needs at number of sites. Another element that made this analysis even more complex was the fact that a huge number of sites were included in it. Although the original plan was to purchase and implement Electronic Health Record across the entire Federation of BiH, the Federation of BiH Ministry of Health did not obtain approval from two cantons go purchase the software applications for Primary Health Care and such concept was botched. Subcomponent 1.4 Communication Activities and Media Campaign Objective Under this specific project assignment, the Federation of BiH Ministry of Health defined the following priorities and objectives which were to be achieved by the media campaign: • Adequate, timely and continued presentation of new elements in the health sector reform with focus on development and benefits of family medicine model, and also the process of patient registration. • The second objective was to promote health, in particular healthy living styles aiming to improve health habits of the population. Results achieved This project assignment was realized within the timeframe as provided under the contract and was coordinated by ad hoc expert board appointed by the Federation of BiH Ministry of Health. In order to implement tasks of the media campaign, it was necessary to select a number of cantonal family medicine representatives and representatives of cantonal stakeholders who received the relevant training and then actively participated in realization of the campaign. During the realization of this project assignment, the Consultant developed timeline of activities and methodological framework and ways of interpretation of ten creative packages of the campaign titled “We Change the Health Care System, You Change Your Habits.” These packages that included a number of press conferences, production and distribution of flyers and brochures, 55 posters and billboards, radio and TV shows, jingles and TV clips that effectively focused on specific topics. As a result of the media campaign, general public’s interest increased in topics related to Primary Health Care, general health sector-related issues and it was reported that media actually increased coverage of health related topics either though media reports or special TV and radio shows. One of the effects of the media campaign is the continuation of collaboration between the Federation BiH Ministry of Health and major media outlets in the Federation BiH. Performance-Based Payment Promotion and Prevention Task for Family Medicine teams – Performance-Based Incentive Payments model testing Goal of this subcomponent was to implement performance-based contracts for implementation of relevant promotion and prevention tasks and to standardize the set of promotion and prevention tasks to be implemented by Family Medicine Teams. A standardized set of promotion and prevention services, based on life cycle needs assessment model, should serve as a foundation for formulation of result based incentives paid to Family Medicine Teams. These promotion and prevention services covered most common risk factors to the health of population in Federation of BiH including high blood pressure, obesity, physical inactivity, smoking, high blood sugar, and high blood lipids. They also included screenings for most common malignant diseases (breast cancer, colon cancer, and cervical cancer). Based on the Rulebook and contracts signed with pilot community health care centers/Family Medicine Teams in Sarajevo, Zenica, Tuzla, and Mostar, the Project tested implementation of provision of the prevention and promotion services set and incentive payments to Family Medicine Teams. During the implementation of these activities SPI made quarterly payments for the services provided on the basis of agreed payment indicators. A web application was also developed in support of this assignment’s implementation. It is important to note here that the set of prevention and promotion services has been incorporated into the new reporting forms and it can be expected that it will be possible to monitor and collect information about this segment of health services though routine health statistics system. Difficulties in implementation and lessons learned Although the Project facilitated framework for prevention activities and models for incentive payments, sustainability of these activities is unclear and implementation of this concept across all Family Medicine teams is uncertain, and this is responsibility of the cantonal authorities. Specifically, health insurance institutes should made decision and go ahead and formally include incentive payments for such activities provided by Family Medicine Teams and incentive payment for excellence in implementation of a set of prevention and promotion services in contracts with community health care centers. 56 Component 2 – Improvement of Health Sector Management Capacity Objective Overall aim of this component is to support sustained professional development in health sector and development of capacities needed for implementation of essential changes in the way health care services are provided at different levels of health system. Special focus was placed on strengthening and implementing health management training that was previously initiated under the Basic Health Project Another very important objective was to create, through implementation of this project assignment, necessary prerequisites that were to allow sustainability of the system put in place to strengthen managerial skills/health management in the Federation of Bosnia and Herzegovina. Results achieved Under the Health Sector Enhancement Project, the Federation of BiH Ministry of Health appointed both Expert and Management Board of the Health Management Center which included representatives from the three medical schools in the Federation of BiH, representatives from the Federation of BiH Health Insurance and Reinsurance, the Federation of BiH Public Health Institute, and the School of Economics. The project activities were implemented in Sarajevo, Tuzla, and Mostar Medical Schools and in the Bihać Health Care Center. The training process included gradual involvement of local experts in health management who also actively participated in the trainings, in particular in the training for family medicine doctors and nurses. As a result of this process, 600 family medicine teams received the training in change management; all directors of community health care centers and 51 member of middle management received the training in business and strategic planning; and 54 specialists from health insurance institutes received the training in contracting skills. Because this project assignment aimed at ensuring establishment of a sustainable health management system in the Federation of Bosnia and Herzegovina, SPI and the Consultant developed the specific levels and formally responsibilities for the training in health management. Thus graduate and PhD academic development in health management is now provided by medical schools, while short training including targeted courses in health management (for which there is a far greater need within the health system) are responsibility of the Federation of BiH Public Health Institute. This is why this agency formed a Health Management Unit. In fact, the Federation of BiH Ministry of Health believes this agency will be our future partner in continued process of strengthening of managerial skills, which was also planned under the HSEP’s Additional Financing. Starting from September 2014 total of 199 trainees successfully completed Basic Program, 132 trainees successfully completed Intermediate Program, and 75 trainees successfully completed Advances Program. This continued professional development process continued after the Project ended its support and the Health Management Center now provides the training in this area independently. 57 Project Component 3 – Health Policy Formulation and Project Support Objective Primary objective of this component is to provide assistance to key stakeholder agencies aimed at strengthening of the capacity for data collection and analysis in support of sectorial policy formulation and to guide priority setting and resource allocation in health sector. This component also provides grants for different innovations projects aimed at prevention of diseases and promotion of health. This component also included a survey titled “Evaluation of Primary Health Care Reform with focus on Family Medicine Model." Findings of this survey served as starting point to design and develop project activities and then implement them through the Project's Additional Financing. Subcomponent 3.1 Technical assistance in improvement of monitoring and evaluation system Further Development of Monitoring and Evaluation System Another goal of the Project’s Additional Financing was to continue to build capacities of ministries of health, public health institutes, and health insurance institutes for data collection and analysis by strengthening their routine health statistics system (improved data collection system) and its harmonization with the WHO and EU requirements (type and form of data, data standardization). The goal was also to establish framework for monitoring and evaluation of strategic goals by implementing strategic plans in the Federation of BiH health sector. The Project also worked to prepare for implementation of this framework addressing day-to-day operations of the relevant institutions (the Federation of BiH Public Health Institute and the Federation of BiH Health Insurance and Reinsurance Institute) which are required to do so under the law. Survey Objective The purpose of this survey was to evaluate reform in number of selected areas of primary health care with a focus on the development of a Family Medicine model and to review the achievement of specific goals featured in the Primary Health Care Development Strategy. Results Achieved The survey was a cross-section study which used surveying, interviewing and focus group methods. The tool which were used included questionnaire for Family Medicine physicians and Family Medicine nurses, questionnaire for patients, guidelines for semi-structured individual interview, guidelines for focus groups to include directors of community health care centers, guidelines for focus groups to include specialists with community health care centers other than Family Medicine specialist. The survey results provided several general facts: considerable progress has been made in training of staff members, equipping and refurbishing of Family Medicine clinics. Respondents 58 felt that progress has been made in improving the availability and quality of health care and that the community health care centers in the Federation of BiH are being gradually reorganized as the Family medicine model grows. However it was also felt that there are objective and subjective factors which impede the process of the implementation of the Family Medicine model in the Federation of BiH. Key findings of the survey primarily identified lack of human resources, in particular doctors in the primary health care, slow reforms of health services financing and lack of adequate allocation of funds within community health care centers, as well as the need for new regulatory mechanisms and legislation that will speed up further development of family medicine model. Based on these key findings, the Consultant provided recommendations for future actions regarding the health policies. Beyond this, the survey also served as groundwork for development of concept of the Project's extension and definition of priority subcomponents. Another survey was conducted through the Project’s Additional Financing – it was actually a follow up survey into health status of the Federation of BiH’s population Subcomponent 3.2 - Innovation Grants Objective Objectives of the grant projects included improvement of general public awareness on prevention of risk factors for most common and mass non-communicable diseases through different innovative prevention approaches, strengthening of prevention through Family Medicine teams and promotion of healthy life styles habits and life styles through introduction of new preventive methodologies for provision of health care services, better understanding and acceptance of Family Medicine model in general public as a system of primary prevention. In order to achieve these general objectives, grants were awarded to project that involved different prevention and educational programs. Results achieved Thus far total of 57 innovation projects were implemented. The projects were phased out in three cycles – 8 in first cycle, 13 in the second cycle, and 19 in the third and final cycle, whereas another 17 innovation projects were implemented through the HSEP Additional Financing which focused on prevention of risk factors in the most common mass noncontiguous diseases. Difficulties in implementation and lessons learned Each of these projects helped in the process of providing health care services, they developed cooperation between health care sector, other social sectors and non-governmental sector. What is even more important, these projects provided health care services to targeted population groups for which official health sector institutions still do not have capacities to provide. Another significant result of these projects for the health system is the fact that these projects effectively demonstrated existing weaknesses of the health system in terms of organized prevention and health care for vulnerable groups and that such type of cooperation should continue in health care provision. Also, as demonstrated by results achieved under this subcomponent, it is obvious that efforts should be made to secure adequate funds and provide health care to everyone. Beyond this, 59 the assignment exposed weaknesses of local communities and poor understanding of local authorities in terms of benefits of such interventions. Subcomponent 3.3 – Project Support During the course of the transition period, or rather in the first year of implementation, the Project utilized local consultants who were already working with the SITAP Project Implementation Unit (PIU). This resulted in a series of problems and hampered their full-time work on the Project which effectively depended on their engagement. On the other hand, procedure to recruit and hire staff for the SPI through the Civil Services Agency was taking too long in relation to the needs and dynamics of the Project. Although both Project Appraisal Document and Project Implementation Manual envisioned that all members of the PIU team would transition into the SPI as full-time employees, this did not happen although the vacancies were advertised: Full time staff of the project team has included: • Assistant Minister • Financial and accounting manager • Technical monitoring officer • Administrative assistant SPI outsourced, though consultancy contacts, an interpreter, a procurement consultant and a financial management consultant. In the course of the Project’s Additional Financing, Sector for Project Implementation hired an expert advisor to coordinate project activity. The expert advisor was already a civil servant employed at the Federation of BiH Ministry of Health. As the Federation of BiH Ministry of Health previously formed Monitoring and Evaluation Department, the monitoring and evaluation specialist left the SPI. Another employee of the Federation of BiH Ministry of Health was hired by SPI to monitor realization of refurbishment and provision of equipment for Family Medicine Ambulantas. An IT consultant was hired to coordinate activities related to the Primary Health Care Informatization and coordination of activities in the area of performance-based incentive payments for Family Medicine Teams for prevention and promotion activities they provided was conducted also by a hired local expert/consultant. During the entire duration of the implementation, the Project had seen difficulties with regards to hiring a full-time procurement specialist which has significantly encumbered performance of this very demanding Project. Generally, SPI is not staffed at the level that could effectively meet requirements and challenges of the Project and the project activities, although Staffing Rulebook calls for additional positions to be staffed. SPI still needs to fill out two vacant positions. The priority is to hire a full-time procurement specialist and financial manager, in particular because SPI was required to maintain 'double' accounting and financial reporting – in accordance with requirements of both creditors and local legislation. 60 Procurement The Agreement between Bosnia and Herzegovina and the International Development Association (IDA) on financing of HSEP defined guidelines for procurement of goods, works, and services. The Project Agreement stipulated that all purchases made under the Project should be made in accordance with agreed Procurement Plan and on the basis of the World Bank's guidelines. Procurement of goods and works has been conducted in accordance with the World Bank's Guidelines on Procurement of Goods, Works, and Non-Consultancy Services under IBRD loans and IDA Credits and Loans of May 2004 as revised in October 2006, May 2010, January 2011, and July 2014. Different types of available procurement methods have been used. Procurement of constancy contracts has been conducted in accordance with the World Bank's Guidelines for Selection and Employment of Consultants for World Bank Borrowers under IBRD Loans and IDA Credits and Grants of May 2004 as revised in October 2006, May 2010, January 2011, and July 2014. Different types of available procurement methods have been used. Financial Management SPI maintained an adequate financial management system which provided all necessary financial functions and operations, as well as an accounting system, financial reporting and audit in order to provide timely information that can be relied upon in terms of sources and expenditures of the Project. Project’s financial management has successfully sustained the following financial functions: • Definition of budget, definition of plans and controls of all financial, logistic, and administrative operations and staff-related operations including payments, purchasing and registration of Project supplies and fixes assets; • Relations with the banks and the government institutions; • Development of reports for the World Bank and the Federation of BiH Government. The accounting system has entailed a computerized accounting process that used specific accounting software. Pursuant to the Legal Agreement on the Project, within 6 months following end of each fiscal year SPI provides to both the World Bank and CEB annual audited financial project reports acceptable for the creditors. The Project has been subject to financial audits three times and the each time the external auditors provided positive opinion. Financial management provides routine quarterly reports to creditors. As regards the reporting in accordance with the legislation of the Federation of BiH, SPI has reported on quarterly basis to the Federation of BiH Ministry of Health on status of the Project. After the end of each fiscal year, independent auditors hired by the BiH Ministry of Finance and Treasury, being a signatory to the Credit Agreement, audited financial management of the Project. The audits never identified any issues with financial operations of the Project. Findings of the audits were also provided to the BiH Ministry of Finance and Treasury, the World Bank, and the Federation of BiH Ministry of Finance, as a signatory of the Subsidiary Credit Agreement. 61 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders No comments were provided by CEB on the Implementation Completion and Results Report, following the Bank’s letter of May 29, 2015. 62 Annex 9. List of Supporting Documents • Project Appraisal document for BiH Health Sector Enhancement Project, dated March 4, 2005 (Report No. 31108-BA). • Aide Memoires, Back-to-office reports, Implementation Status Reports (ISRs) and Project Implementation Plan. • Project Paper document for Additional Financing of BiH Health Sector Enhancement Project, March, 2011. • Country Framework Partnership (under completion) • Project progress reports, including Government report during the project MTR, as well as key technical assistance reports. 63 64