Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004995 IMPLEMENTATION COMPLETION AND RESULTS REPORT IDA-48960 AND IDA-52310 ON A CREDIT IN THE AMOUNT OF SDR 59.6 MILLION (US$ 93 MILLION EQUIVALENT) AND ADDITIONAL FINANCING OF SDR 60.6 MILLION (US$ 93 MILLION EQUIVALENT) TO THE REPUBLIC OF UZBEKISTAN FOR A HEALTH SYSTEM IMPROVEMENT PROJECT August 31, 2020 Health, Nutrition & Population Global Practice Europe And Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective Dec 31, 2019) Currency Unit = So’m (UZS) UZS 9516.37 = US$1 US$ 1.38 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Anna M. Bjerde Country Director: Lilia Burunciuc Regional Director: Fadia M. Saadah Practice Manager: Tania Dmytraczenko Task Team Leader(s): Elvira Anadolu ICR Main Contributor: Predrag Djukic and Sevil Salakhutdinova ABBREVIATIONS AND ACRONYMS AF Additional Financing IR Intermediate Results BCR Benefit Cost Ratio IRR Internal Rate of Return CAS Country Assistance Strategy M&E Monitoring and Evaluation CMUs City Medical Unions MFERIT Ministry for Foreign Economic Relations, Investment and Trade CBA Cost Benefit Analysis MOF Ministry of Finance CPIB Central Project Implementation MOH Ministry of Health Bureau CVD Cardiovascular Disease NCDs Non-communicable Diseases DALYs Disability-adjusted Life Years NHA National Health Accounts EMF Environment Management NPV Net Present Value Framework EMP Environmental Management Plan PAD Project Appraisal Document EU-15 15 countries of the European PDO Project Development Objective Union before May 1, 2004 FM Financial Management PERs Public Expenditure Reviews FY Fiscal Year PHC Primary Health Care GDP Gross Domestic Product RF Results Framework GNI Gross National Income RMUs Rayon Medical Unions GPs General Practitioners STEPS Stepwise Approach to Surveillance HSIP Health System Improvement TIAME Tashkent Institute of Advanced Project Medical Education IDA International Development TTL Task Team Leader Association IHME Institute of Health Metrics and WG Working Group Evaluation IMF International Monetary Fund WHO World Health Organization TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 10 II. OUTCOME .................................................................................................................... 15 A. RELEVANCE OF PDOs ............................................................................................................ 15 C. EFFICIENCY ........................................................................................................................... 18 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 19 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 19 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 20 A. KEY FACTORS DURING PREPARATION ................................................................................... 20 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 21 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 22 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 22 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 23 C. BANK PERFORMANCE ........................................................................................................... 24 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 25 V. LESSONS AND RECOMMENDATIONS ............................................................................. 26 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 41 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 43 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 44 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 53 ANNEX 6. DETAILED CHANGES OF IR INDICATORS ................................................................ 79 ANNEX 7. SUPPORTING DOCUMENTS .................................................................................. 84 The World Bank Health System Improvement Project (P113349) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P113349 Health System Improvement Project Country Financing Instrument Uzbekistan Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Ministry of Finance Ministry of Health Project Development Objective (PDO) Original PDO The overall Project Development Objectives (PDOs) are to (1) improve access to quality health care at the primary level and atRayonMedical Unions (RMUs); and (2) strengthen the Government’s public health response to the rise in non-communicable diseases(NCDs). Revised PDO The revised PDO is to (a) improve access to quality health care at the primary level and at Rayon Medical Unions; andselectedCityMedical Unions (CMUs); and (b) strengthen the Government's public health response to the rise in non-communicablediseases(NCDs). Page 1 of 84 The World Bank Health System Improvement Project (P113349) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 93,000,000 92,891,690 85,252,931 IDA-48960 93,000,000 36,767,408 30,013,414 IDA-52310 Total 186,000,000 129,659,098 115,266,345 Non-World Bank Financing 0 0 0 Borrower/Recipient 10,100,000 0 0 Total 10,100,000 0 0 Total Project Cost 196,100,000 129,659,098 115,266,345 KEY DATES FIN_TABLE_DAT Approval Effectiveness MTR Review Original Closing Actual Closing 07-Apr-2011 02-Nov-2011 16-Nov-2015 31-Dec-2016 31-Dec-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 07-Mar-2013 1.55 Additional Financing Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financing Plan Change in Legal Covenants 08-Feb-2018 93.04 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Cancellation of Financing Reallocation between Disbursement Categories Change in Implementation Schedule 18-Dec-2019 112.02 Cancellation of Financing Reallocation between Disbursement Categories Page 2 of 84 The World Bank Health System Improvement Project (P113349) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 11-Sep-2011 Satisfactory Satisfactory 0 02 18-Feb-2012 Satisfactory Satisfactory .20 03 16-Oct-2012 Satisfactory Satisfactory .41 04 08-Jun-2013 Moderately Satisfactory Moderately Satisfactory 7.10 05 08-Jan-2014 Moderately Satisfactory Moderately Satisfactory 21.27 06 01-Aug-2014 Moderately Satisfactory Moderately Satisfactory 24.87 07 28-Jan-2015 Moderately Satisfactory Moderately Satisfactory 28.95 08 08-Jul-2015 Moderately Satisfactory Moderately Unsatisfactory 37.04 Moderately 09 31-Dec-2015 Unsatisfactory 44.50 Unsatisfactory Moderately 10 14-May-2016 Unsatisfactory 53.69 Unsatisfactory 11 05-Sep-2016 Moderately Satisfactory Moderately Satisfactory 65.79 12 06-Mar-2017 Moderately Satisfactory Moderately Satisfactory 77.21 13 31-Aug-2017 Moderately Satisfactory Moderately Satisfactory 82.05 14 07-Mar-2018 Moderately Satisfactory Moderately Satisfactory 93.54 15 07-Sep-2018 Moderately Satisfactory Moderately Satisfactory 100.65 16 15-Mar-2019 Satisfactory Satisfactory 108.40 17 17-Sep-2019 Satisfactory Satisfactory 110.16 18 22-Feb-2020 Satisfactory Moderately Satisfactory 115.61 Page 3 of 84 The World Bank Health System Improvement Project (P113349) SECTORS AND THEMES Sectors Major Sector/Sector (%) Health 100 Public Administration - Health 12 Health 88 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Human Development and Gender 72 Disease Control 6 Non-communicable diseases 6 Health Systems and Policies 66 Health System Strengthening 50 Child Health 16 ADM STAFF Role At Approval At ICR Regional Vice President: Philippe H. Le Houerou Anna M. Bjerde Country Director: Motoo Konishi Lilia Burunciuc Director: Mamta Murthi Fadia M. Saadah Practice Manager: Abdo S. Yazbeck Tania Dmytraczenko Task Team Leader(s): Susanna Hayrapetyan Elvira Anadolu ICR Contributing Author: Predrag Djukic Page 4 of 84 The World Bank Health System Improvement Project (P113349) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Country-wide context 1. Uzbekistan is a landlocked country with a gross national income (GNI, Atlas method) per capita of US$1,100 in 2009. However, it is resource rich (gold, copper, natural gas, oil, uranium), with great development potential and is strategically located in the heart of Central Asia. Uzbekistan accounts for 45 percent of the region’s population, and its economic and social prospects are critical not just for the growing population of Uzbekistan (more than 62 million), but also for the whole Central Asian region. The country has a very young, rapidly growing population (at a rate of 1.5-2.8 annually since 20101) and is facing a serious employment generation challenge, especially in rural areas, where two-thirds of Uzbekistan’s population live. The Government’s approach has relied heavily on the use of state controls, planning, and direct interventions in many sectors of the economy, foreign exchange and trade restrictions, directed and sometimes subsidized credits to selected sectors, and large public investments. Uzbekistan made progress in increasing self-sufficiency in both energy and foodstuffs, and has been pursuing a policy of localization, that is, encouragement and protection of domestic production. Poverty levels vary significantly across regions in Uzbekistan, with rural areas showing more poverty than urban areas. With the tightening of budget constraints for state-owned enterprises and utility price adjustments in recent years, a newly emerging concern is the urban poor, particularly in smaller regional towns. This regional diversity also translates into persistent socioeconomic inequalities, although consumption inequality is moderate compared to other former Soviet Union countries (the Gini coefficient is estimated at around 0.392). 2. The health sector in Uzbekistan benefitted from several successful reforms following independence in 1991 , especially in primary health care (PHC). The PHC reforms were supported by the World Bank through two investment projects, Health I and Health II. Since the mid-1990s, Uzbekistan has undergone major reforms, with a health sector that has focused on restructuring PHC and establishing an emergency medical care network. Both primary care and emergency centers are part of the package of free care covered by public funds. In PHC, the country has implemented a standard approach to the training of general practitioners (GP), upgrading the rural PHC infrastructure and the allocation of equipment for rural general practice clinics, and the scope of health services to be provided. To substitute for the previously highly fragmented organization of PHC, the PHC reforms included formation of Rayon Medical Unions (RMUs), consisting of rural physician centers, multidisciplinary outpatient polyclinics and a rayon hospital, that provides basic inpatient care. Similarly, in cities, the city hospitals, multidisciplinary outpatient clinics and urban primary health centers are organized into City Medical Unions (CMUs). It is important to note that, although having inpatient facilities, RMUs and CMUs are considered as facilities at the primary level of care and their operation is fully funded by the state. The reforms have resulted in the increased satisfaction of the population with the improvement of primary health services and increased motivation of service providers as a result of improved working conditions, retraining, and availability of bonus incentives. The PHC reforms have encouraged efficiency as patients shift to lower-cost, outpatient services, and have helped orient facilities toward local needs and improved accountability. The emergency care system was entirely reorganized and now includes the Republican Center and 12 regional branches, which provide free-of-charge quality services to urban and rural populations. However, considerable challenges remain with respect to governance; in particular, financial and 1 World Bank data (https://data.worldbank.org/indicator/SP.POP.GROW?locations=UZ&page=3 ) Page 5 of 84 The World Bank Health System Improvement Project (P113349) management accountability, efficiency in public resource management and quality of service provision. Expansion of the general practice-based PHC in urban settings in place of the former PHC organization was another challenge. 3. Non-communicable diseases (NCDs) accounted for approximately 90 percent of all deaths in Uzbekistan in 2009. Diseases related to the circulatory system were the most common cause of death, accounting for 65.6 percent of age-standardized mortality (World Health Organization-WHO 2009). The mortality rate from diseases of the circulatory system in Uzbekistan in 2006 (754.2 per 100,000 population) was 1.7 times higher than the average in the European Region and 3 to 5 times higher than in the EU-15 (WHO, European Health Report 2009). About half of all cardiovascular-related mortality was due to ischemic heart disease, with one-quarter attributed to cerebrovascular-related diseases. Malignant neoplasms were the second most-prevalent cause of death in the country, while respiratory diseases accounted for about 7 percent of total mortality. Mortality from digestive diseases had increased notably in the country, much of which was due to chronic liver disease and cirrhosis. Poor access to high-quality preventive and treatment services, especially in the rural areas, had greatly affected the indicators of mortality. 4. Serious reforms of inpatient sector were pending at the time of the Project appraisal . The configuration of inpatient service provision was inefficient. Uzbekistan had a large, inefficient, poorly equipped and fragmented network of hospitals and specialized clinics, characterized by multiple vertical programs and many single-specialty facilities that had a low level of autonomy. There was a lack of clarity regarding the specific roles and linkages between the numerous hospitals and specialized care facilities. Inefficiency was largely evident in the oblast hospitals network. At the oblast level, in particular, there was fragmentation and duplication of services across too many separate facilities. In addition, the organization of buildings and departments within each hospital was usually very inefficient; there were multiple buildings on a hospital site, with poor functional layout and connection. These were also characterized by an inefficient management system due to the many disconnected vertical chains of command and reliance on vertical, technical routes for oversight. Furthermore, some of the major adverse effects of the fragmentation was the poor quality of secondary care services and the lack of an institutional and managerial focus on prevention and management of chronic illness and non-acute services for the middle-aged and older populations, compounded by a lack of coordination of services for patients with multisystem diseases that are mostly related to NCDs. Case management, based on quality improvement techniques, evidence-based medicine, and up-to-date clinical practice protocols and standards, was largely absent. The quality assessment of pediatric inpatient care, conducted by WHO in 2008, revealed a number of shortcomings in clinical management (diagnostic, treatment, and monitoring): for 35 percent of the 87 cases observed, the children did not require hospitalization; in 76 percent of the cases, the therapy prescribed was noted as ineffective and unsafe; in 81 percent of the cases observed, clinical management was considered suboptimal; and in 94 percent of the cases, health workers did not have clinical protocols, guidelines, and/or clinical standards. 5. The Health System Improvement Project (HSIP) was included in the Country Assistance Strategy (CAS) for the period Fiscal Year (FY) 2008–2011 (Report No. 43385-UZ, May 14, 2008), responding to 2 of the 4 CAS pillars, and related to (a) enabling an environment for shared growth (under the “increasing the efficiency of public financial management for more effective service provision”), and (b) improving human development and social protection through improved basic services delivery. Page 6 of 84 The World Bank Health System Improvement Project (P113349) Theory of Change (Results Chain) 6. At the time of Project preparation, a Theory of Change was not required. A Theory of Change has been developed by the ICR Team and is shown in Figure 1. Figure 1. Theory of Change/Results Chain Project Development Objectives (PDOs) 7. The original PDOs were to (1) improve access to quality health care at the primary level and at Rayon Medical Unions (RMUs); and (2) strengthen the Government’s public health response to the rise in non-communicable diseases (NCDs). Key Expected Outcomes and Outcome Indicators 8. The main expected outcomes of the Project were that most of the health needs of the population are satisfied at the primary level of care and RMUs, and that the Government takes a more active role in tackling the rise of the NCDs. 9. The achievement of the overall PDO was to be measured through the following PDO level Indicators: For the part related to improving access to quality health care at the primary level and at Rayon Medical Unions: (a) Increased proportion of diabetic and hypertension patients referred from PHC facilities to RMUs, in accordance with treatment standards Page 7 of 84 The World Bank Health System Improvement Project (P113349) (b) Improved perceived quality of PHC and secondary health care services in intervention areas For the related to strengthening of the Government's public health response to the rise in non-communicable diseases (NCDs): (c) Increased proportion of hospitals following NCD treatment standards (at least 20 treatment standards) (d) Issuance of a profile of NCD risk factors and burden of disease 10. There were also 17 Intermediate Results (IR) Indicators as follows: Component 1: Improving Health Service Delivery (1.1) Hospitals equipped with medical and waste management equipment (1.2) New treatment standards developed and adopted by the MOH (CVD, diabetes, etc.) (1.3) Urban polyclinics’ doctors receiving training under the 10-month training GP program (1.4) Health personnel at PHC (doctors and nurses) receiving training under continuous professional education (1.5) Hospital management staff receiving training on hospital management (1.6) Hospital core staff receiving training on waste management (1.7) Health personnel receiving training on clinical case management (NCDs and pediatrics) (1.8) People with access to a basic package of health Component 2: Strengthening Health Financing and Management Reform (2.1) Staff of reformed PHC and hospitals receiving training in financial management (2.2) Percent of recurrent expenditures not related to salary relative to actual expenses of: (a) PHCs, (b) Rayon hospitals, and (c) Urban polyclinics (2.3) National Health Accounts developed and published (2.4) Public Expenditure Review developed (2.5) Volume and Cost contract introduced in 3 hospitals in Fergana oblast Component 3: Institutional Strengthening for NCD Prevention and Control (3.1) Number of video clips on health education on cardiovascular risk factors (3.2) Number of people screened for (a) hypertension, and (b) diabetes through community-based screening programs in targeted areas (3.3) Percent of population in targeted Oblasts knowing cardiovascular risk factors (3.4) Public health specialists (surveillance staff) trained in data collection and analysis, including trainers, in all oblasts Components 11. The Project was intended to be implemented over a five-year period and was expected to finance goods, consultant services, training, and incremental operating costs. The Government was responsible for financing the cost of upgrading the physical condition of selected health facilities, recurrent operating costs, and taxes. The Project had four components, where the allocations were changed during the life of the Project; these are discussed in the Significant Changes during Implementation. The original allocation by Component is summarized in the table below: Table 1: Project Components Component Sub-Component Description Cost Component 1: Subcomponent 1.1. Improvement of hospital service delivery by: (a) refurbishment of at US$ 81.56 million Improving Health Hospital Services least one hundred (100) selected central rayon hospitals with up-to- (IDA credit US$ Service Delivery Improvement date diagnostic and waste management equipment and medical 76.22 million, Page 8 of 84 The World Bank Health System Improvement Project (P113349) furniture; (b) improvement of health service planning at the RMUs Government US$ including the revision of construction specifications and standards and 5.34 million) the development of referral guidelines and equipment maintenance systems; and (c) provision of study tours for health administrators to improve the skills and competencies in hospitals. Rayon-level hospitals in 6 oblasts (Andizhan, Fergana, Kashkadarya, Namangan, Republic of Karakalpakstan, and Samarkand) were initially planned to be covered by this subcomponent, including two to three hospitals in other oblasts. Subcomponent 1.2. Development of PHC by: (a) expansion of the general practice primary US$ 2.66 million Primary Health Care health care model to all urban polyclinics in pilot Fergana, Syrdarya, (IDA credit US$ Development Samarkand oblasts and in Tashkent city; (b) continuation of the ten- 2.44 million, month general practice training programs; and (c) improvement of Government US$ skills and competencies of medical personnel in early diagnosis, 0.22 million) screening and treatment of priority NCDs and development of optimal urban general practice model implementation strategy. Subcomponent 1.3. Enhancement of clinical quality by: (a) provision of on-site training of US$ 5.99 million Clinical Quality RMUs’ pediatric and internal medicine doctors and nurses in new (IDA credit US$ Enhancement clinical treatment standards; (ii) provision of training in clinical case 3.51 million, management and hospital administration; (iii) development and Government US$ introduction of up-to-date relevant and effective practical treatment 2.48 million) standards for RMUs internal medicine and pediatrics staff; and (iv) establishment of quality improvement mechanisms to monitor implementation of the new clinical treatment standards. Component 2: Aims: (a) consolidation and institutionalization of per capita based US$ 4.98 million Strengthening primary health care financing and management reforms by (IDA credit US$ Health Financing development of the strategy for the health sector financing and 4.45 million, and Management appropriate regulatory measures for the implementation of that Government US$ Reforms strategy; and (b) strengthening the role of MOH (Ministry of Health) in 0.53 million) health financing policy formulation and monitoring and local capacity building by: (i) performing health sector expenditures analysis and medium term projections; (ii) development of the national health accounts (NHA) and studies on hospital utilization patterns; (iii) provision of training relevant staff at MOH, rural primary health care clinics and RMUs on financial management; and (iv) development of a health financing information system to support the implementation of the hospital financing pilot. Component 3: Subcomponent 3.1. Improvement of health promotion through strengthening the capacity US$ 1.72 million Institutional Health Promotion of public health institutions in effective prevention and control of non- (IDA credit US$ Strengthening for and NCD Prevention communicable diseases by increasing awareness of, and changing 1.57 million, NCD Prevention and behaviors associated with, increased risk for hypertension, diabetes Government US$ Control and other chronic diseases among the population in two oblasts— 0.14 million) Kashkadarya and Fergana. The activities aimed to contribute to implementation of the National Health Promotion Plan (2011–2014) and planned to build on efforts started under the Health II project to improve the knowledge and skills of health promotion specialists in undertaking effective health promotion and behavior change activities. Specific planned activities included (i) identification of key public health issues in the population, (ii) implementation of a number of focused evidence-based interventions (e.g. reduction of salt intake, tobacco use), (iii) development of appropriate evidence-based approaches to resolve the identified issues, and (iv) evaluation of the impact of the interventions. Page 9 of 84 The World Bank Health System Improvement Project (P113349) Subcomponent 3.2. Strengthen the health surveillance systems by: (a) development of an US$ 1.58 million Strengthening epidemiological surveillance system for NCDs; (b) improvement of the (IDA credit US$ Health Surveillance public health system’s ability to use collected data in effective policy 1.41 million, Systems making and program planning; and (c) development of an effective Government US$ health promotion and disease prevention programs. 0.16 million) Epidemiological surveillance (including behavioral risk factors) for NCDs, is to be piloted initially in the two treatment oblasts with later expansion to routine national-level monitoring. The system was to be based on the WHO STEPS – Stepwise Approach to Surveillance methodology designed specifically for NCD surveillance in low- and middle-income countries. Component 4: Strengthen the capacity of MOH, the CPIB and the PIBs for Project US$ 4.38 million Project management and implementation, monitoring and evaluation, (IDA credit US$ Management environmental management pursuant to the Environmental 3.40 million, Management Framework (EMF), and procurement and financial Government US$ management through the provision of goods, consultants’ services, 0.97 million) training and incremental operating costs. US$ 102.87 million (IDA credit US$ 93 Total: million, Government US$ 9.84 million) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets 12. The original PDOs were revised during the processing of the Additional Financing (AF) for HSIP. As part of the 2013 AF (AF Project Paper dated February 5, 2013), the scope of interventions was widened to include all of the RMUs in the Project activities but also selected CMUs located in small cities and are functionally and structurally similar to RMUs. Thus, the PDOs were revised as follows: to (a) improve access to quality health care at the primary level, at Rayon Medical Unions (RMUs); and selected City Medical Unions (CMUs); and (b) strengthen the Government’s public health response to the rise in non-communicable diseases (NCDs). Revised PDO Indicators 13. With the 2013 AF, PDO Indicator 3 was revised to clarify that the measurement is related only to the use of clinical guidelines developed under the Project and adjust the terminology. The number of hospitals was also added since it was thought that an absolute number would be more demonstrative in showing the quantity of hospitals that use clinical guidelines. Also, the term “treatment standards” in the original PDO Indicator was replaced with “clinical guidelines” to reflect correct terminology. The following table provides the original, and revised PDO Indicator. Changes to the IRs are presented in Annex 6. Table 2: PDO Indicator Revised as Part of AF Original PDO Indicator No.3 Revised PDO Indicator No.3 (after the AF) Increased proportion of hospitals following NCD Increased proportion (number) of hospitals following treatment standards (at least 20 treatment NCD clinical guidelines developed under the project standards) (at least 20 clinical guidelines) Page 10 of 84 The World Bank Health System Improvement Project (P113349) 14. Revision to the baseline and target values for PDO Indicator 1 along with changes to the IR Indicators were processed during the Level 2 restructuring to the AF in 2018 (Restructuring Paper dated February 2, 2018). The explanation for the revision to PDO Indicator 1 is presented in the following table; see Annex 6 for the changes made to the IR Indicators. Table 3: PDO Indicators after the Project restructuring in 2018 PDO level Indicator after PDO level Indicator after AF Comment restructuring in 2018 1. Increased proportion of No change in wording, but The baseline data is revised based diabetic and hypertension baseline/target data revised. on the outcome of the 2015 patients referred from PHC survey that was carried out in 2 facilities to RMUs, in accordance pilot regions of Kashkadarya and with treatment standards. Fergana. Thus, the target values are adjusted to reflect results of the survey Revised Components 15. No changes to the Project design were made as a result of the 2013 AF, as the AF was provided to finance scaling up of activities that were in the original Project design as follows: Table 4: Scale-up Activities by Components Component Sub-Component Description of Scale-up activities Cost Component 1: Total IDA US$ Improving Health 171.12 million Service Delivery (Original IDA US$ 82.17 million, Additional IDA US$ 88.95 million) Subcomponent 1.1. Improvement of hospital service delivery by: (a) refurbishment of Hospital Services additional 57 RMUs and 15 selected CMUs with up-to-date equipment Improvement. (including, inter alia, diagnostic and waste management equipment) and medical furniture; and (b) provision of training to hospital managers and staff on, inter alia, hospital design, planning and management. Subcomponent 1.2. Support primary health care development by expanding the following Primary Health activities to additional urban and rural PHC facilities: (a) provision of Care Development. medical equipment; and (b) training of medical personnel in early diagnosis, screening and treatment of priority NCDs. Subcomponent 1.3. Enhancement of clinical quality by (a) provision of on-site training of Clinical Quality pediatric and internal medicine doctors and nurses from additional RMUs Enhancement. and selected CMUs in new clinical treatment standards; (b) provision of training of staff of the additional RMUs and selected CMUs in clinical case management and hospital administration; and (c) establishment of quality improvement mechanisms to monitor implementation of the new clinical treatment standards. Component 2: Support the development of a health information system for the hospital Total IDA US$ Strengthening financing pilot in preparation for potential national level rollout and 5.95 million Health Financing expand the training program for PHC facility managers in financial and (Original IDA US$ Page 11 of 84 The World Bank Health System Improvement Project (P113349) and Management health care management. 4.45 million, Reforms Additional IDA US$ 1.50 million) Component 3: Strengthen the capacity in the control and prevention of NCDs by Total IDA US$ Institutional expanding surveillance and health promotion activities in selected regions 3.38 million Strengthening for in preparation for national level roll out. (Original IDA US$ NCD Prevention 2.98 million, and Control Additional IDA US$ 0.40 million) Component 4: Strengthen the capacity of the MoH, the CPIB and the Regional Project Total IDA US$ Project Implementation Bureaus for project management and implementation, 5.55 million Management monitoring and evaluation (M&E), environmental management pursuant (Original IDA US$ to the EMF, and procurement and financial management through the 3.40 million, provision of goods, consultants’ services, training and incremental Additional IDA operating costs for the additional 24-months of implementation. US$ 2.15 million) Total: Total IDA US$ 186 million (Original IDA US$ 93 million, Additional IDA US$ 93 million) 16. The table below shows the Project costs and financing plan originally provided, and how the allocation of the 2013 AF was allocated. Table 5: Project Funds per Component with the AF (in US$ millions) Original Component AF Revised Total cost 1: Improving Health Service Delivery 82.17 88.95 171.12 2: Strengthening Health Financing and Management Reforms 4.45 1.50 5.95 3: Institutional Strengthening on NCD Prevention and Control 2.98 0.40 3.38 4: Project Management 3.40 2.15 5.55 TOTAL 93.00 93.00 186.00 17. The table below shows the Project costs and financing plan (IDA funds) and compares components under the original Project costs and the AF, and the allocations after the 2018 restructure, which cancelled US$45 million. Page 12 of 84 The World Bank Health System Improvement Project (P113349) Table 6: Project Funds per Component and Source after 2018 Restructure (in US$ millions) Original AF allocation Component project after 2018 Revised cost allocation restructuring 1: Improving Health Service Delivery 82.17 47.03 129.20 2: Strengthening Health Financing and Management Reforms 4.45 0 4.45 3: Institutional Strengthening on NCD Prevention and Control 2.98 0 2.98 4: Project Management 3.40 0.97 4.37 TOTAL 93.00 48.00 141.00 18. The AF of HSIP was restructured twice. An amount of US$45 million equivalent was cancelled during the 2018 restructuring, and an amount of US$7 million was cancelled during the 2019 restructuring (Restructuring Paper dated December 13, 2019). These amounts were the result of savings from undertaking international competitive bidding for the procurement of medical equipment. 19. The Project closing date was extended twice. Both extensions (2013 and 2018) were to allow sufficient time to complete the scaled-up activities. In total the Project was extended for 36 months. Rationale for Changes and Their Implication on the Original Theory of Change 20. Financing from the original Project was not able to fully cover all the facilities at the rayon (district) level with the Project activities due to insufficient IDA allocation at the time the original Project was prepared (FY11). Therefore, in 2013 (FY13) the Client asked the World Bank for additional funds in the amount of SDR 60.6 million (US$93.00 million equivalent) to ensure that all RMUs and selected CMUs in the country would be supported by the Project in line with the Government’s Hospital Reform Program. Thus, with the 2013 AF the Project was restructured in the following manner: (a) the PDO was revised to include selected CMUs; (b) one PDO level Indicator was reworded; (c) three IR Indicators were dropped; (d) Indicator targets were adjusted to account for the increase in scope of activities; and (e) the closing date of the Project was extended to December 31, 2018 (for 24 months). 21. The Government, through the state budget, was to cover the costs of the additional hospital reconstruction and repair, as it was envisaged under the original Project design . In addition, during preparation of the AF, the Government agreed to invest approximately US$32.5 million of state funds to cover part of related local costs, including taxes. 22. The Government was strongly committed to scaling-up the Project in order to expand coverage to the remaining seven regions of the country and to the selected CMUs nationwide to further improve access to quality health services. The institutional arrangements were to remain unchanged, as they had been effective during the implementation of the original Project. The MOH was to continue to be responsible for the implementation of all health activities. The implementing agency (Central Project Implementation Bureau – CPIB) had built strong Page 13 of 84 The World Bank Health System Improvement Project (P113349) administrative capacity and technical expertise over the previous years and was therefore well positioned to utilize additional resources. 23. The AF, which had strong support of the Government of Uzbekistan, was a better mechanism to maximize development impact and results than a repeater project, a completely new operation, or non-lending instruments. This was mainly because the AF was to use the well performing HSIP implementation and institutional arrangements as an instrument to maximize outcomes, while at the same time bringing additional funds, which was to be particularly important in view of the support it would provide to implementation of the Government’s healthcare reforms. In addition, the activities financed under the AF were to be implemented in parallel with the ongoing Project. The 24-month extension of the closing date was deemed sufficient to complete both the original and AF activities. 24. Implementation of the AF experienced delays in procurement processes throughout 2017, as did other projects in the Uzbekistan portfolio. This was caused by the changes in the Government structure and related redistribution of responsibilities and mandates among state agencies, accompanied by the establishment of new entities, involved in investment project implementation (and preparation). 25. Consequently, starting from October 2017, the World Bank had been in discussion with the Client on the potential partial cancellation of the unused balance (based on the upcoming closing date on December 31, 2018), with a view to re-committing these IDA resources for the proposed Emergency Medical Services Project (another Government priority) for which IDA-18 resources were a constraint. 26. On January 24, 2018, the Government of Uzbekistan formally requested (a) a cancellation of US$45 million from the AF of HSIP; and (b) extension of the Credit Closing Date to December 31, 2019; the latter with a view to continuing implementation and completion of ongoing procurement processes, including delivery and installation of the medical equipment. Based on the World Bank’s assessment, achievement of PDO and implementing the pending activities was realistic with an extension of the closing date by 12 months. Therefore, a second level restructuring of the AF of HSIP consisted of: (a) Revision of the project component costs and cancellation of US$ 45 million equivalent (described above); (b) Revision of the RF, which included changing the target values for some PDO and IR Indicators, as well as dropping three of the IR Indicators (described above); and (c) Extension of the closing date of the Project by 12 months (from December 31, 2018 to December 31, 2019). 27. On November 22, 2019, the Government of Uzbekistan formally requested to cancel a further US$7 million from the AF of HSIP. At that time all activities had been completed, and there had been savings from the medical equipment tender which could not be used by the closing date. As US$45 million has already been cancelled during the 2018 HSIP AF restructuring, the Government asked for the cancellation of a further US$7 million. Therefore, a second level restructuring in 2019 consisted of a revision of the project component costs and cancellation of US$7 million equivalent. Page 14 of 84 The World Bank Health System Improvement Project (P113349) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 28. Relevance of PDOs is rated as Substantial. At the time of approval on April 7, 2011, the original Project was relevant and consistent with development priorities of both the Government (as confirmed through several strategic documents, such as State Program “On main directions on further deepening of reforms and implementation of the State Program on Healthcare Development”) and the World Bank CAS (FY08-FY11). The Project design addressed important Government priorities and followed the footsteps of the previous World Bank-supported health operations by taking into account key lessons learned. At the time of this ICR, the PDOs are broadly consistent with the World Bank’s current Country Partnership Framework (FY16-FY20), through its Focus Area 3 (Public Service Delivery) and its Objective 3.1 (Improved access to quality education and health services). It is also consistent with another area of the current CPF which is support to efficiency of health services delivery. The Project has had significant impact in supporting implementation of health reforms in the country and was used strategically to actively support health reform activities and build up capacity in the MOH. Furthermore, reforms initiated by the first and second health projects supported by the World Bank were rolled out and brought to the next level by the HSIP. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome Objective A: Improved access to quality health care at the primary level and at RMUs and selected CMUs 29. Quality health care has been provided by reconstruction and equipping all RMUs in the country and selected (15) CMUs, by training and equipping staff to use newly purchased equipment and by improving patient care by developing, adopting and using clinical and treatment protocols. Although RMUs and CMUs contain inpatient facilities they are part of the PHC provision in Uzbekistan. 30. By 2016, 22 clinical protocols were developed and approved by the MOH in the areas of gastroenterology, cardiology, pulmonology and endocrinology, surpassing the original target of 20. In order to produce clinical protocols, MOH selected 11 local experts that were trained by WHO and the U.S. Centers for Disease Control and prevention on the methodology of development of clinical protocols. Together with in-country health staff they developed 22 clinical protocols that included diagnostic and treatment standards for GPs with criteria for referral of patients for in-patient care and indicators for health assessment and rehabilitation after discharge. Implementation of those clinical protocols and treatment standards have been monitored. In 2017 and 2018, local consultants hired by the Project monitored implementation of clinical protocols into practice in sample of RMUs covering entire country. They assessed knowledge and skills of physicians and nurses providing both inpatient and outpatient care in the RMU/CMU. Also, a sample of case reports and outpatient records were audited. Furthermore, RMU/CMU staff nationwide were trained for conducting internal monitoring of clinical protocol implementation in their facilities. 31. Care of hypertension and diabetic patients at the primary level was chosen as a PDO level Indicator, as an example for the usage of clinical protocols. The logic behind the Indicator is that, without the developed clinical protocols, it was noted that patients in general were under-referred (not being referred when their condition justifies) or over- referred (being referred to a higher level of care when there is no medical justification for it). Addressing this situation requires introducing of appropriate clinical protocols, providing equipment to facilities and proper training of the Page 15 of 84 The World Bank Health System Improvement Project (P113349) health staff, which would increase the quality of care provided. It was shown that the proportion of referred hypertension and diabetic patients from PHC to RMUs, based on treatment standards, was constantly on the rise – in 2015 43.8% of hypertensive and 72.2% of diabetic patients were referred to RMUs according to treatment standards, while in 2019 (at the end of the Project) those proportions were 87.8% and 87.5% respectively, over-achieving the target of 80%. 32. Purchasing of equipment for all RMUs in the country and selected CMUs was funded by the Project. In total 748 health facilities were provided with new medical equipment. Some facilities needed physical construction or reconstruction; this was financed outside of the Project, from Government funds, but the Project supported the design and layout. Civil works were completed in 391 health facilities. Unfortunately, waste management equipment was not purchased until the end of the Project. 33. A medical equipment maintenance system had been under development since the Health II Project, when the first pilots of the medical equipment maintenance mechanisms were done. Under this Project, the Republic Training and Production Center for Maintenance of Medical Equipment was established, and regional branches are being established. That Center and its branches and subsidiaries will provide medical equipment maintenance services to entire country. Discussion with the RMU management indicate that they are facing no problems with the medical equipment maintenance, as it has been provided when necessary in a timely manner. 34. Staff were trained extensively in facility management, waste management, and care management. Waste management training was provided to 933 core hospital staff, surpassing the target (516) by over 80%. Training in clinical case management in pediatrics was provided to 3,833 physicians and 6,158 nurses, significantly overachieving the original targets of 1,002 and 2,064 respectively. Also, over 3,000 physicians (against a target of 3,000) in urban and rural PHC facilities undertook a 10-month training program for GPs, strengthening their capacity to provide quality health care services at the primary level. Furthermore 14,824 medical doctors and 50,018 nurses attended training as part of their continuing professional education, surpassing the targets at 9,400 doctors and 35,000 nurses. The training, aimed at improving the quality of services provided, strengthened the capacity of health staff. The lead organization for provision of trainings was the Tashkent Institute for Advanced Medical Education (TIAME), that also took part in designing and providing training for trainers who conducted training around the country. Training outside of the capital Tashkent, were provided in training facilities established in regional hospitals, as separate organizational units, equipped, refurbished and financed by the Project. It is understood from MOH that these training facilities will continue to be used. 35. It is not only quality, but access to services, which improved with the support under the Project, namely new equipment, training and protocols. Several examples were provided by one of the regional Departments of Health during a meeting: in the past, if a child swallowed something it had to undergo surgery. Now, with the new equipment, it is not necessary, and the item can (in most cases) be extracted without surgical intervention. Similarly, patients who are bleeding due to liver cirrhosis could die, while now there is equipment available and people in RMU trained to stop the bleeding. 36. Performance and sustainability of the reformed PHC provision through RMUs also depends on the capacity of staff not directly involved in health care provision, especially during health financing reforms. In that regard 255 RMU staff working in financial management were trained on that subject, fulfilling the target originally set. 37. The Project was also effective in strengthening MOH capacity by supporting the capacity building and development Page 16 of 84 The World Bank Health System Improvement Project (P113349) of two National Health Accounts (NHA) reports and two Public Expenditure Reviews (PER), fulfilling these targets. Production of these reports required significant collaboration with international partners (i.e., WHO) and capacity building within MOH. The NHA reports and PERs developed are providing valuable input in analyzing and monitoring the performance of the health system in Uzbekistan. A third NHA report is being prepared and there exists a draft of the resolution of the Cabinet of Ministers of the Republic of Uzbekistan on institutionalization of a national system of health accounts2. Nevertheless, the institutional capacity for the development of PER and NHA should be further strengthened so that this useful analysis is routinely carried out in Uzbekistan to support evidence-based policy decision making. 38. Health financing reforms is the only area where the Project activities did not manage to attain full success. Nevertheless, the Project supported developing the groundwork for health financing reforms, with much work done, especially in the area of primary care financing. All 155 planned urban PHC facilities (urban family polyclinics) converted to a per capita financing system that has already been applied to all RMUs. Nevertheless, despite hard work and all the preparatory activities and ultimate simulation of the mechanism, the Volume and Cost contracts were not introduced in the three pilot hospitals within RMUs in Fergana oblast as planned. It is the only IR Indicator which was not reached during the Project. Lack of adequate in-country capacity in such a complex payment mechanism and its implementation has been singled out by various stakeholders as the main reason for not conducting the Volume and Cost contracts pilots. However, the piloting, finetuning and rolling out of the new payment mechanism is still in the plan of the Government and is expected to take place. 39. The PDO level indicator related to perceived quality of PHC and secondary health care services in intervention areas included assessment of infrastructure and waiting times for doctor’s consultations. It showed constant improvement over the course of the Project. For PHC, the baseline value (in 2011) of satisfaction of patients was 85.5%, 82.7% in 2015, and at the Project end it was 90.7% against a target of 90%. Satisfaction with secondary health care rose as well, from 69.5% in 2011, to 86.8% in 2019, against a target of 85%. Objective B: Strengthened the Government’s public health response to the rise of NCDs. 40. MOH capacity in NCD surveillance has been strengthened through training of public health specialists (surveillance staff) and health promotion specialists. Public health specialists were trained on methodology of collection and monitoring of NCD risk factors data, based on WHO STEPS principles. Health promotion specialists were trained on healthy lifestyle, and the addictions of tobacco and alcohol, etc. 41. Capacity strengthening of the health system with respect to health personnel in RMUs and CMUs were conducted through training on NCD case management, among other training provided to the health staff, and also through development, adoption and implementation of 22 clinical guidelines for different conditions, including integrated management of major NCDs. Development and application of clinical guidelines/protocols, most of which related to NCDs, and monitoring of their implementation in individual facilities as described above, greatly contributed to better detection and reporting of NCDs to Government authorities. 42. An example of the positive contribution of newly purchased medical equipment to NCDs was provided by management of the TIAME during a meeting, who noted that with the new laboratory analyzers provided to all RMUs enabled greater analytic capability than just plain level of blood glucose, which improved the detection of diabetic 2 See Government ICR, Annex 5. Page 17 of 84 The World Bank Health System Improvement Project (P113349) patients. Previously, the incidence of diabetes in Uzbekistan was reported as 5%, while the international average is between 25 and 30%. With the new laboratory equipment, appropriate training, application of guidelines and provision of services to the population, detection of diabetic patients has improved, and the incidence is now moving toward 25%, which is much closer to the international average. 43. All the PDO Indicator targets have been achieved, as presented in the table below: Table 7: Achievement of PDO level Indicators Indicator Actual value achieved Link to individual PDO Increased proportion of diabetic and ACHIEVED – 87.8% for hypertensive hypertension patients referred from patients and 87.5% for diabetic patients PDO1 PHC facilities to RMUs, in accordance (Target was 80%) with treatment standards Improved perceived quality of PHC and ACHIEVED – 90.7% for PHC and 86.7% for secondary health care services in secondary health care services PDO1 intervention areas (Target was 90% and 85%) Increased proportion (number) of ACHIEVED – 172 hospitals (157 RMUs and hospitals following NCD clinical 15 CMUs) PDO2 guidelines developed under the project (Target was 172) (at least 20 clinical guidelines) Issuance of a profile of NCD risk factors ACHIEVED – one report published PDO2 and burden of disease (Target – one report) Justification of Overall Efficacy Rating 44. The achievement of the PDOs (efficacy) is rated as Substantial. All four PDO Indicators were achieved or surpassed, while only 1 out of the 13 final IR Indicators was not achieved. C. EFFICIENCY Assessment of Efficiency and Rating 45. The efficiency of the Project is rated as Substantial The analysis of efficiency examines three areas to assess whether the costs involved in achieving Project objectives were reasonable in comparison with both the benefits and with recognized norms. These include: (a) a review of aspects of the design and implementation that helped contribute to, or negatively affect, efficiency (operational efficiency); (b) an estimation of the Project efficiency using a standard cost-benefit analysis (CBA) which is detailed in Annex 4; and (c) a financial sustainability assessment taking into account the overall health sector spending and the country’s fiscal prospects. 46. From the operational perspective, the Project had many features to help assure allocative efficiency, i.e., spending resources where return on investment is potentially highest. These features included focuses on: (a) health promotion, prevention, and PHC, including the first level hospitals, which – if effective – could address a majority of health issues and save higher level hospitals from having to provide costly services; (b) clinical quality of care, an inherent weakness in the system and a necessary step for converting services to better health outcomes; and (3) most critical health conditions (NCDs, child health, etc.) which account for a major share of disease burden in the country. The actual implementation experienced delays, which represented an opportunity cost of funding already set aside Page 18 of 84 The World Bank Health System Improvement Project (P113349) for the Project. The slow and partial disbursement means that the expected results were not produced early on and, in some cases, in full. 47. The CBA adopts the same methodology as the analysis at appraisal, which remains adequate and relevant. Benefits were derived mainly from health gains produced by activities in Components 1 and 3, which were measured in terms of Disability Adjusted Life Years (DALYs) averted. Input data has been updated and most assumptions are revised to be on the more conservative side, given the implementation experience. Under the most realistic scenario, the main model shows a net benefit of US$879.2 million in real terms, which translates into a net present value of US$176.5 million and an internal rate of return of 199%. The benefit-cost ratio is estimated at 13.7, which means that for every dollar invested, the Project generated benefits valued at 13.7 times higher. Estimates of the Project’s benefits vary with sensitivity analysis but remain highly positive. 48. The fiscal impact of the Project is assessed against health sector spending, using data from 2012 to 2017, the latter being the latest year for which health financing data is available . Even in its peak year of disbursement (2016), the Project financing only accounted for 1.77% of the general government spending on health. The cost for maintaining the Project achievement beyond its closing date is significantly small and should not impose any appreciable impact on the fiscal envelop of the country. D. JUSTIFICATION OF OVERALL OUTCOME RATING 49. Overall Outcome of the Project is rated as Satisfactory. This is based on the relevance, the efficacy, and the efficiency all being rated as Substantial. In summary, the Project performed successfully and there were only minor shortcomings in its relevance, the achievement of its objectives, and its efficiency. E. OTHER OUTCOMES AND IMPACTS Gender 50. By strengthening PHC and improving the quality of the care offered at that level, the Project has a clear gender impact. Reconstructed, refurbished and newly equipped RMUs and CMUs throughout the country now have a constituent delivery facility, making family planning and delivery services more readily available; family planning is especially important, bearing in mind the fertility rate in Uzbekistan. Furthermore, an example of increased access and quality of care for women has been provided by one of regional health departments, who stated that the availability of colposcopes and other equipment provided to PHC level (RMUs) enabled earlier detection of cervical cancers. In the past, cervical cancers were usually detected at a later stage (e.g., stage 4), but are now diagnosed in earlier stages, which provides for a better prognosis. Additional training of health staff, including the 10-month trainings for GPs providing services outside of main facilities, further improved the quality of the services provided. Among the training on clinical case management provided, pediatrics (along with NCDs) was a particular focus. Institutional Strengthening 51. The Project had a clear focus on institutional strengthening, as expressed in the PDOs. The strengthening of MOH’s capacity was through capacity building for producing NHA and PERs, that gave decision-makers a better insight into the financial state of affairs in the health sector. Furthermore, the Project supported institutional strengthening in monitoring the burden of disease, including NCDs, thus enabling more coordinated and informed response in combating the NCD epidemic. Page 19 of 84 The World Bank Health System Improvement Project (P113349) Poverty Reduction and Shared Prosperity 52. The Project contributed to poverty reduction by strengthening PHC at RMU and selected CMU levels, especially since the benefit package for the population in Uzbekistan covers PHC level services but does not cover secondary health care (oblast hospitals). Before the Project, referrals from primary to secondary care facilities were done without the benefit of any treatment or clinical protocols. Reforms in primary care, equipping and training staff to use the new medical equipment, and especially the introduction of protocols for both primary and secondary levels of care, including for NCDs, prevented unnecessary referrals to higher levels of care and also reduced unnecessary expenses for patients. Reforming, refurbishing and equipping the RMUs, together with extensive training programs enabled the population, especially in rural areas, to access quality care. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 53. The Project was in line with the long-standing engagement in the health sector of Uzbekistan , and it was designed on the basis of results from the Health I and Health II Projects. Health I (1998-2004, US$30 million) was the first health sector operation in Uzbekistan and was part of a broader rural social infrastructure development approach of the Government of Uzbekistan and evolved with the Government’s PHC reform strategy (1996). The reform focused on rationalization of excessive infrastructure at primary level of care and creation of a sustainable network of integrated PHC services. Health I supported the Government’s efforts in three pilot oblasts, focusing mainly on rural physician centers and initiating development of general practitioners and family medicine staff by establishing a 10-month training program. Health II (2004-2011, US$40 million) also focused on PHC and on rolling out successes of Health I to other regions. During Health II, rural physician centers in other regions were refurbished and equipped, the GP training program was strengthened, rural PHC sector financing and management reforms (per capita financing) were implemented while urban PHC financing and management reforms were piloted. Therefore, the HSIP addressed, in more systematic way, the challenges of the provision of quality outpatient and inpatient services on primary level of care to the country’s rural population, reforming the way those services are organized, provided and paid for, general public health education and promotion and behavioral change of the population. 54. The model of PHC implemented in Uzbekistan is based at the rayon level . Previously, parts of the system providing PHC consisted of different rural physician centers, different multidisciplinary outpatient clinics (polyclinics) providing outpatient specialist care and rayon hospitals; their operation is funded by the state. However, the care provided was of variable quality and the communication and collaboration of different facilities covering the same population was lacking (e.g., outpatient polyclinics were mostly stand-alone clinics that were not considered part of a rayon hospital). Together with the establishment of 10-month GP training course, introduction of per capita payment system and restructuring, refurbishing and equipping of the facilities, organizational changes happened as well. Rayon health departments have been renamed and transformed into RMUs that comprise of rayon hospital, multidisciplinary outpatient clinics and primary care units (rural physician centers), making them a single functional entity. RMUs also provide emergency services and have an emergency medicine unit within their inpatient facility (hospital) and provide sanitary-epidemiological services. Similar organizational changes happened in cities, with the establishment of CMUs as functional entities, comprising a city hospital, multidisciplinary outpatient clinics and urban primary care centers. 55. Planned financing mechanisms of RMUs/CMUs will be combined. The source of funding for RMUs/CMUs will Page 20 of 84 The World Bank Health System Improvement Project (P113349) remain with the state, however provider payment mechanism will be different for different parts of the unions. Rural physician centers and multidisciplinary outpatient clinics have been paid on per capita basis nationwide since 2013, while inpatient services are planned to be covered by volume and cost contracts based on a case. 56. The Project concept at the design stage set realistic objectives. The project design aimed to continue the reforms and activities from the two World Bank supported Projects (Health I and Health II). It envisaged further strengthening of the primary and secondary care at the rayon level by supporting network optimization initiated in previous projects through provision of equipment and providing training for health staff. Financing and management reforms initiated under the previous Projects were to be strengthened by rolling out the primary care financing reforms to all primary facilities in the country. Also, it envisaged taking a step further in the health financing reform at the secondary level of care. Lastly, the Project would also focus on the capacity building of the health system to strengthen tackling the NCD epidemic in Uzbekistan. The design was appropriate for the health sector’s needs and consistent with the country’s development strategy to implement programs for poverty alleviation. 57. The Project objectives, scope and institutional arrangements were appropriate . The Project design incorporated the main lessons of the previous health projects relating to policy, sustainability and implementation capacity issues. The Government’s decision to have the CPIB staffed with seasoned local technical consultants that had worked in the previous Projects, and to coordinate the implementation of HSIP was appropriate and led to more efficient use of funds, better planning and synchronization of activities. 58. The RF was well designed, with a relevant and manageable number of Indicators. However, some of the Indicators selected proved later to be difficult or impossible to follow and monitor; these were dropped from the RF. This emphasizes the importance of careful and adequate selection of the Indicators at the Project preparation phase, but also on the importance of close monitoring and timely actions related to changing or dropping Indicators with the aim to be able to correctly present the Project activities and achievements. B. KEY FACTORS DURING IMPLEMENTATION 59. As indicated above, AF was requested by the client in the amount of SDR 60.6 million (US$ 93 million equivalent) and processed in 2013. Concurrently, the PDO was revised to cover inclusion of CMUs, along with RF and an extension by 24 months of the Closing Date (to December 31, 2018). The AF was necessitated by the unavailability of sufficient IDA funding at the time of Project preparation to undertake all the envisaged activities. 60. Significant delays in implementation of Project activities were experienced from the beginning and continued until the end of the fourth year of implementation. This resulted in the low disbursement rate (24% at Mid-Term Review) after 4 years of implementation; the DO and Implementation Progress rating were downgraded to Moderately Unsatisfactory and Unsatisfactory respectively in November 2015 (the Mid-Term Review) until September 2016. Reasons for the delays were twofold, and the result of the in-country environment: (a) the delay in physical construction/reconstruction of the facilities and the approval of bidding documents for the equipment; and (b) the lengthy practice of price verification for import contracts that was in place until 2017. 61. The Government was late in undertaking the necessary physical construction/reconstruction of the facilities , for which the Project provided the layout along with the description of services to be performed in the facility. Both aspects required the approval by an inter-ministerial commission that included MOH representation, among others. In addition to the exceptionally long time for the commission to provide the relevant approvals, that actual Page 21 of 84 The World Bank Health System Improvement Project (P113349) construction/reconstruction also took a much longer time than originally expected. The requirement that the tender documents for the facility equipment also had to be approved by the same inter-ministerial commission resulted in a much longer than expected lead time, and consequently delayed the timing of the equipment procurement which affected disbursement rates. 62. Another reason for delays related to the practice at the time in Uzbekistan of price verification of already signed import contracts. It particularly affected technical assistance contracts, especially ones involving foreign consultants; this affected activities under Component 2 that contributed to downgrading of the Project during the Mid-Term Review. The price verification, which is in violation of the procurement practice of the World Bank, was being carried out by the Ministry for Foreign Economic Relations, Investments and Trade (MFERIT); MFERIT also needed a very long time to come to a decision, which resulted in huge delays in contract executions, withdrawal of bids and non- materialization of contracts in few cases and the risk of misprocurement declaration in several cases. Furthermore, this imposed a significant additional burden on CPIB staff time needed to address such administrative requirements. It is important to note that this approach, by MFERIT, was applied all World Bank supported projects in Uzbekistan and was not HSIP specific. After political changes of 2017, the new Government adopted a reform agenda aiming to modernize the bureaucratic system. In accordance with the new reform agenda MFERIT was restructured and the price verification practice was abolished. 63. The Mid-Term Review was conducted from November 16-25, 2015. Issues causing delays in HSIP implementation and the low disbursement rate were listed, analyzed and steps for improvement agreed. In the months after the Review, the situation improved, disbursement picked up, and in September 2016 both PDO and Implementation Progress ratings were upgraded to Moderately Satisfactory. 64. Ultimately, procurement of the medical equipment for all the planned facilities resulted in significant savings. Consequently, in October 2017 the Client started discussing with the World Bank the potential partial cancellation of the unused balance. This was in light of the upcoming (December 31, 2018) closing date and Government’s desire to re-commit these for another priority - an Emergency Medical Services Project - for which IDA-18 resources were constrained. 65. The Level 2 restructuring of the AF of HSIP Project was processed in 2018 (Restructuring Paper dated February 2, 2018). The restructuring consisted of (a) cancellation of US$45 million equivalent; (b) revision of the RF that included revision of the target values for some Indicators and dropping three IR Indicators; and (c) extension of the closing date for an additional 12 months (to December 31, 2019). The World Bank’s assessment at the time was that achievement of the PDO and implementation of the pending activities was realistic within the extension. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) 66. Design, implementation, and utilization of M&E are rated as Modest for reasons elaborated below. 67. Design. The PAD defined a reasonable set of PDO and IR Indicators, which reflected the PDO and Project components. PDO Indicators selected allowed for the monitoring of the achievement of the PDOs and the IR Indicators aimed to reflect efforts within the Components. During the life of the Project, of the initial 17 IR Indicators, 6 were Page 22 of 84 The World Bank Health System Improvement Project (P113349) dropped for various reasons (3 in 2013 and 3 in 2018). Having in mind that the Project did not change its structure or scope of work, dropping that many Indicators could give rise to the adequacy of the initial choice of IR Indicators. The PDO Indicators were not changed in number or scope, but one was slightly reworded for clarity and baseline and target values were updated for another. 68. All elements of the M&E framework were implemented as designed. At the start of the Project, there was one IR Indicator that did not have a baseline value; the Indicator was later dropped. Nevertheless, scarcity of other available data that would help in assessing the results and outcomes of the Project as well as the observed system of data collection raises questions of the reliability of data that were collected. Furthermore, it would be expected that, in the course of the Project, some additional data for measuring the quality of care would become available, which was not the case. 69. Utilization. The M&E under the Project was used as a management tool to evaluate the status of implementation of activities. In addition, the opportunity provided by the AF was used to introduce Core Sector Indicators in the RF early on in the Project implementation. 70. Justification of Overall Rating of Quality of M&E. The Project’s M&E framework was sufficient to assess achievement of Project objectives, to inform the direction of the Project, and was effective for strategy development and/or future projects. However, there were shortcomings in the M&E system’s design and implementation as described above. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 71. OP 4.01 was triggered from the start, since the Project indirectly supported the rehabilitation of health facilities financed by Government’s budget. The Environmental Management Framework (EMF) was prepared, providing environmental management guidelines, including those for waste management. Based on the EMF, a site-specific Environmental Management Plan (EMP) was prepared prior to beginning construction at each site. 72. OP 4.12 (Involuntary Resettlement) was triggered during the implementation, as noted in the MTR. This was not triggered initially as the civil works at health facilities were to take place without changes to the footprint of the medical facilities with no consequent physical or economic displacement or land acquisition. However, during the Project implementation the scope of civil works, in a number of cases, expanded, and went beyond the existing floor plans, requiring relocating some small businesses. In order to ensure compliance with OP 4.12 the Project prepared a set of the following resettlement instruments: Resettlement Audit and Social Action Plan, Resettlement Policy Framework, and Resettlement Action Plan. An independent audit was conducted in 2016 to assess all private sites located within the potential territory of the RMU. Results of the audit showed that no involuntary resettlement took place and that only two (2) private sites potentially kept the risk to be involuntary resettled. Upon completion of the social audit, construction on the territory of those two sites was not allowed. 73. Financial Management (FM) arrangements at CPIB were quite satisfactory. Overall, the CPIB had acceptable budgeting and planning capacity. The CPIB prepared annual budgets on a timely basis which were approved by the Ministry of Finance (MOF) and entered into the accounting system by the FM staff of the CPIB. 74. CPIB’s internal controls system was assessed on a regular basis by the World Bank’s FM Specialist and was found in general to be reliable and capable of providing timely information and reporting on the Project. In particular, the CPIB performed monthly formal reconciliation of disbursement data with the Project’s accounting records; this was Page 23 of 84 The World Bank Health System Improvement Project (P113349) done using the World Bank’s Client Connection system. Proper data back-up arrangements were followed. The CPIB utilized the 1C accounting software that was specially designed for World Bank-financed projects; the software was able to generate Financial Management Reports and statement of expenditures. At the end of each quarter, the CPIB developed Financial Management Reports and submitted them to the World Bank; these reports were usually submitted on time and found to be satisfactory. 75. Independent audits were carried out regularly during the life of the Project, in line with the covenants. 76. There were problems with procurement activities, that led to significant delays, but these were outside the control of the Project. Two issues were identified as major ones affecting the Project procurement: practical implementation of the principle of seniority of World Bank’s procurement guidelines, and the contract registration requirement of Government agencies (i.e., MFERIT) which are described above. 77. The principle of seniority of World Bank’s procurement guidelines was difficult to implement . The Financing Agreement signed between the Republic of Uzbekistan and the World Bank has the status of the international treaty, and therefore any provisions and references therein prevail over the national legislation in cases of conflict. Concurrently, the provisions of the World Bank Procurement Guidelines referred to in the Financing Agreement should prevail when there is a discrepancy with the national procurement procedures. There was a continuous effort from the World Bank team to call Government counterparts for adherence to this principle throughout the life of the Project. The CPIB had to shuttle between the World Bank and MOH trying to find a balance by incorporating partial World Bank comments and complying with local requirements. Repeated instances of non-acceptance of World Bank comments on the bidding documents or bid evaluation reports also led to increased risk of potential misprocurement cases and further protracted the procurement process. Thus, processes dragged for weeks and months resulting in delays of contract awards, disbursements and affecting overall Project implementation. It is worth noting that this was not unique to HSIP. C. BANK PERFORMANCE Quality at Entry 78. As described earlier, the Project: (a) objectives and scope were appropriate for the stage of development of the sector in the country; (b) built on the implementation experience of the previous World Bank-financed health operations in Uzbekistan and elsewhere in the Europe and Central Asia Region; (c) addressed the Government’s objective of cost-containment while promoting improvement in quality and access to PHC; and (d) supported key elements of the CAS and Government’s strategic documents for the health sector. The World Bank accurately judged the lack of support or possibility for more engagement in the hospital sector at the oblast (province) level, so the focus remained on rayon (district) level facilities. The Project components remained unchanged and there were only minor changes to the activities after eight years of implementation, confirming the relevance of the Project throughout implementation. It should be noted that more than 90% of the funds were planned and spent on medical equipment; it must be understood, however, that the state of the health facilities in Uzbekistan was very poor and there was decades of neglect in terms of investment in equipment at the primary level. Furthermore, with the funds for the procurement of equipment, the World Bank was assisting the Government in its reform of the provision of primary care, while at the same time optimizing the vast network of health care facilities, with the aim of providing better and more accessible services to the population. Page 24 of 84 The World Bank Health System Improvement Project (P113349) 79. Implementation arrangements. The Project was executed under the direction of the MOH; overall Project oversight was assumed by a structural unit chaired by the First Deputy Minister as stipulated in point 6 of the Decree of the Cabinet of Ministers of the Republic of Uzbekistan, No. 229 of August 12, 2009. The Oblast Hospital Program Oversight Committees were established at the local government – Khokimiyat – level in all oblasts and the Council of Ministers of the Republic of Karakalpakstan. The Committees comprised of representatives of the oblast health department, the oblast and rayon finance departments, and oblast administration. The main role of these Committees was to oversee Project implementation progress, ensure timely coordination with the State Hospital Investment and Recurrent Program, and decide on actions to address issues that arose during implementation. To ensure that Project objectives were reached, the Working Groups (WGs) established by the MOH under the Health II Project continued to function for specific Project components. The WGs comprised appropriate leading specialists from the Ministry of Economy, MOF, MOH, and other related organizations, and each WG was managed by an appointed WG leader. The activity of all WGs were coordinated by respective deputies to the Minister of Health. Decisions made by WGs became effective after their approval by the MOH. The CPIB carried out the overall coordination with all Project participants, providing with necessary documents and other technical assistance. Quality of Supervision 80. From the outset it was recognized that the Project would require close monitoring and a hands-on supervision approach to ensure successful implementation. Supervision was intense, with frequent technically adept missions, ensuring consistency of the policy dialogue and the messages delivered to the Government. The Task Team Leader (TTL) responsible for processing the Project for approval continued as TTL for four years after Credit Effectiveness. The second TTL guided Project implementation until its closing date, i.e., for another four years. The engagement of the World Bank staff in-country provided helpful continuity and responsiveness from the World Bank team. 81. For the most part, teams were responsive to client needs and demonstrated flexibility in adapting to evolving priorities within the parameters of the PDOs. Equally, teams were diligent in their communication with Government and World Bank management, providing up-to-date information and analysis on the status and impact of Project activities, issues encountered, and suggesting options to address issues that arose as a result of evolving needs. The fact that both TTLs were fluent in Russian, understood the Soviet and post-Soviet systems, helped build trust with the client and made it easier for the World Bank to convey challenging messages. 82. Project monitoring and supervision was adequate. The Mid-Term Review and Project Implementation Status Reports were filed appropriately, with the required annexes on procurement, financial management and safeguards. Project monitoring documents were candid about the progress of the Project. Team leadership was stable, with only one change of TTL over the life of the Project. Changes in the results framework occurred twice, thanks to the close supervision and monitoring of the results. Justification of Overall Rating of Bank Performance 83. For the reasons stated above, Overall Bank Performance is rated as Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 84. The political and financial risk at the time of this Report that the development outcomes will not be maintained is low. The Government and related institutions’ ownership and commitment to sustaining gains are strong. There are, however, technical risks associated with MOH capacity and MOF eagerness for further health financing reforms at the secondary level of health care. Furthermore, the COVID pandemic and its effect on the functioning and financing of Page 25 of 84 The World Bank Health System Improvement Project (P113349) the health care system in Uzbekistan could present a potential danger to the development outcomes. Support to the health sector is continuing through the Emergency Medical Services Project and Health Financing Advisory Services and Analytics. V. LESSONS AND RECOMMENDATIONS 85. Strong Government ownership is critical, particularly with reform-oriented projects. Reform does not only concern technical health-related changes but relies heavily on the political process. Government ownership and commitment were critical in the support of the reforms through adequate legislation and strategies. 86. Continuous and long-term involvement of the World Bank in the specific sector of a country is very important for the success of the operation in that sector. Some of the significant achievements of the Project would not be as such if there had not been long-term World Bank engagement in the transformation of the primary care in Uzbekistan, through the previous Health I and Health II Projects. 87. Close coordination with other technical agencies can be very important. Close coordination and collaboration with WHO in supporting implementation aspects of the Project helped keep key activities on track (such as development of clinical protocols, developing NHA reports, conducting STEPS survey, etc.). 88. Complex reforms require thorough preparation. Failure to implement a successful pilot of case-based financing in hospitals show that such complex reforms require thorough preparation and fulfillment of all preconditions. Those include adequate equipment, but also trained staff and adequate in-country technical capacity at all levels, from individual facilities to the implementing unit, MOH and other central institutions, to be able to implement complex reforms and processes that those entail. 89. Training of health staff and having appropriate tools are very important. In the absence of clinical protocols and the lack of continuous training of the health staff, patients were at risk of being under- or over-referred to higher levels of care in treating their condition. It is very important to provide timely and adequate health care service at the appropriate level of care. To help in that task, robust clinical protocols accompanied with intensive trainings needs to be developed and delivered. 90. Providing continuous guidance and actively engaging in the overall sector policy dialogue was crucial in ensuring consistency of the messages being delivered to the Government. World Bank supervision must be continuous and intensive and having adequate supervision resources, particularly in the context of limited in-country institutional capacity (as was the case in Uzbekistan) allowed this to happen. 91. Supporting Government’s policy, along with comprehensive and careful project design greatly facilitates project implementation. The HSIP built on the Government’s strategic documents and on achievements of previous World Bank projects incorporating lessons learned. A strong Project Implementation Unit (CPIB) was maintained to implement the Project adequately. It was proved appropriate to have CPIB be, as much as possible, part of the MOH to enable smooth implementation of Project activities and transfer of knowledge. Project activities planned were follow-up of activities and results from the Health I and Health II Projects, especially in relation to PHC reform in Uzbekistan. 92. The design, mechanism and rules of IDA financing cycles should be revisited for possible improvements. Current Page 26 of 84 The World Bank Health System Improvement Project (P113349) design and mechanism of IDA financing cycles and the positive desire of the Government of Uzbekistan to secure sufficient funds for further reforms in health sector obviously had an influence on rather early and substantial AF for the parent Project (less than two years after the Project became effective). The cancellation of more than 50% of the AF (total US$52 million equivalent) which was then transferred from HSIP to another health project shows that it might have been prudent to wait a bit longer in the Project implementation to set the amount, apply for and approve the AF. However, the IDA financing cycle mechanism is such that the funds were available in that period (2013) without a guarantee that it would be available at a later date, so the Government of Uzbekistan used the opportunity. If they knew that the funds would be available later under the same conditions, they might have waited and applied for the . AF later in an amount that would have been more realistic to be disbursed. Page 27 of 84 The World Bank Health System Improvement Project (P113349) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Improve access to quality health care at the primary level and at Rayon Medical Unions (RMUs) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased proportion of Percentage 0.00 10.00 80.00 88.00 diabetic and hypertension patients referred from PHC 02-Oct-2012 31-Dec-2016 31-Dec-2019 15-Jan-2020 facilities to RMUs, in accordance with treatment standards Comments (achievements against targets): Target surpassed Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved perceived quality Percentage 70.00 90.00 90.00 90.00 of PHC and secondary health care services in intervention 02-Oct-2012 31-Dec-2016 31-Dec-2019 15-Jan-2020 Page 28 of 84 The World Bank Health System Improvement Project (P113349) areas Comments (achievements against targets): Target achieved Objective/Outcome: Strengthen the Government's public health response to the rise in non-communicable diseases(NCDs) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased proportion Percentage 0.00 100.00 172.00 172.00 (number) of hospitals following NCD clinical 02-Oct-2012 31-Dec-2016 31-Dec-2019 15-Jan-2020 guidelines developed under the project (at least 20 clinical guidelines) Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Issuance of a profile of NCD Text 0 At least 3 reports At least 1 report The report is risk factors and burden of published published published on the MOH disease site December 2018. Page 29 of 84 The World Bank Health System Improvement Project (P113349) 10-Mar-2011 31-Dec-2016 31-Dec-2019 15-Jan-2020 Comments (achievements against targets): Target achieved. A.2 Intermediate Results Indicators Component: Improving Health Service Delivery Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New clinical guidelines Number 0.00 25.00 20.00 22.00 developed and adopted by the MOH (CVD, diabetes, 10-Mar-2011 31-Dec-2016 31-Dec-2019 20-Sep-2019 etc.) Comments (achievements against targets): Target surpassed. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Doctors of urban and rural Number 670.00 3670.00 3000.00 3067.00 Page 30 of 84 The World Bank Health System Improvement Project (P113349) PHC facilities receiving the 10-Mar-2011 31-Dec-2016 31-Dec-2019 20-Sep-2019 training under the 10-month GP training program Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Health personnel at PHCs Text 0 doctors, 0 nurses 6000 doctors, 57000 9400 doctors, 40500 14824 doctors, 50018 (doctors and nurses) nurses nurses nurses receiving training under continuous professional 10-Mar-2011 31-Dec-2016 31-Dec-2019 15-Jan-2020 education Comments (achievements against targets): Target surpassed. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Health personnel receiving Number 16182.00 104418.00 22518.00 43714.00 training 05-Feb-2013 31-Dec-2018 31-Dec-2019 20-Sep-2019 Page 31 of 84 The World Bank Health System Improvement Project (P113349) Comments (achievements against targets): Target surpassed. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Hospitals equipped with Number 0.00 100.00 516.00 748.00 medical and waste management equipment 10-Mar-2011 31-Dec-2016 31-Dec-2019 15-Jan-2020 Comments (achievements against targets): Target surpassed. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Hospital core staff receiving Number 0.00 300.00 516.00 933.00 training on waste management 10-Mar-2011 31-Dec-2016 31-Dec-2019 20-Sep-2019 Comments (achievements against targets): Target surpassed. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 32 of 84 The World Bank Health System Improvement Project (P113349) Target Completion Health personnel receiving Text 0 2200 1002 doctors/2064 3833 doctors/6158 training on clinical case nurses to be trained nurses have been management (NCDs and trained pediatrics) 10-Mar-2011 31-Dec-2016 31-Dec-2019 20-Sep-2019 Comments (achievements against targets): Target surpassed. Component: Strengthening Health Financing and Management Reforms Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion National Health Accounts Text 0 2 reports published. 2 report published. 2 reports are developed and published published. 10-Mar-2011 31-Dec-2016 31-Dec-2019 09-Sep-2019 Comments (achievements against targets): Target achieved. Page 33 of 84 The World Bank Health System Improvement Project (P113349) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Staff of reformed PHC and Number 0.00 600.00 255.00 255.00 hospitals receiving training in financial management 10-Mar-2011 31-Dec-2016 31-Dec-2019 28-Feb-2019 Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Public Expenditure Review Text 0 2 reports published 2 reports published 2 reports are developed published 10-Mar-2011 31-Dec-2016 31-Dec-2019 16-Jan-2020 Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Volume and Cost contract Text 0 All 3 hospitals paid All 3 hospitals paid 3 in virtual mode introduced in 3 hospitals in according to contract according to contract Page 34 of 84 The World Bank Health System Improvement Project (P113349) Fergana oblast 10-Mar-2011 31-Dec-2016 31-Dec-2019 16-Jan-2020 Comments (achievements against targets): Target not achieved (partially achieved). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of urban PHC Number 25.00 155.00 155.00 155.00 facilities (urban family policlinics) converted to per 05-Feb-2013 31-Dec-2018 31-Dec-2019 28-Feb-2019 capita financing system Comments (achievements against targets): Target achieved. Component: Institutional Strengthening for NCD Prevention and Control Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Public Health specialists Number 0.00 300.00 140.00 177.00 (surveillance staff) and (b) health promotion specialists 10-Mar-2011 31-Dec-2016 31-Dec-2019 15-Jan-2020 trained Page 35 of 84 The World Bank Health System Improvement Project (P113349) Comments (achievements against targets): Target surpassed. Page 36 of 84 The World Bank Health System Improvement Project (P113349) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 – Improve access to quality health care at the primary level and at Rayon Medical Unions (RMUs) and selected City Medical Unions (CMUs) 1. Increase proportion of diabetic and hypertension patients referred from PHC facilities to RMUs, in Outcome Indicators accordance with treatment standards 2. Improve perceived quality of PHC and secondary health care services in intervention areas Component 1: Improving Health Service Delivery 1. Health facilities constructed, renovated and/or equipped (Health facilities equipped) 2. New clinical guidelines developed under the Project and adopted by the MOH (CVD, diabetes, etc.) 3. Doctors of urban and rural PHC facilities received the training under the 10-month training GP program 4. Health personnel at PHCs (doctors and nurses) received training under continuous professional education 5. Hospital core staff received training on waste management Intermediate Results Indicators 6. Health personnel received training on clinical case management (NCDs and pediatrics) 7. Health personnel received training Component 2: Strengthening Health Financing and Management Reforms 8. Staff of reformed PHC and hospitals received training in financial management 9. National Health Accounts developed and published 10. Public Expenditure Review developed 11. Volume and Cost contract introduced in3 hospitals in Fergana oblast 12. Number of urban PHC facilities (urban family policlinics) converted to per capita financing system Component 1: Improving Health Service Delivery Key Outputs by Component 1. Procurement of medical and diagnostic equipment (144 different items) for 543 (105.2%) health (linked to the achievement of the facilities - RMUs Objective/Outcome 1) 2. Developed guideline on hospitals (RMU and CMU) functional planning and management, including health care waste management Page 37 of 84 The World Bank Health System Improvement Project (P113349) 3. Established Center for Maintenance of Medical Equipment and its regional branches to provide medical equipment maintenance 4. Vehicles and toolkits for the repair of medical devices and maintenance of medical equipment procured and delivered to RMUs 5. General Practice PHC model extended to all urban polyclinics in pilot regions (Fergana, Syrdarya and Samarkand regions and in Tashkent city). 6. Improved curricula for pre- and post-graduate GP training 7. 3139 GPs (including 77 teachers) went through 10-month GP training 8. Continuous professional education of physicians on primary level in different areas of medicine, including screening and management of NCDs 9. Continuous professional education for nurses, including screening and management of patients with NCDs (hypertension, diabetes, asthma and COPD) 10. Establishment of Regional Training Centers at regional multidisciplinary medical centers (RMMC), as separate, self-supporting departments – to provide logistics for different trainings. 11. Training of pediatricians and nurses in RMUs and Central Rayon Hospitals to increase effectiveness of medical care for children 12. Training of internists and nurses in RMUs and Central Rayon hospitals to increase quality of care 13. Active monitoring of implementation of clinical protocols (including protocols concerning NCDs) into practice in RMUs/CMUs Component 2: Strengthening Health Financing and Management Reforms 14.Per capita financing reforms extended to urban family policlinics. 15. PHC per capita financing model developed, based on assessment indicators and incentives for effectiveness and quality of services provided. 16. Volume and Cost contract-based financing piloted by simulation in three pilot Central Rayon Hospitals in Fergana region 17. Health Financing and Management Reforms Monitoring Department has been established and worked on health facilities staffing requirements, staffing incentive mechanisms, organization of different departments in health facilities and proposal of reforms and establishment of different national centers (e.g. for blood transfusion and medical technologies) 18. Two Public Expenditure Reviews in health sector conducted. 19. Two National Health Account reports developed. Page 38 of 84 The World Bank Health System Improvement Project (P113349) 20. Training in financial management has been developed and provided for MOH, regional health department and RMU staff in Fergana region 21. Health Management Information System has been developed to support hospital financing system. It included a development of a local software for collecting and processing patient’s data from 3 pilot hospitals in Fergana region; a concept for organizing repair and maintenance of medical equipment; and a software to automate the family polyclinic activities. Objective/Outcome 2 – Strengthen the Government’s public health response to the rise in non-communicable diseases (NCDs) 1. Increase proportion (number) of hospitals following NCD clinical guidelines developed under the Outcome Indicators Project 2. Issue a profile of NCD factors and burden of disease Component 1: Improving Health Service Delivery 1. New clinical guidelines developed under the Project and adopted by the MOH (CVD, diabetes, etc.) Intermediate Results Indicators 2. Health personnel received training on clinical case management (NCDs and pediatrics) Component 3: Institutional Strengthening for NCD Prevention and Control 3. (a) Public Health specialists (surveillance staff) and (b) health promotion specialists trained Component 1: Improving Health Service Delivery 1. Developed 22 clinical protocols for the NCD related main causes of hospitalizations to RMU (cardiology, pulmonology, gastroenterology and endocrinology) 2. Continuous professional education of physicians on primary level in different areas of medicine, including screening and management of NCDs 3. Continuous professional education for nurses, including screening and management of patients Key Outputs by Component with NCDs (hypertension, diabetes, asthma and COPD) (linked to the achievement of the 4. Active monitoring of implementation of clinical protocols (including protocols concerning NCDs) Objective/Outcome 2) into practice in RMUs/CMUs Component 3: Institutional Strengthening of NCD Prevention and Control 5. Capacity of public health facilities (like Research Institute of Public Health and Health Management and its regional branches) is strengthened on health promotion and prevention of NCDs in Kashkadarya and Fergana regions, including trainings and conduction of campaigns to combat tobacco, unhealthy diet and sedentary lifestyle. Page 39 of 84 The World Bank Health System Improvement Project (P113349) 6. Measures to improve services provided in PHC facilities for the prevention of NCDs based on (1) WHO-PEN protocols (prevention, treatment and counselling on arterial hypertension, diabetes, asthma and COPD), and (2) interventions on healthy nutrition, physical activity, quitting tobacco and reducing alcohol consumption were implemented in Fergana and Kashkadarya regions, with capacity built on the central level (TIAME, Regional Health Departments) 7. Experience from the pilot regions of Kashkadarya and Fergana on integrated approach to the prevention and control of NCDs have been launched in all regions of Uzbekistan, including campaigns, training materials, educational events. Ultimately WHO PEN protocols were implemented into practice of all PHC facilities. 8. Two WHO Stepwise Approach to Surveillance (STEPS) surveys have been completed nationwide (2015 and 2019) 9. Protocol for epidemiological monitoring of NCDs with key indicators has been approved, implementation is pending. Page 40 of 84 The World Bank Health System Improvement Project (P113349) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Susanna Hayrapetyan Senior Health Specialist, Task Team Leader Ghada Youness Senior Counsel Yuling Zhou Senior Procurement Specialist Fasliddin Rakhimov Procurement Specialist Johanne Angers Senior Operations Officer Antonio Giuffrida Senior Health Economist Iqboljon Ahadjonov HD Operations Officer Gabriel Francis Program Assistant Gulnora Kamilova Program Assistant Galina Alagardova Financial Management Specialist Wezi Marianne Msisha Health Specialist Johnson Appavoo Operation Analyst, Environment Prabbat Jha Health Economist, Consultant Supervision/ICR Elvira Anadolu Task Team Leader(s) Nurbek Kurmanaliev, Fasliddin Rakhimov Procurement Specialist(s) Djamshid Iriskulov Financial Management Specialist Lingzhi Xu Team Member Gabriel C. Francis Team Member Sevara Abdusamatova Team Member Iqboljon Ahadjonov Team Member Ekaterina Romanova Social Specialist Rustam Arstanov Environmental Specialist Volkan Cetinkaya Team Member Page 41 of 84 The World Bank Health System Improvement Project (P113349) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY09 1.915 18,141.23 FY10 14.945 107,416.35 FY11 34.705 209,577.36 FY12 .450 1,883.04 FY16 0 2,047.15 Total 52.02 339,065.13 Supervision/ICR FY12 28.309 123,825.80 FY13 23.577 124,014.09 FY14 47.474 190,623.58 FY15 43.698 175,526.40 FY16 31.361 174,234.15 FY17 26.250 121,513.89 FY18 15.650 106,809.94 FY19 14.050 80,575.84 FY20 19.689 94,030.51 Total 250.06 1,191,154.20 Page 42 of 84 The World Bank Health System Improvement Project (P113349) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Improving Health Service 171.12 133.27 77.88 Delivery Strengthening Health Financing and Management 5.95 1.48 24.87 Reforms Institutional Strengthening for NCD Prevention and 3.38 1.88 55.62 Control Project Management 5.55 4.37 78.74 Total 186.00 141.00 75.81 Page 43 of 84 The World Bank Health System Improvement Project (P113349) ANNEX 4. EFFICIENCY ANALYSIS 4.1 Efficiency in this Report is defined as “a measure of how economically resources and inputs have been converted into results.” The assessment of efficiency asks whether the costs involved in achieving Project objectives were reasonable in comparison with both the benefits and with recognized norms. 3 To answer this question, the current analysis examines three areas. First, it assesses aspects of the design and implementation that could help contribute to, or negatively affect, efficiency. It then presents an estimate of the Project efficiency using a standard CBA. Finally, financial sustainability is assessed, taking into account the overall health sector spending and the country fiscal prospects. A. Aspects of the Project design and implementation that affect efficiency 4.2 The focus on PHC, including the first level hospitals, at the outset offers a favorable prospect to optimize the Project’s allocative efficiency. PHC represents the most frequent point of contact between the general population and the health system and a well-functioning PHC system should be able to handle 80%-90% of the overall health issues. Although equity is not an explicit objective of the Project, investing in PHC also benefits the poor due to its wide reach. The economic case for PHC was corroborated in a recent scoping review which found convincing evidence linking primary care to improved health outcomes, health system efficiency and health equity.4 4.3 In the context of Uzbekistan, improving clinical quality of care is likely to bring about significant returns given the already comprehensive network of health institutions inherited from the Soviet Union time . As is the case with many countries in the former Soviet Union, the most critical gap exists in access to quality care, rather than physical coverage of health services. In this regard, the emphasis on quality helps assure that services provided to the population actually produce the desired health outcomes. The Project activities aimed to improve quality of clinical care by, among others, upgrading and equipping health facilities, training personnel, and developing and implementing clinical guidelines. 4.4 The health issues targeted in the Project (NCDs – especially hypertension and diabetes – and child health) account for the most significant share of the disease burden in Uzbekistan. At the time of the Project appraisal, the country was faced with a persistent challenge in maternal and child health and communicable conditions, at the same time an increasingly heavy toll of NCDs. As shown in Figure 4.1 below, in 2011, conditions accounting for the largest share of disease burden included ischemic heart disease, stroke, diabetes, cirrhosis, neonatal, and lower respiratory infection. Between 2007 and 2017, the burden of disease due to diabetes, as measured by DALYs, increased by more than 45% (IHME, 2020).5 NCDs were responsible for 79% of all deaths in the country and the probability of premature death (before the age of 70 years) from 1 of the 4 major NCDs was more than 25% in 2015. One third of the adult population has hypertension, and one fifth is at high risk of having a heart attack or stroke. The economic cost of NCDs to the Uzbekistan economy was estimated at 9.3 trillion Sum per year, or 4.7% of its Gross Domestic Product (GDP).6 3 Bank Guidance on Implementation Completion and Results Report (ICR) for Investment Project Financing (IPF) Operations. September 27, 2018. 4 World Health Organization (2018): Building the economic case for primary health care: a scoping review. 5 Institute of Health Metrics and Evaluation (IHME) (2020): data download ( http://www.healthdata.org/uzbekistan ). 6 WHO and UNDP (2018) Prevention and control of noncommunicable diseases in Uzbekistan - The case for investment. Page 44 of 84 The World Bank Health System Improvement Project (P113349) Figure 4.1: Share of major health conditions in the total disease burden in Uzbekistan, 2011 (%) Source: IHME (http://www.healthdata.org/uzbekistan) 4.5 The Project’s activities in health promotion and NCDs prevention are grounded on a strong investment case. Risk factors such as tobacco use, harmful use of alcohol, unhealthy diet and physical inactivity are an increasing public health and development challenge in Uzbekistan. Over a quarter of men smoke tobacco and half the adult population is overweight or obese. Measures supported by the Project have been proved to be highly cost-beneficial. Specifically, activities to control the use of tobacco was estimated to have a return on investment ratio of 13 over a 15-year period, interventions to reduce salt intake have a return on investment ratio of 64.8, and increases in physical activity have a return on investment ratio of 9.6.7 4.6 While the Project’s design has features that help optimize its allocative efficiency, certain aspects of implementation appeared to hinder its achievement of technical efficiency. Although institutional arrangements remained stable throughout, the execution of planned activities experienced a number of delays (as documented in various ISRs and the Mid-Term Review report). Delays in executing major procurement packages and conducting key activities represented an opportunity cost for funding that had been committed to the Project. These also led to a failure in producing tangible results early on in the Project cycle. 4.7 Implementation progress could be partly demonstrated in an analysis of disbursements provided in Table 4.1 below. Column 4 shows annual disbursement projections at appraisal for the original loan and for AF combined, column 5 shows planned cumulative disbursement taking into account the cancellation of US$45 million in January 2018, while columns 6 and 7 show annual and cumulative actual disbursements respectively. Column 8 compares actual disbursement against the planned amount on a yearly basis. It does not have values for FY 2018 and 2019 because the US$45 million equivalent cancellation was not reflected in the plan at appraisal. As shown, the Project experienced a spike in FY 2014, with an actual disbursement of US$17.77 million against a planned amount of US$12.6 million (88%). However, from there on, disbursement experienced a setback, with actual performance ranging from 48%-57% of the plan. Column 9 presents the story from a different angle: it uses cumulative value to allow for catching up in the subsequent year if there were special circumstances in the preceding year that caused disbursement delays. As shown, even in the final year, cumulative disbursement represented only 78% of funding committed by that time, leading to a last minute cancellation of US$7 million 7 Ditto Page 45 of 84 The World Bank Health System Improvement Project (P113349) equivalent and unused funds by the Project closing of US$3.96 million (ISR as of February 2020). Table 4.1: Analysis of Planned and Actual Disbursement (in US$ million) Fiscal Planned, Planned, Planned, Planned, Actual, yearly Actual, Actual Cumulative Year yearly yearly yearly cumulative (combined) cumulative as % of actual as % of (original (AF) (combined) and with $ 45 planned, cumulative loan) million yearly planned cancellation (1) (2) (3) (4) (5) (6) (7) (8) (9) 2012 4.7 4.7 4.7 0.41 0.41 9% 9% 2013 12.6 12.6 17.3 6.69 7.1 53% 41% 2014 20 0.2 20.2 37.5 17.77 24.87 88% 66% 2015 24.2 10 34.2 71.7 19.63 44.5 57% 62% 2016 24 20.8 44.8 116.5 21.29 65.79 48% 56% 2017 7.5 25 32.5 149 16.26 82.05 50% 55% 2018 34 34 138 18.6 100.65 73% 2019 3 3 141 9.51 110.16 78% 2020 5.45 115.61 Total 93 93 186 141 115.61 115.61 Source of data: Datasheet 4.8 On the other hand, the analysis of component and administrative costs reveals that HSIP is a rather “lean” operation. As shown in Table 4.2 below, the operating expenses represent 3% of the total loan proceeds, well below the average for administrative cost in World Bank operations (in the range of 5%-10%). Likewise, the cost to the World Bank team for preparation and implementation support over the course of 10 years was US$1.45 million, or slightly higher than 1% of the total loan amount. Table 4.2: Cost Breakdown by Categories of Expenses (in US$ million) Categories Planned Actual % (based on actual) Goods 165.50 107.10 91.53 Training and consulting services 15.00 6.23 5.32 Operating expenses 5.55 3.80 3.25 Total 186.00 117.00 Source of data: Uzbekistan government ICR report Note: Discrepancy in disbursement figures compared to records in the World Bank system may be due to difference in exchange rate between Special Drawing Rights and US$ at the time of withdrawal B. Quantifying the Project’s Efficiency 4.9 The economic analysis at appraisal followed a standard CBA. Costs included the original loan and contribution from the Government of the Uzbekistan. Benefits were derived mainly from health gains produced by activities in Components 1 and 3. Gains from improved efficiency in the health sector, presumably derived from Component 2 were deemed negligible and not included in the analysis. Health gains were estimated using assumptions on the Project effectiveness and measured in terms of DALYs averted, which were subsequently assigned a monetary value using standard formula in the economic literature. Both cost and benefits were Page 46 of 84 The World Bank Health System Improvement Project (P113349) assumed to accrue through the year 2030. The analysis yielded an internal rate of return (IRR) of 24.4% and a net present value (NPV) of US$189.6 million in the low scenario, assuming the value of one DALY equals to a one time GDP per capita in the country. 4.10 The current analysis adopts the same methodology as the analysis at appraisal, which remains adequate and relevant. Due to the absence of a rigorous impact evaluation, it is not possible to draw a true causal inference on the Project’s effects on intended outcomes. Attributing changes in the disease burden over time to the Project will not be correct given the concurrent existence of many factors outside the sphere of the Project, such as other Government programs, projects supported by other partners, and changes in the socio-economic and demographic conditions, which in turn affected the disease burden. In the absence of a true “counterfactual” scenario, certain assumptions will need to be made about the Project’s effects. Furthermore, although the PDOs mark important milestones in the result chain toward improved health outcomes, converting improved quality of care and strengthened NCDs prevention to health gains is beyond the scope of this analysis. Finally, challenges in obtaining more granular epidemiological data on the targeted health conditions from the country itself justify the use of DALYs for the estimate of health gains. 4.11 While the overall methodology remains unchanged, input data have been updated and key assumptions are revised in the current analysis. Instead of using projected growth rates throughout, this analysis utilizes, as much as possible, actual data for the project period. Most assumptions are revised to be on a more conservative side given the implementation experience discussed earlier. Even though the Project was extended for 3 years, the final year for costs and benefits accrual is kept at 2030 to correspond with the typical useful life of equipment bought by the Project. Details of data and assumptions and how they differ from the original analysis are provided below. I. Computing Project’s Costs 4.12 The financial costs involved in producing the Project’s results consist of actual disbursements from IDA loans and from the Government co-financing . Following the original analysis, this analysis also includes an amount for recurrent cost to be incurred by the Government during 2020-2030, which equals to a half of the final year spending. This allows for continuing maintenance of medical equipment, refresher training, and distributing guidelines to assure that Project’s benefits can be generated beyond its closing date of December 2019. Page 47 of 84 The World Bank Health System Improvement Project (P113349) Table 4.3: Project Spending by Year (in US$ thousand) IDA loan Government financing Total Total (current $) (current $) (current $) (2010 constant $) 2012 410.0 546.8 956.8 711.1 2013 6,690.0 6,871.4 13,561.4 8,688.2 2014 17,770.0 5,004.9 22,774.9 12,578.4 2015 19,630.0 6,004.8 25,634.8 12,205.1 2016 21,290.0 13,828.8 35,118.8 14,414.2 2017 16,260.0 10,238.0 26,498.0 9,735.8 2018 18,600 2,901.4 21,501.4 6,558.5 2019 9,510.0 1,284.6 10,794.6 2,838.5 2020 5,450.0 642.3 6,092.3 1,381.0 2020-2030 6,423.0 6,423.0 703.7 Total 115.61 53,746 169,356.0 69,454.4 Source of data: IDA loan (datasheet), government financing (Govt ICR report) 4.13 By the closing date of the Project, disbursements in nominal terms were US$115.61 million from IDA and US$46.68 million from Uzbekistan Government. Spending in nominal terms was converted to US$ 2010 constant using GDP inflators obtained from WDI for 2012-2018 with projection for 2019-2030. The average inflation during this period is 16%. This generates a total Project cost of 69.45 million in 2010 US$ terms. II. Estimating Project’s Benefits The counterfactual scenario of disease burden measured by DALYs 4.14 In the economic analysis at appraisal, the counterfactual scenario for DALYs was extrapolated from WHO’s estimates for Europe and Central Asia (ECA) for the year 2002 with a projection to 2015 and 2030 . The extrapolation relied on total and age specific population size, practically assuming that DALYs per person in Uzbekistan was the weighted average of such value for ECA. This yielded a value of 2.73 million DALYs in 2002 for the Project catchment population of 15.11 million, which represented 54.38 % of the total population in the country. Thus, using population size and composition to extrapolate burden of disease from ECA overall value, there would be 5.02 million DALYs for the whole of Uzbekistan in 2002. 4.15 Instead of deriving value for Uzbekistan from ECA aggregates, the current analysis uses data reported specifically for the country by the Institute for Health Metrics and Evaluation (IHME).8 All causes of DALYs were obtained for each of the years 2011-2017 and projected for 2018-2030 using the average rate of change between 2011-2017. Conceptually, the observed (or reported) DALYs for 2011-2017 already incorporated the Project’s effects, which means that without the Project, the DALYs during this period would have been higher. The (unobserved) counterfactual scenario is the sum of the observed DALYs and the number of DALYs averted due to the Project. 4.16 As a side note, the number of DALYs reported by IHME in 2002 specifically for Uzbekistan was 8.5 million, significantly higher than the estimate at appraisal based on ECA aggregates (5.02 million). It is likely that the latter had underestimated Uzbekistan’s burden of diseases by assuming weighted average for ECA. The ECA 8 Data can be accessed from http://ghdx.healthdata.org/gbd-results-tool. Page 48 of 84 The World Bank Health System Improvement Project (P113349) region in the WHO 2002 estimate included low and upper middle-income countries, a majority of them could have lower DALYs/population than Uzbekistan. Number of DALYs averted due to the Project 4.17 The economic analysis at appraisal assumed different levels of DALY reduction in each major disease category and for each major intervention supported by Components 1 and 3 . Assumptions on intervention effectiveness were based primarily on the review of literature complied in the Disease Control Priorities volume 2 (DCP-2). Specifically, a 3% DALY reduction was linked with the Project goals of expanding treatment guidelines for NCDs, 1% from expanded hospital coverage, and 2%-4% from the health promotion components. The analysis also assumed that effects of the different interventions were additive. 4.18 The current analysis adopts substantially more conservative assumptions. First, an overall DALY reduction of 1% is applied because adding effects from different interventions is likely to overestimate the Project’s overall impact. Second, acknowledging that the 1% reduction materializes only when the Project is implemented in full and delays in implementation need to be accounted for, the Project’s effects are further discounted to reflect actual disbursement each year against the total loan amount (US$186 million). This is justified by the understanding that the full loan amount was deemed necessary to achieve the intended results. With this, the Project is found to generate very little effect in 2012 when disbursement was less than US$1 million, and the effect did not peak until 2019. Lastly, instead of assuming continuous increase in the Project’s impact beyond closing, the number of DALYs averted is reduced by 15% on a yearly basis after 2022. The analysis also takes into account the expansion of the catchment population since the AF. Figure 4.2 shows estimated number of DALYs averted due to the Project on an annual basis (undiscounted). Figure 4.2: Estimated Number of DALYs Averted (annually) 60 50 40 Thousands 30 20 10 - Page 49 of 84 The World Bank Health System Improvement Project (P113349) Monetary value of DALYs saved 4.19 The value of DALYs averted was calculated assuming each DALY equals to one time of GDP per capita. GDP per capita in 2010 constant US$ was based on actual value through 2018 and projected for 2019-2030 .9 The projection is based on the International Monetary Fund (IMF)’s latest report (April 2020), which incorporates the expected effect of novel coronavirus (COVID-19) on economic growth.10 According to IMF’s projection, GDP growth was adjusted down from 6% to 1.8% in 2020, but expected to bounce back to 7% in 2021. After that, we assume a year-on-year growth of 5% to be on the conservative side. The downstream value of DALYs is discounted at 3% (future discount) in line with the usual practice in the economic literature. Sensitivity analysis 4.20 Several scenarios are simulated using different assumptions on inputs to the CBA. These include: (a) an inflation of 11% instead of 16%, which was the value used at appraisal; (b) a future discount rate of 5% rather than 3% for DALYs; and (c) a grimmer prospect for GDP growth given the yet uncertainty about the magnitude of the COVID-19 crisis at the time of this analysis. In this scenario, GDP growth will be 2% in 2021 rather than 7% and stay at this level for 2 more years until it reaches to 4% starting in 2024; and (4) a scenario combining all above 3 adjustments. Summary of results 4.21 Table 4.4 presents the results of the CBA for the main model and two sensitivity analyses . With the key parameters described above, the main model shows a net benefit of US$879.2 million in real terms, which translates into an NPV of US$ 176.5 million and an IRR of 199%. The benefit cost ratio (BCR) is estimated at 13.7, which means that for every dollar invested, the Project generated benefits valued at 13.7 times higher. This BCR ratio is relatively high with regard to conventional health interventions. Table 4.4: Results of the CBA Variable Total costs Total benefits Net benefits Main results Values (2010 constant US$, thousand) 69,454 948,668 879,214 NPV (2010 constant US$, thousand) 176,553 IRR 199% BCR 13.7 Alternative scenario 1 (deflator rate of 11%) Values (2010 constant US$, thousand) 89,475 948,668 859,193 NPV (2010 constant US$, thousand) 266,100 IRR 160% BCR 10.6 Alternative scenario 2 (DALY discount of 5%) Values (2010 constant US$, thousand) 69,454 779,991 710,536 NPV (2010 constant US$, thousand) 146,550 IRR 181% 9 World Development Indicator databases 10 IMF World Economic Outlook Database, April 2020 Page 50 of 84 The World Bank Health System Improvement Project (P113349) BCR 11.2 Alternative scenario 3 (GDP growth of 2% during 2021-2023 and 4% during 2024-2030) Values (2010 constant US$, thousand) 69,454 895,975 826,520 NPV (2010 constant US$, thousand) 170,562 IRR 199% BCR 12.9 Alternative scenario 4 (deflator rate of 11%, DALY discount of 5%, GDP growth of 2% during 2021-2023 and 4% during 2024-2030) Values (2010 constant US$, thousand) 89,475 739,811 650,336 NPV (2010 constant US$, thousand) 208,939 IRR 144% BCR 8.3 4.22 Estimates of the Project’s benefits vary with sensitivity analysis but remain highly positive . Specifically lowering the deflator rate led to increased cost in real terms, from US$69.4 million to US$89.4 million, subsequently reducing the BCR to 10.6. Assuming a higher DALY discount rate (5%) led to a lower estimate of the Project benefits. Even in the highly conservative scenario that has large DALY discount and GDP growth significantly lower than projected by the IMF, benefits remain substantial, with NVP equaling more than US$208.9 million, IRR of 144% and a BCR of 8.3. The model is more sensitive to deflator and DALY discount rate than the assumption on GDP growth. C. Project’s fiscal impact 4.23 The fiscal impact of the Project is assessed against health sector spending using data from 2012-2017, with 2017 the latest year for which country’s health financing data are available. As shown in table 4.5 below, government spending on health in Uzbekistan was around 50% during 2012-2013 but went down to only 43% in 2017. As a percentage of GDP, government spending had not tipped over 3%, which is on a low side internationally. Even in its peak year of disbursement (2016), the Project financing only accounted for 1.77% of the general government spending on health. The cost for maintaining the Project achievement beyond its closing date (not shown in the table) is significantly small and should not impose any appreciable impact on the fiscal envelop of the country. Table 4.5: Uzbekistan’s Basic Health Financing Indicators and Project Financing 2012 2013 2014 2015 2016 2017 Total health expenditure (million current US$) 3,163 3,509 3,609 4,120 4,313 3,200 Total GHE (million current US$) 1,588 1,769 1,884 2,050 1,984 1,385 GHE as % of total health expenditure 50.23 50.41 52.20 49.78 46.02 43.30 GHE as % of general government expenditure 9.84 9.75 9.52 10.15 10.09 10.17 GHE as % of GDP 3.04 3.05 2.96 3.04 2.91 2.78 Project cost as % of GHE 0.06 0.77 1.21 1.25 1.77 1.91 Project cost as % of total health expenditure 0.03 0.39 0.63 0.62 0.81 0.83 (GHE – Government Health Expenditure); Source of data: WDI (2020) and Project D. Summary Page 51 of 84 The World Bank Health System Improvement Project (P113349) 4.24 In summary, the efficiency analysis reveals that HSIP has many positive aspects which helped ensure its value for money. The Project was designed to tackle the most challenging disease burdens and shortcomings in the health sector, and it had low administrative and operating costs. On the other hand, Project impacts could have been more profound had the implementation been faster. Nevertheless, the Project was found to have significant positive results and was highly justified from the economic standpoint. It also has a high prospect for sustainability, given the rather insignificant fiscal impact on Government health expenditure. The Project’s rating for efficiency is Substantial. Page 52 of 84 The World Bank Health System Improvement Project (P113349) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS BORROWER’S CONTRIBUTION TO ICR Page 53 of 84 The World Bank Health System Improvement Project (P113349) Ministry of Health of the Republic of Uzbekistan Health III Central Project Implementation Bureau HEALTH SYSTEM IMPROVEMENT PROJECT (HEALTH III) and ADDITIONAL FINANCING FOR HEALTH SYSTEM IMPROVEMENT PROJECT IDA Credit of US$ 186 million HEALTH-III PROJECT IMPLEMENTATION COMPLETION AND RESULTS REPORT November 2019 Page 54 of 84 The World Bank Health System Improvement Project (P113349) Report of the Ministry of Health of the Republic of Uzbekistan on the results of the completion of the World Bank’s investment project “Health System Improvement” (Health 3) 1. Project overview at entry 1.1. State policy on health system reform. Under the State Healthcare Reform Program, the Republic of Uzbekistan has undergone wide-scale modernization of all levels of the healthcare sector by optimizing the structure of medical institutions, improving their infrastructure and equipping them with modern medical equipment, introducing state-of-the-art clinical technologies, upgrading the professional skills of healthcare providers, developing and introducing new prospective forms of financing and management at all levels of the healthcare system. The basis of the organization of medical services in the Republic of Uzbekistan is made up of district and regional administrative territorial units: the city of Tashkent (capital), the Republic of Karakalpakstan and 12 regional centers. At the time of preparation and start of the Project, there were 1,132 hospitals and 4,310 outpatient facilities in the country. At entry, the number of district-level hospitals was 159, and the number of city-level hospitals - 847, of which 264 were private hospitals. The Government of the Republic of Uzbekistan with the support of the World Bank through two investment projects “Health-1” and “Health-2” carried out large-scale reforms of primary health care in urban and rural areas by organizing general practice based medical care in rural areas and family medicine in cities. Further cooperation of the Government of the Republic of Uzbekistan and the World Bank in the context of the current project was aimed at improving the quality of medical services for outpatient and inpatient care at the district and city levels. Within the framework of this project, the Government of the Republic of Uzbekistan planned to increase the effectiveness of medical care and improve the managerial structure in rayon (district) and city medical unions (RMU and CMU), as well as improve the provision of specialized services at the level of central multi-profile polyclinics and regional multi-profile centers by developing a system of patient referral and prevention of the development of NCD complications. Page 55 of 84 The World Bank Health System Improvement Project (P113349) For this, the following key strategic directions and areas were identified for further modernization of primary and secondary health care: - improvement of inpatient medical services by outfitting rayon and individual city medical unions (RMU and CMU) with modern medical equipment, training of personnel of medical institutions covered by the project, implementation of state- of-the art technologies for the functional management of hospitals and new evidence medicine based clinical practices, and improving the system for maintenance of medical equipment; - further support of primary health care reform in urban family polyclinics by (i) outfitting them with medical equipment and (ii) training of general practitioners (GP); - facilitating the implementation of general principles of public health through (i) the introduction of health promotion programs and screening programs to prevent diabetes, hypertension and other chronic diseases, (ii) improving epidemiological surveillance of risk factors for noncommunicable diseases; - supporting the natural development of health financing and management reforms, with a possible emphasis on strengthening financing and primary health care management reforms, extending the reforms of the urban PHC model and management reforms, as well as introducing reforms to the financing and management of inpatient facilities in the pilot regions. 1.2. Project Development Objective The overall Project Development Objectives (PDOs) are to (1) improve access to quality health care at the primary level and at RMUs; and (2) strengthen the Government’s public health response to the rise in noncommunicable diseases (NCDs). 1.3. Project Components and Subcomponents. Objectives and Expected Results. Design of the Health System (Health-III) Project envisages the following four components: Component 1: Improving Health Service Delivery This component consists of three subcomponents: Page 56 of 84 The World Bank Health System Improvement Project (P113349)  Sub-component 1.1.: Hospital Services Improvement This subcomponent is designed to improve hospital care through the following measures: (a) refurbishing central rayon hospitals (CRH) and central rayon multi-profile polyclinics (CRMP) with up-to-date medical and technological equipment, as well as with waste management equipment and some types of medical furniture; (b) improving health service functional planning and management at the RMU and CMU, including the revision of hospital building regulations and standards, equipment maintenance systems; (c) organization of study tours for employees of the administration of healthcare institutions to increase the level of knowledge and competence.  Sub-component 1.2: Primary Health Care Development This subcomponent aims at developing the primary health care (PHC) by: (a) expanding the general practice model of PHC facilities to urban polyclinics; (b) completing the preparation of general practitioners (GP) under the ten-month training program for rural health facilities; (c) improving skills and competencies of PHC physicians and nurses in early diagnosis, screening and treatment of priority NCDs and development of optimal urban general practice model implementation strategy, through training of personnel, provision of consultants’ services and outfitting with modern medical equipment. - Sub-component 1.3: Clinical Quality Enhancement The key aim of the subcomponent is to enhance clinical quality by: (a) advanced training of pediatricians, therapists, infectious disease specialists and nurses of regional medical unions (RMU) on the use of new clinical protocols - standards for diagnostics and treatment; (b) training in clinical case management and hospital administration; (c) development and implementation of modern and effective treatment standards for physicians and nurses of CRH and CRMP; (d) establishing quality improvement mechanisms to monitor application of the new clinical protocols – diagnostics and treatment standards, through the training of personnel, provision of consultants’ services and outfitting with training Page 57 of 84 The World Bank Health System Improvement Project (P113349) equipment. Expected results: improved quality and availability of medical services of primary, specialized and secondary health care at district and city levels, and decreased number of visits to specialized national level hospitals. Component 2: Strengthening Health Financing and Management Reforms This component aims at: (a) consolidating and institutionalising (summation and expansion) per capita based primary health care financing and management reforms by expanding the principles of per capita financing and management in urban family polyclinics; (b) developing the strategy for the health sector financing and appropriate regulatory measures for the implementation of that strategy through the provision of consultants’ services; (c) improving the system of hospitals financing through the implementation of (i) services cost-and-volume contracts in 3 district hospitals of Fergana region and (ii) block contracts in other district hospitals of regions covered by the project; (d) strengthening the leadership role of MOH in health financing policy formulation and oversight and local capacity building by: - conducting health sector expenditures analysis and medium term projections; - developing the national health accounts (NHA); - training relevant staff at MOH, rural primary health care facilities and RMUs on financial management; - developing a health financing information system to support the implementation of the hospital financing pilot through provision of goods, consultant’s services and training. Expected results: strengthened (i) system of per capita based primary health care financing and (ii) reforms of financing and management system; implemented hospital financing reforms at district level; developed a health sector financing information system; developed and implemented national health accounts; and strengthened role and responsibility of the Ministry of Health in the development and oversight of health sector financing policies. Page 58 of 84 The World Bank Health System Improvement Project (P113349) Component 3: Institutional Strengthening of the System for NCD Prevention & Control The overall aim of this Component is to strengthen the capacity of public health institutions in effective prevention and control of noncommunicable diseases (NCDs). This component consists of two sub-components:  Sub-component 3.1: Health Promotion and NCDs Prevention The subcomponent is aimed at strengthening the capacity of the Institute of Health and Medical Statistics and its territorial branches in pilot areas (Kashkadarya, Ferghana) in the effective prevention and control of noncommunicable diseases by increasing population awareness of risk factors and changing habits entailing increased risk of arterial hypertension, diabetes and other noncommunicable diseases, through the provision of goods, consultant services, training and seminars.  Sub-component 3.2: Strengthening NCD Surveillance System This sub-component is aimed at strengthening the system of epidemiological surveillance in the pilot (Kashkadarya, Fergana) and control (Syrdarya) regions by: (a) developing an epidemiological surveillance system for NCDs; (b) enhancing the ability of the public health system to use medical and statistical data for planning NCDs prevention programs, through the training of medical specialists in statistics, the provision of consultants’ services, goods and software. Expected results: The planned activities would contribute to implementing the National Health Promotion Plan (2011–2014) aimed to improve the knowledge and skills of health promotion specialists in undertaking effective health promotion and changing behavior of population. Globally, the component will contribute to the development of public health in the Republic of Uzbekistan, including inter-sectoral communication and improvement of public response. Component 4: Project Management The component is aimed at strengthening the capacities of the Ministry of Health, the Central Project Implementation Bureau(CPIB) and the Regional Project Implementation Bureaus (RPIB) for the management, implementation, monitoring and evaluation of the project, as well as procurement and financial management, through the provision of goods, consultant’ services, training and operating Page 59 of 84 The World Bank Health System Improvement Project (P113349) expenses. Expected results: improved knowledge and experience of health system managers in coordinating, monitoring and evaluating of ongoing health system reforms; improved procurement and financial management of the project. 1.4. Project Cost and Financing by Component (IDA) Components Initial cost Additional financing Revised cost Improving Health 82,17 88,95 171,12 Services Delivery Strengthening Health 4,45 1,50 5,95 financing and Management Reforms Institutional 2,98 0,4 3,38 Strengthening on NCD Prevention Control Project Management 3,4 2,15 5,455 Total Project Cost 93,00 93,00 186,00 2. Project Design, Implementation and Impact on the Development and Improvement of Health System 2.1. Assessment of Project Design The Government of the Republic of Uzbekistan is concerned about low efficiency of the existing inpatient care and is ready for active measures to modernize the extensive network of medical institutions. The Welfare Improvement Strategy of Uzbekistan for 2008-2010 and the Republic of Uzbekistan Presidential Order ((№700 dd. September 19, 2007) “On main directions on further deepening of reforms and implementation of the State Program on Healthcare Development” identify inpatient and specialized care as important areas of focus for the next stage of health sector reforms with the aim of improving the performance of medical establishments. The goal of the Government is to improve the quality of secondary health care services in districts / cities, as well as to provide high-quality tertiary services in Page 60 of 84 The World Bank Health System Improvement Project (P113349) regional institutions in order to increase access and reduce cases of self-referral to Republican specialized centers. The project aims to support ongoing health care reforms by strengthening the infrastructure of the hospital health sector and expanding the ongoing health care reforms of primary health care by strengthening the infrastructure of primary and secondary health care facilities, expanding the coverage of the general practice model for urban primary care facilities, and ensuring continuing medical education among doctors and nursing staff, increase efficiency of services provision by combining per capita payments in primary care. In the hospital sector, the project focuses on investments in equipping project hospitals with modern diagnostic and treatment equipment, developing and implementing clinical protocols for priority noncommunicable diseases, improving the skills of medical personnel, improving funding for district hospitals and creating a modern public health model for developing healthy lifestyle skills among the population. The project was a continuation of the collaboration between the Government and the World Bank in conducting large-scale reforms of the healthcare system. 2.2. Key performance indicators per components of the Project Component 1: Improving Health Services Delivery. Subcomponent 1.1. Hospital Services Improvement - As part of implementation of the State Investment Program, using the project co-financing funds, the Government carried out civil works on reconstruction and capital repairs of rayon and city medical unions, including hospitals, multi-profile polyclinics, urban and rural family polyclinics, and rural medical centers . - The Project (i) outfitted 543 (105,2 %) health facilities with medical equipment, furniture, computer equipment and vehicles; and (ii) carried out the procurement of 140 (out of 144) items of diagnostic equipment, medical furniture, sets of instruments and accessories for maintenance of medical and computer equipment. - Summarizing the results of the Project, healthcare waste management equipment for 172 hospitals, electric generators, urologic syringe and device for hermetic packaging remain not procured. - Over 274 thousand pieces (131 items) of goods and equipment totaling to the amount of 105.2 mln. USD were delivered to medical institutions. Page 61 of 84 The World Bank Health System Improvement Project (P113349) - The project created the Republican Training and Production Center for the maintenance of medical equipment, and work is underway to establish its territorial branches. To ensure their mobility, the project procured and provided them with vehicles and toolkits for the repair of medical devices (supplied under the project). - Within the period of 2017-2018, in line with the MOH Order No. 366 dated July 03, 2017, in association with the professionals of Tashkent Institute of Postgraduate Medical Education (TIPME), 1327 physicians of RMU underwent 36-72 hours training to operate nine (9) types of high-technology equipment supplied under the project. At the end of the training, the trainees obtained the certificates of completion with permit to operate the equipment. - To train district-level health managers, in consultation with an expert, a guide on hospitals functional planning and management was developed and approved, and 402 heads of rayon and city hospitals were trained. - In 2011, at the stage of completing the preparation of the project, a study tour was organized for health managers in the Republic of Korea to study the experience of organizing hospital services. Subsequently, a number of regional health authorities went to Barcelona to study at WHO courses on reforming the health system, including visits to primary and secondary healthcare facilities. - There were developed 22 clinical protocols on main priority types of noncommunicable diseases being the main reasons for hospitalization of patients in district hospitals. The clinical protocols were based on diagnostics and provision of medical care, starting from the level of initial addressing for outpatient services and up to the provision of medical care in rayon level hospitals. Sub-component 1.2: Primary Health Care Development - Over the years of the project implementation, 3062 general practitioners and 77 teachers were trained under the 10-month training program. The curriculum was revised and updated in line with the recommendation of an international expert. Teachers of training centers (GP training chairs /courses) improved their skills at 20 TOT seminars. - GP training centers under medical universities were reorganized into the chairs/courses for GP professional development and retraining; and faculties of advanced training were established. Starting from 2017, all the chairs/courses for GP professional development and retraining conduct short-term seminars within the Page 62 of 84 The World Bank Health System Improvement Project (P113349) system of continuing professional education for general practitioners. - 264 rayon level trainers for Tashkent region and Tashkent city were trained in 9 areas of general medical practice; 663 rayon level trainers for all regions were trained in 4 areas of general medical practice; and 395 rayon level trainers underwent the training on the implementation of WHO PEN protocols; subsequently, through the cascade methods, they trained 11,569 GPs. - Altogether for all the regions, there were prepared (i) 222 rayon level trainer-nurses on 6 areas of nursing care and (ii) 415 rayon level trainer-nurses on implementation of the WHO PEN protocols, who trained 25195 visiting nurses in the training rooms of RMUs. Sub-component 1.3: Clinical Quality Enhancement - Within the framework of the project implementation, jointly with the EU-UNICEF project, pediatricians and pediatric infectious disease specialists were trained in providing medical care to children under 5 years of age in inpatient conditions according to the strategy of Integrated Management of Childhood Illnesses (IMCI) for hospitals; there were trained 828 pediatricians from 8 regions. - The guideline “Care for child patients” for nurses was developed; 181 rayon level trainers were prepared; they trained 5830 nurses from the pediatric departments of district hospitals. - According to the 22 clinical protocols, which were developed under the project, the teachers of TIPME elaborated a curriculum; 104 regional trainers were prepared, who subsequently conducted 118 seminars at the regional level and trained 2,684 therapists of district hospitals and specialists (cardiologists, endocrinologists, pulmonologists and gastroenterologists) of multi-profile polyclinics. Clinical protocols were implemented into the curriculum of specialized chairs of TIPME. - Clinical protocols have been developed for 6 areas of nursing for nurses of rayon level hospitals. According to these protocols, in all regions, there were prepared 355 rayon level trainer-nurses, who trained 1,413 nurses of the therapeutic departments of rayon hospitals. The curriculum was implemented into the programme for professional development of nurses at the Republican Center for Advanced Training of Nurses and Pharmacists and its regional branches. Component 2: Strengthening health care financing and management reforms. Page 63 of 84 The World Bank Health System Improvement Project (P113349) This component aims to strengthen the country's health care reforms, which include introduction of effective financing systems in the health sector. Component structure: 2.1: Consolidation and institutionalization (aggregation and dissemination) of reforms of per capita financing and primary health care management 2.2: Development of health sector financing strategy and related regulatory measures to implement this strategy. 2.3: Improving the hospital financing system through the introduction of contracts for the cost and volume of services in 3 district hospitals of the Ferghana region 2.4: Strengthening the role of the Ministry of Health in development and monitoring of health financing policies and local capacity-building Project implementation outcome: Development and partial implementation of result-oriented effective financing methods based on the equitable distribution of funds for healthcare. The main results of component implementation - Per capita financing was implemented at all family polyclinics of the republic on the basis of the Decree of the Cabinet of Ministers No. 37 dated February 13, 2013, which is confirmed in the process of monitoring and evaluation; - Report forms and indicators for assessing the performance of urban family polyclinics in the new financing and management environment have been developed and approved for monitoring the transfer of family polyclinics to per capita financing; - orientation seminars were held to explain the methodological foundations of per capita financing and calculation of the budgets of family polyclinics; - there is developed the strategy for financing the health sector and also related regulatory measures; - the Regulation and the procedure for the normative method for determining the bed capacity of the Central Rayon Hospital / Central City Hospital were developed taking into account the number of people served; - a simulation of the financing system was carried out to test the contract based payment system at the cost for the volume of provided medical services (treated case) in 3 pilot CRHs; - developed a universal system for calculating the cost of the treated case; Page 64 of 84 The World Bank Health System Improvement Project (P113349) - prepared two analytical reports on the analysis of public health spending in Uzbekistan, which are published on the website of the Ministry of Health, http://minzdrav.uz/projects/ detail.php?ID=55593 - there are developed proposals and recommendations for the implementation of the NHA system throughout the republic to be coordinated at the interagency commission. The first NHA report is published on the website of the Ministry of Health http://minzdrav.uz/projects/detail.php?ID=55593 and http://www.minzdrav.uz/projects/detail.php?ID=56482; - the household survey on population expenditures for medical services is being completed and the third NHA report is being prepared; - in pursuance of the Decree of the President dated December 7, 2018 No.PP- 4055, with a view to further institutionalizing the National System of Health Accounts, a draft decree of the Cabinet of Ministers of the Republic of Uzbekistan that defines the main tasks of the NHA and the methods for collecting and analyzing data on health accounts was developed. Component 3: Strengthening the prevention and control of noncommunicable diseases (NCD) systems. Subcomponent 3.1. Formation of a healthy lifestyle and prevention of noncommunicable diseases. In order to achieve the goals and objectives, there was implemented the project “Sog’lom Hayot - Healthy Life” in the pilot regions - Kashkadarya and Ferghana - in collaboration with WHO. The project was aimed at development of mechanisms and tools for implementing integrated approach to prevention and control of NCDs - to create examples of best practice, and ultimately as a result, development of a package of effective interventions, a roadmap and recommendations for it’s effective implementation at the country level. Subcomponent 3.2. Strengthening the surveillance system The project planned the creation of epidemiological surveillance system through piloting the WHO STEPS study in the Ferghana, Kashkadarya and Syrdarya regions, and then, at a later stage, to deploy and transform it into permanent/ongoing monitoring system at the national level. In collaboration with WHO, two studies were carried out to determine the risk factors for the population according to the WHO STEPS methodology in 2012 and Page 65 of 84 The World Bank Health System Improvement Project (P113349) in 2019, which make it possible to systematically provide epidemiological surveillance of the population health and assess the impact of risk factors. The results of the first report were published on the website of the Ministry of Health. In addition, under support of WHO and with the participation of representatives of the Ministry of Health, National Statistics Committee, a Protocol for the epidemiological monitoring of NCDs of the Republic of Uzbekistan was prepared and approved by the Ministry of Health on August 15, 2019. Based on the Protocol of the Republic of Uzbekistan for Epidemiological Monitoring of NCDs, there were collected data and prepared a Bulletin of the main NCD indicators for the period 2014-2017. To ensure sustainability, a draft order of the Ministry of Health was prepared, providing for the approval of the Bulletin of the main indicators of NCDs for the period 2014-2017. and identifying responsible parties for its annual update. Component 4: Project management. - Daily project management was carried out in accordance with the procedures of the World Bank and the legislation of the Republic of Uzbekistan; - For the implementation of project activities, the operational manual of the project “Improving the Health Care System” (Health-3) was prepared; The central and regional project implementation bureaus were created and staffed with qualified personnel, consisting of the Executive Director of the CPIB, directors of the regional project implementation bureaus, four coordinators, a procurement department, and a financial manager; - The general management of the project was carried out by the Ministry of Health and the project supervisor in the person of the First Deputy Minister; - A mid-term evaluation of the project was carried out; - There was carried out comprehensive survey of medical institutions, suppliers and recipients of services for evaluating medical services and population satisfaction with their quality; - Two surveys were conducted to study the prevalence of risk factors for noncommunicable diseases using the WHO methodology (STEPS) - A sociological study was conducted among households to assess the financial costs of medical care; Page 66 of 84 The World Bank Health System Improvement Project (P113349) - There are prepared and published two analytical reports on public health spending; - Two analytical reports on national health accounts were prepared and published on the website of the Ministry of Health; - There is prepared the manual on functional planning and management of hospitals and provided training for district and city health managers; - With the participation of the Tashkent Institute for Advanced Medical Education, short-term training courses for physicians of RMUs and CMUs on the operation of high-tech equipment were organized and conducted; - There are conducted training activities on fiduciary procedures, investment project management and awareness of the implementation of World Bank projects for representatives of the Ministry of Finance, the Ministry of Health and employees of the central and regional project implementation bureaus; - There was held the III Republican conference for personnel of primary health care institutions "Improving the quality of medical service in primary health care"; - With the participation of the coordinators and the procurement department of the project, there was prepared a design for the investment project “Improving emergency medical care”; - The experience of the CPIB staff was used by the Ministry of Health in the preparation and implementation of a number of new investment and grant projects in other areas of health care reform; - The project coordinators were included in the interdepartmental working group on the development of the draft law on compulsory health insurance, provisions on the structure of the Compulsory Health Insurance Fund, strategy for financing health care within the framework of compulsory health insurance and the draft package of guaranteed medical services and other preparatory regulatory documents for implementation of insurance medicine in the Republic of Uzbekistan. - Also, the coordinators of the project as part of the working group of the Ministry of Health developed legislative and regulatory acts to limit the distribution and consumption of alcohol and tobacco products. According to the results of monitoring and evaluation of the project, all four main indicators have been implemented, out of 13 intermediate indicators there are implemented 12 and 1 indicator is in the process of implementation. 2.3. Project impact on health sector reform and programs Page 67 of 84 The World Bank Health System Improvement Project (P113349) Over the years of implementation, the Project participated in preparation and implementation of two state programs for reforming the healthcare system: Resolution of the President of the Republic of Uzbekistan No. 700 of September 19, 2007 “On the main directions for further deepening reforms and implementation of the State program on healthcare system development” and Presidential Decree No. 5590 of December 7, 2018 "On comprehensive measures to radically improve the health care system." Thanks to the flexibility of planning project activities and support of the World Bank, Ministry of Economy and Industry, Ministry of Finance and Ministry of Health, the Government’s instructions within the framework of the project were fully implemented and served to strengthen health services. 3. A critical analysis of the actions taken by the World Bank, the Government and of technical assistance provided in the implementation of the Project. 3.1. World Bank Performance Assessment during Project Implementation The continuous assistance of the World Bank team and direct supervision were very significant, both for preparation of the project and for all the stages of its implementation. The preparation of the project was supported by extensive analytical work and technical assistance from World Bank experts. The objectives of the project were in line with the priority health care reforms stated by the Government, with a focus on improving access to quality health care services at the primary care level and hospital services in rayon and city medical unions (RMUs, CMUs), and strengthening the Government’s public health response to the rise in noncommunicable diseases (NCDs). Changes made to project components / subcomponents during implementation were consistent with the initial objectives of the project. Additional financing and redistribution of project loan funds were made during the project life cycle at the request of the Government, which significantly contributed to effective implementation and successful completion of the project. Direct control by the World Bank project team has contributed to timely identification of problems and the development of possible solutions. The World Bank Local Project Officer has been providing high-quality collaboration on an ongoing basis. Immediate response to inquiries, opinions of experienced experts and extremely useful comments and suggestions from the WB team were very important for the successful implementation of the project. The Ministry of Health and the CPIB prized effective collaboration with the WB. Page 68 of 84 The World Bank Health System Improvement Project (P113349) The World Bank has been involved in extensive coordination among donor organizations, in particular the WHO and UNICEF. The most tangible result was significant contribution of WHO experts to development and publication of reports on National Health Accounts (NHA), conducting research on prevalence of risk factors for NCDs using the WHO STEPS methodology, development and implementation of clinical protocols for priority NCDs (WHO PEN), development and implementation of measures to prevent risk factors for NCDs, including tobacco consumption, unhealthy diets and insufficient physical activity, as well as development of a protocol on epidemiological monitoring of noncommunicable diseases in the Republic of Uzbekistan. 3.2. Assessment of Government actions during implementation Key decisions contributing to the implementation of the project. The project enjoyed strong support and active interest from the Government throughout its implementation. An important step was the Government’s commitment to support the experiment on financing 15 central rayon hospitals of Ferghana Region on the basis of concluding two types of contracts: to pay based on the cost and volume of services provided per treated case - in 3 hospitals and based on estimate costs in accordance with the approved normative indicators - in 12 hospitals; to carry out, starting from 2013, an experiment on financing the central rayon hospitals of the Ferghana region on the basis of contracts between the regional finance department, the public health department and the rayon medical union, and agreed with the proposal of the Ministry of Health to implement a National System of Health Accounts using instruments applied in world practice as recommended by the World Health Organization. The sustainable success of the project in the field of public health development was promoted by the Decree of the Cabinet of Ministers of the Republic of Uzbekistan No. 841 of October 20, 2018, aimed at achieving the Sustainable Development Goals, approved by the UN Global Agenda until 2030, in the field of healthcare system reform, ensuring universal health coverage, access to quality basic health services, and reduction of premature mortality from priority NCDs. The overall management of the implementation of project activities was carried out by the Ministry of Health through monitoring and supervision of each component of the project. The Ministry of Health organized expert commissions from amongst the leading specialists and managers of the Ministry of Health, which examined the proposals of the CPIB for technical assistance from international and national consultants, conducted expert reviews and approved reports on the results their Page 69 of 84 The World Bank Health System Improvement Project (P113349) work. Tender commissions were organized for procurement. The Ministry of Health annually reviewed and approved the project implementation plans, coordinated and approved in the Ministry of Economy and Industry, the Ministry of Finance, the network schedule for the procurement and utilization of loan funds of the project and assisted the CPIB and RPIB to carry out annual activities for timely disbursement of financial resources, improve financial management and accounting and payment of state taxes. Timely availability of project co-financing The Government's contribution covered the costs associated with the modernization of the physical condition of hospitals and other institutions covered by project activities, recurring maintenance costs and taxes. The infrastructure of the institutions within the project framework was strengthened as part of the state investment program for the construction and reconstruction of medical institutions. Construction works were carried out with the involvement of regional and district khokimiyats (administrations). In general, there were no significant delays in government funding during the project life cycle. 3.3. Evaluation of the effectiveness and quality of cooperation between the World Bank and the Government for the duration of the project In the framework of the preparation, implementation and completion of the Health- 3 Project, the Government of the Republic of Uzbekistan successfully cooperated with the World Bank, timely resolving emerging issues. 3.4. Evaluation of providing various organizations (for example, training centers, etc.) companies and technical support during implementation of the project The successful implementation of the project was largely determined by the technical assistance of 13 international and 126 local consultants. In addition, in coordination with the Ministry of Health and the World Bank, with participation of local and foreign companies, annual audit and a number of sociological studies were carried out to assess implementation of various focus areas of the project. Most of the contracts were successfully completed, and this allowed for implementation of strategically important reforms in the context of professional interaction and a combination of international and local experience. 4. Economic and financial assessment of the project Page 70 of 84 The World Bank Health System Improvement Project (P113349) 4.1. The total cost of the project The total cost of the project, taking into account additional financing of the project in 2013, at the stage of the start of project implementation amounted to 186.0 million US dollars (73.08%), project co-financing by the Republic of Uzbekistan - 68.5 million US dollars (26.92%). During the implementation of the project, the feasibility study of the project was revised twice in connection with additional financing and extension of the project. On the proposal of the Republic of Uzbekistan (Letter of the Ministry of Finance No. AX / 13-05-27 of January 24, 2018) and the World Bank (letter No. 00850/18 of February 22, 2018), it was decided to reduce the loan amount by redistributing $ 45 million to the project “Improving the emergency medical care service” and to extend the implementation of the project until December 31, 2019. Cost of the project by components after restructuring (in millions of US dollars) Component Plan Redistribution Actually on Deviations plan 31.10.2019 Component 1 171,12 133,27 110,78 22,49 Component 2 5,95 1,48 0,495 0,985 Component 3 3,38 1,88 0, 533 1,347 Component 4 5,55 4,37 5,191 -0,825 Total 186,00 141,00 117,00 24,00 When raising funds for the project, the currency in SDR is provided, therefore, during the implementation of the project, an exchange rate difference has formed in the range of 20.0 million US dollars. Breakdown by categories of expenses from the initial cost of the project (in millions of US dollars). Categories Plan Actually on 31.10.2019г Goods 165,50 107,10 Training and 15,00 6,23 consulting services Page 71 of 84 The World Bank Health System Improvement Project (P113349) Operating expenses 5,55 3,80 Total 186,00 117,00 4.2. Forecasted value at the time of negotiations in comparison with the actual cost by components, expense categories and sources of financing (in US dollars equivalent) As of October 31, 2019, 163.7 million USD were disbursed, of which $ 117.0 million came from IDA loans (72%) and 46.7 million USD - from co-financing funds (28%). Disbursed by years Total in IDA loan in Republic of thousands USD thousands USD Uzbekistan in thousands USD 2012 1598,6 1051,8 546,8 2013 28854,3 21982,9 6871,4 2014 13000,1 7995,2 5004.9 2015 23463,3 17458,5 6004.8 2016 45097,6 31268,8 13828.8 2017 29862,5 19624,5 10238.0 2018 18426,2 15524,8 2901.4 2019 3382,0 2097,4 1284.6 Total 163684,6 117003,9 46680.7 4.3. Key lessons learned from project outcomes Goals, Tasks and Structure of the Health-3 project, aimed at further improving the healthcare system, are in line with the main areas of healthcare system reform approved by the state program of the Republic of Uzbekistan. Key indicators for monitoring and evaluating the project based on the results of the implementation clearly demonstrate the achievement of the main objectives of the project, provided for by the feasibility study of the investment project. At the same time, the implementation of the project was not always carried out in a timely manner and part of the project activities were not fully implemented. The Page 72 of 84 The World Bank Health System Improvement Project (P113349) experience gained related to implementation of the pilot case-based financing program has demonstrated that despite the Government having conceptual potential and available successful experience in implementing this methodology in the new specialized centers and republican hospitals in the country, sufficient prerequisites for its implementation at the rayon and city hospital levels have not been created. In addition, the heads of territorial health and finance authorities prefer the current system of allocation of resources based on the bed method, which is considered more manageable and understandable. Thus, the preparation of a pilot project to implement a new hospital financing system required more sound and thorough training of representatives of local authorities - khokimiyats, regional financial authorities, hospital managers and economists. Certain successes of the project took place in the implementation of clinical skills and the implementation of clinical protocols for NCDs and PEN. At the same time, the sustainable and effective success in using new clinical technologies requires the use of a comprehensive approach, including thorough and long-term training of doctors and nurses, thorough training of hospital managers and specialists in the management mechanism for improving the quality of medical services. For this, it is necessary to expand the training program and continue introducing topics related to the quality of medical services and the implementation of clinical protocols into the curriculum of the continuing education of the Tashkent Institute for Advanced Medical Education and its branches. During implementation of the project, part of the saved and undisbursed loan funds of the project were redistributed to implementation of a new project to improve emergency medical care. Structural transformations in the Republic, including changes in legislation in preparation, implementation and management of investment projects, as well as in procurement legislation, required long periods of time for the approval of technical specifications and tender documents, which led to forced delay in bidding (2017- 2018). During preparation of Health-3 Project, taking into account the experience gained in implementation of the Health-1 and Health-2 projects, there were fears of achieving goals in the public health development sector. At the same time, achievements in this area were successful, due to the high degree of interest of the Government and the Ministry of Health and active support of WHO. 4.4. The impact of the project on the institutional structures created or financed by the project Page 73 of 84 The World Bank Health System Improvement Project (P113349) Under support of the Ministry of Health and the World Bank, most of the project goals have been achieved: - The equipping of 172 RMU and CMUs with modern diagnostic and medical equipment and medical furniture was completed. The average cost of equipment supplied to RMU and GMU is approximately 314 -335 thousand USD. - 39 CCMPs, 180 city family polyclinics and 126 rural medical centers are equipped. The average cost of equipment supplied to CCMP was 142 thousand USD, one city family clinic - 59 thousand USD and one SVP - 13 thousand USD - 26 clinical protocols on priority noncommunicable diseases have been developed and introduced into medical practice. - As part of the continuing medical education program, there are trained 7896 physicians at the regional level and 11569 physicians at the district level and 1327 physicians were trained on operation of medical equipment. - On the basis of regional multidisciplinary medical centers (RMMC), there were organized training centers as self-supporting departments of hospitals with their own sub-account. - There was completed equipping 14 educational centers of universities with educational, presentation equipment and medical literature, and they were converted into departments / courses for retraining and advanced education of GPs, where salaries of department employees paid from budgetary funds. - The Republican Center for Advanced Training of Nurses and Pharmacists and its 13 regional branches are equipped with mannequins, presentation equipment and medical literature, teachers are trained, clinical protocols for nurses are introduced into the curriculum of the center. - There have been completed phased expansion of per capita funding to all city family polyclinics of the republic and virtual piloting of a hospital payment system based on contracts at a cost for the volume of provided medical services in certain areas of the Ferghana region. - A healthcare management information system has been developed and is being virtually introduced in three districts of the Ferghana region to support the testing of a new hospital financing system. - A manual and training program for managers of health care institutions has been developed, aimed at increasing the financial management capacity in medical institutions on the basis of functional planning and management. Page 74 of 84 The World Bank Health System Improvement Project (P113349) - The development of methods for the prevention and control of noncommunicable diseases has been completed by expanding the measures on healthy lifestyle promotion and epidemiological surveillance in a number of individual regions in order to develop prerequisites for subsequent nationwide implementation. 5. Sustainability of project investments The Government of the Republic of Uzbekistan, with the support of the World Bank, through two investment projects Health-1, Health-2, carried out large-scale reforms of primary and secondary health care in urban and rural areas, by providing medical care based on general practice in rural areas and family medicine in cities. Further cooperation between the Government of the Republic of Uzbekistan and the World Bank in the framework of the current Health-3 project was aimed at improving the quality of medical services for outpatient and inpatient care at the district and city levels. Under this project, the Government of the Republic of Uzbekistan has improved the management structure of district and city medical unions (RMUs and CMUs), and improved provision of specialized services at the level of central multidisciplinary polyclinics and regional multidisciplinary medical centers by developing a referral system in provision of medical services and by preventing common noncommunicable diseases. The project from the point of view of geographical coverage allowed improving inpatient medical services in RMUs and CMUs throughout the Republic of Uzbekistan by equipping them with modern medical equipment, training personnel of the medical institutions in the project framework, introducing modern technologies for the functional management of hospitals and creating a system of technical maintenance of medical equipment. It facilitated further supporting the primary health care reforms in urban family polyclinics by equipping them with medical equipment and training general practitioners (GPs). The project assisted the Government in creating a sustainable public health system through implementation of programs on promotion of healthy lifestyles, prevention and early detection of diseases, such as diabetes mellitus, arterial hypertension, bronchial asthma and chronic obstructive pulmonary diseases, as well as improving epidemiological surveillance of risk factors for noncommunicable diseases. It supported development of reforms in health financing and management with a possible use in strengthening financing and management reforms in primary health care in urban and rural areas, and laid the foundation for reforming the financing and management of inpatient Page 75 of 84 The World Bank Health System Improvement Project (P113349) facilities in the pilot regions. The project developed and implemented clinical protocols for priority noncommunicable diseases, a program for continuing medical education of personnel became sustainable, and medical personnel was trained in operation of the provided equipment. The latest survey on the level of satisfaction of population with the quality of medical services at medical institutions in rural and urban areas convincingly showed that the majority of the population who approached district hospitals and rural medical centers were satisfied with the volume and quality of medical and preventive services. Thus, the population constituted the main beneficiary of the impact of the project. The medical personnel of the institutions within the project framework is also a beneficiary of of the project impact, since the provided professional training programs in clinical skills for diagnosis and treatment, disease prevention and the skills of maintaining a healthy lifestyle and good nutrition allowed physicians to expand and improve the quality of services provided for prevention, early diagnosis and treatment of noncommunicable diseases. The training provided contributed to their professional development and upgrade of their qualifications in providing safe and high-quality medical services. Many educational programs for training of physicians and paramedical personnel have received a steady distribution, since included into educational programs of medical universities at the pre- and postgraduate levels, as well as in the postgraduate education programs of the Republican Center for Advanced Studies for secondary medical workers and pharmacists and its regional branches. The project contributed to strengthening the capacity of managers of primary and secondary levels of health care system, as well as the Ministry of Health by providing training on functional planning and management, as well as an opportunity of direct participation in ongoing innovations in health system reform, covering various aspects of management with a focus on structural and medical transformations. 6. Conclusions and epilogue Creation of a high-quality health care system in the country, ensuring the preservation and improvement of population health, creation of conditions for upbringing a healthy generation, is a priority area of state policy. Thanks to the measures taken in the country within the framework of the state program for healthcare development from 2008 and a new concept for the Page 76 of 84 The World Bank Health System Improvement Project (P113349) development of the healthcare system for the period 2019-2025 created socio- economic conditions for their implementation. As a result of implementation of Health-3 Project, a more efficient system of modernized and non-duplicating institutions has been created at district level, and their activities will be result-oriented with a focus on noncommunicable diseases. At the same time, medical unions will be able to pay more attention to outpatient care, using the support and services of GPs to work with the population. The reform of the district branch of health care within the framework of the “Improving the healthcare system” (Health-3) project made it possible to ensure the effective integration of medical institutions of primary and secondary levels of healthcare system, improve accessibility and continuity in the provision of care, improve the quality and efficiency of medical services, and expand the scope of medical aid and preventive care for adults and children, men and women of all ages. In the primary health care sector, the project ensured: (i) development of primary health care reform by expanding the model of general medical practice in urban PHC facilities; (ii) improved the system of continuous professional development of GPs and nurses; and (iii) contributed to the development of a common vision of the PHC financing system and to improvement of efficiency of service delivery by consolidating per capita payments in PHC. In the secondary health care sector, the project ensured: (i) strengthening the infrastructure of medical facilities at rayon and city levels (RMU and CMUs), made strategic investments in diagnostic and curative equipment of rayon and city hospitals, family and city polyclinics; (ii) improved clinical services management for priority NCDs, (iii) strengthened referral systems, and (v) conducted piloting of financing for rayon hospitals. In conclusion, the project supported strengthening the capacity of the Ministry of Health to develop health-financing policies and explore public health priorities. The project focused on implementing programs to improve healthy lifestyles and developing a system for NCD surveillance. Improving the quality and effectiveness of medical services is largely determined by the organization of the management system and functional planning of the services provided, the organization of the patient care system, highly qualified personnel, accounting and distribution of medical equipment. As a result of implementation of Health-3 project at the district level, there was created a more efficient system of modernized and non-duplicating institutions, the Page 77 of 84 The World Bank Health System Improvement Project (P113349) activity of which is result oriented with an emphasis on noncommunicable diseases. At the same time, there have been created the conditions, so that medical associations can pay more attention to outpatient care, using the support and services of GPs in working with the population with an emphasis on the development of home visiting/patronage services. Thus, according to the results of the Health-3 project, the Ministry of Health, with participation of the World Bank experts, has created a conceptually new model of primary and secondary health care at the rayon and city levels that can satisfy the population’s need in high-quality and effective medical and preventive care. Page 78 of 84 The World Bank Health System Improvement Project (P113349) ANNEX 6. DETAILED CHANGES OF IR INDICATORS 6.1 Under the AF of July 18, 2013, three intermediate results (IR) Indicators were dropped from the Results Framework (RF), while others have been revised or added (e.g., in some cases to match that of Core Sector Indicators). Also target values for most of the IR Indicators were changed to reflect scaling up of the activities and the 24-month extension of the closing date as part of the AF. Table 6.1: – IR Level Indicators Revised as Part of AF Original IR Indicator IR Indicator Revised as Part of AF Comment 1.1 Hospitals equipped with 1.1 Health facilities constructed, Wording revised to match that of medical and waste management renovated and/or equipped the Core Sector Indicator. Target equipment (Health facilities equipped). has been adjusted to reflect scaling up of activities and 24-months extension. 1.2 New treatment standards 1.2 New clinical guidelines Indicator revised to reflect correct developed and adopted by the developed under the Project and terminology of clinical guidelines. MoH (CVD, diabetes, etc.) adopted by the MoH (CVD, Target also adjusted. diabetes, etc.) 1.3 Urban polyclinics’ doctors 1.3 Number of doctors of urban Indicator reworded to reflect that receiving training under the 10- and rural PHC facilities receiving doctors in both urban and rural month training GP program training under the 10-month PHC facilities are to be trained training program under the Project. 1.4 Health personnel at PHCs No change. Target adjusted to reflect (doctors and nurses) receiving additional scope. training under continuous professional education 1.5 Hospital management staff No change. Target adjusted to reflect receiving training on hospital additional scope. management 1.6 Hospital core staff receiving No change. Target adjusted to reflect training in waste management. additional scope. 1.7 Health personnel receiving No change. Target adjusted to reflect training on clinical case additional scope. management (NCDs and pediatrics) Page 79 of 84 The World Bank Health System Improvement Project (P113349) Original IR Indicator IR Indicator Revised as Part of AF Comment 1.8 People with access to a basic Dropped. The definition of this Indicator was package of health. unclear and not possible to track within current context of Uzbekistan health services and this Project in particular. New Indicator. 1.8 Health personnel receiving This Indicator in newly added with training. the AF. It is a Core Sector Indicator. 2.1 Staff of reformed PHC and No change. Target adjusted to reflect hospitals receiving training in additional scope. financial management. 2.2 Percent of recurrent Dropped. Annual health sector expenditures expenditures not related to salary for the different types of facilities relative to actual expenses of: is determined by the overall budget allocated to the sector. As a) PHCs such, the Project is not able to b) Rayon hospitals influence this allocation. Therefore, the Indicator was c) Urban polyclinics dropped. 2.3 National Health Accounts No change. Only serial number changed (from developed and published 2.3 to 2.2) 2.4 Public Expenditure Review No change. Only serial number changed (from developed 2.4 to 2.3) 2.5 Volume and Cost contract No change. Only serial number changed (from introduced in 3 hospitals in 2.5 to 2.4) Fergana oblast. New Indicator 2.5 Number of urban PHC facilities This Indicator is to reflect advances (urban family polyclinics) in the implementation of the converted to per capita financing provider payment reforms in system. outpatient facilities that are supported by the Project. 3.1 Number of video clips on No change. health education on cardiovascular risk factors 3.2 Number of people screened Dropped. There is no special registration of for (a) hypertension, and (b) people who underwent screening Page 80 of 84 The World Bank Health System Improvement Project (P113349) Original IR Indicator IR Indicator Revised as Part of AF Comment diabetes through community- for hypertension and diabetes, so based screening programs in obtained information may be targeted areas. unreliable. 3.3 Percent of population in No change. Only serial number changed (from targeted oblasts knowing 3.3 to 3.2) cardiovascular risk factors 3.4 Public Health specialists 3.3 (a) Public Health specialists Indicator revised to reflect that (surveillance staff) trained in data (surveillance staff) and (b) health both surveillance and health collection and analysis, including promotion specialists trained promotion public health specialists trainers, in all oblasts will be trained. 6.2. Under the Level 2 restructuring of the AF of HSIP of February 02, 2018, three additional IR Indicators were dropped from the RF, while target value was adjusted for one IR Indicator. Table 6.2 – PDO and IR level Indicators after the Project restructuring in 2018 IR level Indicator after AF IR level Indicator after Comment restructuring Component 1: Improving Health Service Delivery 1.1 Health facilities constructed, No change. renovated and/or equipped (Health facilities equipped). 1.2 New clinical guidelines No change. developed under the project and adopted by the MoH (CVD, diabetes, etc.) 1.3 Doctors of urban and rural No change. PHC facilities receiving the training under the 10-month training GP program. 1.4 Health personnel at PHC No change in wording, but target The annual target value for nurses (doctors and nurses) receiving values changed. is decreased to reflect the trends training under continuous of the past years (explained by professional education initially high target volume). Page 81 of 84 The World Bank Health System Improvement Project (P113349) IR level Indicator after AF IR level Indicator after Comment restructuring 1.5 Hospital management staff Dropped. Some of the modules of this receiving training on hospital training took place under the management original project. 1.6 Hospital core staff receiving No change. Only serial number changed (from training on waste management. 1.6 to 1.5) 1.7 Health personnel receiving No change. Only serial number changed (from training on clinical case 1.7 to 1.6) management (NCDs and pediatrics). 1.8 Health personnel receiving No change. Only serial number changed (from training. 1.8 to 1.7) Component 2: Strengthening Health Financing and Management Reforms 2.1 Staff of reformed PHC and No change. hospitals receiving training in financial management. 2.2. National Health Accounts No change. developed and published. 2.3 Public Expenditure Review No change. developed. 2.4 Volume and Cost contract No change. introduced in 3 hospitals in Fergana oblast. 2.5 Number of urban PHC facilities No change. (urban family polyclinics) converted to per capita financing system. Component 3: Institutional Strengthening for NCD Prevention and Control 3.1 Number of video clips on Dropped. Similar activities were carried out health education on by the Government independently cardiovascular risk factors. of the Project. Page 82 of 84 The World Bank Health System Improvement Project (P113349) IR level Indicator after AF IR level Indicator after Comment restructuring 3.2 Percent of population in Dropped. There is no baseline data from the targeted oblasts knowing beginning of the Project and no cardiovascular risk factors. rationale to conduct a survey in the last year of the project. STEP survey conducted in the beginning of the project and to be carried out at the end of the project will provide information on the awareness rate among the population. 3.3 (a) Public Health specialists No change. Only serial number changed (from (surveillance staff) and (b) health 3.3 to 3.1) promotion specialists trained. Page 83 of 84 The World Bank Health System Improvement Project (P113349) ANNEX 7. SUPPORTING DOCUMENTS 1. Project Appraisal Document 2. Project Paper for Additional Financing 3. Financing Agreements (parent project and additional financing) 4. Country Assistance Strategy for the Republic of Uzbekistan FY08-FY11 5. Country Partnership Framework for the Republic of Uzbekistan FY16-FY20 6. Restructuring papers 7. Central Project Implementation Bureau documents 8. Aide Memoires 9. Management and other important letters and memoranda 10. Implementation Status and Results Reports (ISRs) 11. ICRs of Health I (P009125) and Health II (P051370) Projects 12. Borrower’s contribution to ICR Page 84 of 84