Document of The World Bank Report No: ICR00002967 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-75510 TF-92893) ON A LOAN IN THE AMOUNT OF US$50 MILLION TO THE PEOPLE’S REPUBLIC OF CHINA FOR A RURAL HEALTH PROJECT June 22, 2015 Health, Nutrition and Population Global Practice China Country Unit East Asia and Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective December 31, 2014) Currency Unit = Renminbi (RMB) Yuan (Y) Y1.00 = US$0.16 US$ 1.00 = Y6.20 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS AAA Analytic and Advisory Activities BOH Bureau of Health CMS Commune-based Cooperative Medical Scheme CPS Country Partnership Strategy CPSM Center for Project Supervision and Management DFID Department for International Development DRGs Diagnostic Related Groupings EER Economic Rate of Return EMP Medical Waste Management Plan FLO Foreign Loan Office GOC Government of China GDP Gross Domestic Product MA Medical Assistance Scheme M&E Monitoring and Evaluation MIS Management Information System MOF Ministry of Finance MTR Mid-term Review NDRC National Development and Reform Commission NEP National Experts Panel NHFPC National Health and Family Planning Commission NCDs Noncommunicable Diseases NCMS New Cooperative Medical Scheme NPV Net Present Value OOP Out-of-pocket PDO Project Development Objectives PMO Project Management Office UHC Universal Health Coverage Senior Director Timothy Grant Evans Director Olusoji O. Adeyi Practice Manager: Toomas Palu Project Team Leader: Shuo Zhang ICR Team Leader: Tania Dmytraczenko ICR Primary Author: Shiyan Chao and Tania Dmytraczenko CHINA RURAL HEALTH PROJECT CONTENTS Data Sheet A. Basic Information ........................................................................................................... ii B. Key Dates ....................................................................................................................... ii C. Ratings Summary ........................................................................................................... ii D. Sector and Theme Codes............................................................................................... iii E. Bank Staff ...................................................................................................................... iii F. Results Framework Analysis ......................................................................................... iv G. Ratings of Project Performance in ISRs ........................................................................ x H. Restructuring (if any) ..................................................................................................... x I. Disbursement Profile .................................................................................................... xii 1. Project Context, Development Objectives and Design ................................................... 1 2. Key Factors Affecting Implementation and Outcomes .................................................. 6 3. Assessment of Outcomes .............................................................................................. 11 4. Assessment of Risk to Development Outcome ............................................................. 19 5. Assessment of Bank and Borrower Performance ......................................................... 20 6. Lessons Learned............................................................................................................ 23 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............... 25 Annex 1. Project Costs and Financing .............................................................................. 26 Annex 2. Outputs by Component (from the Project) ........................................................ 27 Annex 3. Economic and Financial Analysis ..................................................................... 32 Annex 4. Bank Lending and Implementation Support/Supervision Processes................. 40 Annex 5. Beneficiary Survey Results ............................................................................... 42 Annex 6. Stakeholder Workshop Report and Results ....................................................... 43 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 44 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 53 Annex 9. List of Supporting Documents .......................................................................... 54 MAP IBRD 33387 i A. Basic Information Country: China Project Name: Rural Health Project Project ID: P084437 L/C/TF Number(s): IBRD-75510,TF-92893 ICR Date: June 22, 2015 ICR Type: Core ICR PEOPLES REPUBLIC Lending Instrument: SIL Borrower: OF CHINA Original Total USD 50.00M Disbursed Amount: USD 49.97M Commitment: Revised Amount: USD 49.97M Environmental Category: B Implementing Agencies: Ministry of Health Cofinanciers and Other External Partners: Department for International Development (DFID) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 02/21/2006 Effectiveness: 12/31/2008 12/31/2008 03/31/2011 Appraisal: 06/25/2007 Restructuring(s): 08/01/2012 04/30/2013 Approval: 06/24/2008 Mid-term Review: 09/30/2011 09/13/2011 Closing: 12/31/2013 12/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Satisfactory Performance: Performance: ii C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time No None (QEA): (Yes/No): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 32 32 Compulsory health finance 36 36 Health 27 27 Sub-national government administration 5 5 Theme Code (as % of total Bank financing) Health system performance 67 67 Rural services and infrastructure 33 33 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg James W. Adams Country Director: Bert Hofman David R. Dollar Director, GHNDR Olusoji O. Adeyi Practice Toomas Palu Emmanuel Y. Jimenez Manager/Manager: Project Team Leader: Shuo Zhang John C. Langenbrunner ICR Team Leader: Tania Dmytraczenko ICR Primary Author: Shiyan Chao Tania Dmytraczenko iii F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project development objective is increased and more equitable access to quality health services, improved financial protection, and better management of public health threats in pilot provinces and counties, with lessons to support reforms in non-project areas. Revised Project Development Objectives (as approved by original approving authority) The Project Development Objectives (PDO) were not revised during project implementation. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Ratio of hospitalization rate of MA (Medical Assistance Scheme) vs. non-MA Indicator 1 : beneficiaries. Average1.5 Value 12 counties <1 quantitative or Positive trend N/A Average 2.1 29 counties >1-2 Qualitative)1 9 counties >2 Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments Target achieved and the average ratio increased by 0.6 percentage points. (incl. % However, the sample size for MA beneficiaries who were hospitalized is very achievement) small and the data for this indicator is not robust. Ratio of outpatient visit rate of the bottom 20% vs. top 40% in the income Indicator 2 : distribution in the previous year. The number of Average 1.1; Value counties with the 18 counties <1; 6 counties with <1 quantitative or compared ratio N/A 14 counties 1-1.5; ratio Qualitative) lower than 1 is 8 counties >1.5 reduced to 6 Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments Target achieved. The number of counties with ratio lower than 1 has reduced (incl. % from 18 to 6, a 67% reduction. achievement) 1 Government had a baseline of 30 counties>1-2 and 8 counties >2. For the purpose of this report, we are using the baseline in the World Bank ISR. iv % of households with annual health expenditures in excess of 20% of total Indicator 3 : income. 5 percentage Value points lower than quantitative or 19.2% N/A 13.2% the baseline Qualitative) (14.2%) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target surpassed by 1 percentage point. achievement) Indicator 4 : Rate of overall satisfaction with rural health services among men. 4 percentage Value points higher than quantitative or 63.8% N/A 85.4% the baseline Qualitative) (67.8%) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target surpassed by 17.6 percentage points. achievement) Indicator 5 : Rate of overall satisfaction with rural health services among women. Value 6.8 percent points quantitative or 66.6% higher than the N/A 85.5% Qualitative) baseline (70.60%) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target surpassed by 14.9 percentage points. achievement) Indicator 6 : Public health system scorecard rating. Value quantitative or 61.78 80 points N/A 90.22 Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target surpassed. achievement) Indicator 7 : Innovation accepted and rolled out at provincial or regional level. Value quantitative or 0 Positive trend. N/A 40 counties2 Qualitative) Date achieved May 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments Target achieved. At the end of the project, all 40 project counties have (incl. % disseminated their innovations to other counties. achievement) 2 The 199 figure reported in the ISR refers to number of pilots tested. v (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years % of total annual (individual) inpatient expenses financed through out-of-pocket Indicator 1 : payments for NCMS members. Value (quantitative 74.70% 67.7% N/A 48.4% or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments Target surpassed. Portion of inpatient care cost financed by out-pocket (incl. % expenditure has reduced from 74.7 to 48.4%, surpassing the target of 67.7% by achievement) 19.3 percentage points. % of total annual (individual) inpatient expenses financed through out-of-pocket Indicator 2 : payments for MA beneficiaries covered by the NCMS. Value (quantitative 72.3% 65.3% N/A 35.6% or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target surpassed by 29.7 percentage points. achievement) Indicator 3 : Total annual NCMS expenditures as percentage of total annual NCMS funds. Value (quantitative 72.7 >85 N/A 98.6 or Qualitative)3 Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments Target achieved and the NCMS fund has become the predominate source for (incl. % NCMS expenditure. achievement) Indicator 4 : % NCMS enrollment rate. Value (quantitative 93.9% Maintain >90% N/A 97.3% or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target achieved and the enrollment of NCMS increased by 3.6%. achievement) 3 Government had a baseline of 68.1%. For the purpose of this report, we are using the baseline in the World Bank ISR. vi Average number of outpatient visits per health professional per day over last Indicator 5 : year at THC. Value (quantitative 3.9 Positive trend N/A 4 or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments Target achieved. The health professional workload has been maintained and (incl. % the change, though insignificant, is in the right direction. achievement) Indicator 6 : Rate of change in average cost per inpatient case in county hospitals. Project counties Value should be lower (quantitative 7.7% than provincial N/A 7.5% or Qualitative) average rate of change Date achieved Dec. 31 2009 Dec. 31, 2013 Dec 31, 2014 Comments Target achieved. The project counties’ average rate was 7.5% at the end of the (incl. % Project, lower than the baseline of 7.7% and lower than the provincial average of achievement) 8.5%. Indicator 7 : % of deliveries by caesarean section. # of counties Value moving towards (quantitative 28.90% N/A 28.5% WHO standards, or Qualitative) which is 15% Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 This target was not well defined. First, the target did not specify how many Comments counties should have moved toward the WHO standard for the target to have (incl. % been considered achieved. Second, it did not clearly specify what constituted a achievement) positive trend towards the WHO standard.4 4 At the beginning of the Project, 13 project counties already had C-section rates below the WHO standard of 15% and 10 of them had rates below 10%. That is, they had an underuse of the procedure and a positive trend would have been to increase in the C-section rates to around 15%. At the end of the Project, five out of these 10 counties increased C-section rates from underuse to overuse, i.e., their rates surpassed 15% by a non-negligible amount. In another four counties, rates changed little, and in one county, the rate declined to 4.4 compared to 7.1% at baseline. At the onset of the Project, 26 counties had rates higher than the WHO standard and a positive trend would have been to reduce their rates. Among these counties, 38% managed some reductions, but most have rates that are still considerably higher than 15%. Since progress towards the target can be either an increase or decrease in the rate, the average rate is not a meaningful measure. It would not capture the Project’s achievements and efforts. Indeed, the objective of the indicator was not well understand by many Project counties. Further interventions needed to address the problem of misuse of C- sections, would have had to be tailored to each county depending on whether it was over or underusing the procedure. This was not done. vii % of outpatients at township health centers and village clinics that received two Indicator 8 : or more antibiotics. Value 18.9% township health 4 percentage 4.8% township (quantitative centers and 19.8 village points reduction N/A health centers and or Qualitative) clinics from baseline 6.5% village clinics Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments Target surpassed by 14.1 percentage points at township level and 13.3 (incl. % percentage points at the village level. achievement) Indicator 9 : % of women who have access to a female qualified health worker. Value 28 counties >90% 95.9, 36 counties (quantitative 32 counties >90% N/A 12 counties <90% >90 or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target surpassed by 4 counties (12.5%). achievement) Number of administrative villages that have been initiated the implementation of Indicator 10 : “healthy village”5. 50% of the 35 out of total 40 project counties project counties have initiated (87.5%) launched the “healthy village” implementation Value and 277 villages of healthy (quantitative 0 met the standard of village and at or Qualitative) “healthy village”. least 1 village All project in each provinces have province meet healthy villages that healthy village’ met the standards. standards Date achieved Dec. 31, 2008 Dec 31 2014 Dec. 31, 2014 Comments (incl. % Target surpassed by 37.5 percentage points. achievement) 5 “According to WHO, a village or rural community can be considered healthy when rates of infectious diseases are low and prevention, control and management of people with non-communicable diseases is available, and when community members have access to basic services and healthcare. [The concept also] encompasses health beyond healthcare by providing a clean environment, potable water, good sanitation and open areas for safe and pleasurable recreation and physical activity. (http://www.worldbank.org/en/news/feature/2014/01/29/welcome-to-the-healthy-villages-in-china) viii % of women between age 15 and 49 who undergo gynecological check-up in last Indicator 11 : year Value 44.1% (with disparities (quantitative 54.1% N/A 66.3% across provinces) or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target surpassed by 12.2 percentage points. achievement) % of individuals over 35 with hypertension who have been followed-up in the Indicator 12 : three months. Value (quantitative 66.2% >80% N/A 89.4% or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target achieved. achievement) Annual per capita government expenditures on county level public health Indicator 13 : institutions and programs. Value (quantitative 14.6Yuan 20.9Yuan N/A 56Yuan or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments (incl. % Target surpassed. achievement) Indicator 14 : Project lessons and experiences are documented and disseminated (Qualitative). Value (quantitative 0 100% N/A 100% or Qualitative) Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 Comments Target achieved. All 40 project counties carried out dissemination of the (incl. % project experiences. achievement) Indicator 15 : Project experiences are extended or adopted outside project areas (Qualitative). 100%. 60% of the All 8 project project provinces provinces had Value disseminate carried out (quantitative 0 experiences in N/A promotion and or Qualitative) their respective application of non-project project experiences counties in non-project regions. Date achieved Dec. 31, 2008 Dec. 31, 2013 Dec. 31, 2014 ix Comments (incl. % Target surpassed the target by 40 percentage points. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 12/22/2008 Satisfactory Satisfactory 0.00 2 12/01/2009 Moderately Satisfactory Moderately Satisfactory 3.70 3 07/27/2010 Moderately Satisfactory Moderately Satisfactory 8.98 4 01/08/2011 Moderately Satisfactory Moderately Satisfactory 12.30 5 02/18/2012 Satisfactory Satisfactory 22.72 6 01/27/2013 Satisfactory Satisfactory 35.50 7 08/12/2013 Satisfactory Satisfactory 42.85 8 03/05/2014 Satisfactory Satisfactory 48.30 9 09/09/2014 Satisfactory Satisfactory 48.61 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions 03/31/2011 N MS MS 15.07 Reallocation of loan Proceeds 08/01/2012 N S S 27.49 Reallocation of loan Proceeds Extension of Closing Date to December 31, 2014. The Loan Agreement was amended as follows: The National Expert Panel replaced the National Validation Panel as the entity responsible for monitoring and 04/30/2013 N S S 38.63 evaluating the progress of sub- projects with focus on whether milestones set out in sub-project MOUs were achieved and whether remedial actions or rewards were warranted, depending on implementation progress of the sub-projects. Additionally, the Project x ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Outcome Indicator 106 was amended to read: “number of administrative villages that have initiated the implementation of 'healthy village"'. 6 In the PAD, PDO indicators and intermediate outcome indicators are numbered continuously. Therefore, indicator 10 in this document is equivalent to indicator 17 in original project documents. xi I. Disbursement Profile xii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. China witnessed a remarkable period of rapid growth in shifting from a centrally planned to a market based economy. Income per capita increased on average 8.5 percent annually from 1980–2007, more than in any other country.7 China’s economic miracle, which lifted approximately 640 million people above the international poverty line of $1.25/day, was pivotal in reducing global poverty from 1.5 billion people in the early 1980s to 800 million today.1,8 2. This period of unprecedented growth was, at least in part, due to health policies that yielded marked improvements in health outcomes in the preceding decades. Starting in the 1950s, through the deployment of “barefoot doctors” and other programs, China made substantial inroads in improving public health, resulting in a rapid decline in child mortality, from 225 deaths per 1,000 live births in 1960 to under 64 per 1,000 by 1980.1 Declines in child mortality and the accompanying reduction in the fertility rate, increased the portion of the working-age group in population and contributed to reducing the dependency ratio, spurring economic growth. Paradoxically, the fast pace of improvements in health status achieved from 1950–1980—when China had a worse relative position in the world in terms of its per capita income, economic growth and health infrastructure—was not matched during its economy’s strongest period of growth.2 Economic gains did not translate into health improvements for the entire Chinese population. The transition from a collective economy to a market economy resulted in the collapse of the commune-based cooperative medical scheme (CMS), which was the main source of health financing for the rural population. By 2003, eighty percent of China’s rural population was not covered by any health insurance. 3. By the turn of the millennia, the Government of China (GOC) had become increasingly aware of the challenges it faced in the health sector and began to formulate a series of policy initiatives to address them. The Government’s 2002 Paper “Decision on Further Strengthening Rural Health Work” laid the foundation for a new health financing system for rural areas, which included the New Cooperative Medical Scheme (NCMS) and the Medical Assistance Scheme (MA) as cornerstones. The 11th Five Year Plan for the period of 2006-2010 promoted “balanced development” and health sector development, particularly in rural areas, was one of the priorities of the Plan. The Government also launched a number of system wide health reforms—such as a comprehensive payment system reform, regional health service integration and development of multilevel management platform for prevention and treatment of noncommunicable diseases (NCDs)—though progress in implementation of these reforms 7 Adam Wagstaff, Magnus Lindelow, Shiyong Wang, and Shuo Zhang. 2009. Reforming China’s Rural Health System. Washington, D.C.: The World Bank. 8 Angus Deaton. The Great Escape. 2013. The Great Escape: Health, Wealth, and the Origins of Inequality. Princeton, N.J.: Princeton University Press. 1 was modest. More comprehensive reforms would be needed to overcome existing challenges and achieve the stated objectives of containing costs and improving provider performance. 4. At appraisal in 2008, the project identified four major challenges faced by the health sector: (a) persistence of significant health inequalities—for example, the maternal mortality ratio in the poorest quintile was three times higher than in the richest quintile; (b) decreasing affordability of health care as indicated by increasing out-of-pocket spending for health care and more people falling into poverty due to health spending; (c) poor quality and inefficiencies in the health system, with unnecessary prescriptions and longer-than- needed hospital stays; and (d) weaknesses in execution of public health functions, as evidenced by the response to the 2003 SARs outbreak. Pilots financed under the project would test innovative approaches to address these challenges. 5. The project was aligned with the World Bank Group’s Country Partnership Strategy (CPS) for China 2006–2010 (Report No. 35435) discussed by the Board of Directors on May 23, 2006 and the GOC’s co-terminus 11th Five Year Plan. Specifically, the project was expected to contribute to the CPS outcomes of (a) strengthened social insurance, including rural health insurance, and (b) increased and more appropriate use of health services, especially among the rural poor. The rationale for the World Bank’s involvement was twofold. First, the World Bank had in-depth knowledge of the country, built through its long-standing partnership with the GOC in the health sector, dating back to the early 1980s and involving ten health sector operations. Second, the World Bank brought global knowledge from working with client countries on topics relevant to China, such as health insurance, provider payment methods, and delivery of quality NCD care. Indeed, in preparation for the 11th five-year plan, the GOC had requested World Bank’s assistance in carrying out an extensive program of Analytic and Advisory Activities (AAA) to capitalize on the ability of the World Bank to draw on international experiences relevant to a middle-income. More so than the incremental financing it generated,9 the value-added of the project was to test various policy options and develop new approaches that can be expanded nationwide to strengthen the health system so it can more effectively address the changing health needs of the population. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 6. The PDO was to achieve increased and more equitable access to quality health services, improved financial protection, and better management of public health threats in pilot provinces and counties, with lessons to support reforms in non-project areas. The project’s results were measured through seven PDO indicators and 15 intermediate results indicators.10 9 Project financing represented less than 1% of government health expenditures. 10 PDO indicator is shorthand for PDO Level Results Indicator from the April 2013 restructuring paper or Project Outcome Indicator from the Project Appraisal Document (PAD). Intermediate result indicator is the term used in the restructuring paper and replaces the PAD’s Project Outcome Indicator. 2 7. The Project’s PDO indicators were: a. PDO Indicator 1: Ratio of hospitalization rate of MA (Medical Assistance scheme) vs. non-MA beneficiaries; b. PDO Indicator 2: Ratio of outpatient visit rate of the bottom 20% vs. the top 40% in the income distribution in the previous year; c. PDO Indicator 3: % of households with annual health expenditure in excess of 20% of total income; d. PDO Indicator 4: Rate of overall satisfaction with rural health services among men; e. PDO Indicator 5: Rate of overall satisfaction with rural health services among women; f. PDO Indicator 6: Public Health System scorecard rating; and g. PDO Indicator 7: Innovation accepted and rolled out at provincial or regional level.  1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 8.  The PDO was not revised during project implementation. However, one intermediate result indicator was modified during the April 2013 project restructuring. Indicator2 Original indicator Modified indicator Reason for change 10 % of villages that Number of The ‘health village’ criteria meet ‘healthy administrative are very comprehensive and village’ standard villages that have multi-dimensional, requiring initiated the efforts across sectors and implementation of levels of government, some “healthy village” of which are outside the sphere of control of the project. The revised indicator measures progress of activities that were within the purview of the counties participating in the project. 1.4 Main Beneficiaries   9. The primary beneficiaries of the project were to be the rural residents in the 40 project counties, with a population of 20.6 million, across eight provinces/ municipality (Gansu, Heilongjiang, Henan, Jiangsu, Qinghai, Shaanxi and Shanxi; Chongqing).11 Additionally, knowledge generated from the pilots was intended to benefit 11 For the sake of brevity, in this document we use the term provinces when referring to the seven provinces and one municipality in which the Project was active. 3 non-project counties. Project outputs, such as technical guidelines and innovative practices, have been disseminated to non-project areas and some have become government policies and laws, which benefits a much larger population within China. In addition, the knowledge and good practices generated from the project have been disseminated to Uzbekistan, Tajikistan, Laos, Vietnam, Myanmar, Thailand and Cambodia through workshops and exchange visits, and are expected to benefit more people globally. 1.5 Original Components 10. The project consisted of two components, with the following initial loan allocation (final expenditure in parenthesis) and main activities (Annex 2). 11. Component 1: Health Reform Innovations Supported via Block Grants. The project was to support innovations that would contribute lessons on practical approaches for implementing government health reform policies. The primary activity of the project was to pilot and then evaluate innovations in reform areas the government identified as priorities. At a minimum, pilots had to involve: (a) measures to improve equity of access to quality health services, including by gender; (b) measures to strengthen the NCMS design and management (management information systems, governance and accountability arrangements, resource mobilization, benefit package design, links with the MA program, etc.); (c) reforms of provider payment methods; (d) reforms to enhance quality that link quality standards and procedures with financial and other incentives; and (e) institutional and financing reforms to improve public health functions and delivery of public health services. Counties were to receive block grants to finance pilot activities. Forty counties were selected to implement the project. These counties would submit proposals to be evaluated according to the agreed upon criteria, including demonstrated commitment to reform, piloting innovations in line with the project concept documents, ability and capacity to lead and support innovations, and overall quality of the proposal. 12. Three reform areas were identified:  Reform Area One: Improving Rural Health Financing. This reform area covered efforts to: (a) develop information flows and integrate information systems, and strengthen fund management and control capability; (b) introduce international experience on feasible approaches to control costs, and pilot alternative provider payment mechanisms, such as capitation and diagnostic-related groupings; (c) better link the NCMS and MA programs; (d) improve financial protection while improving access to affordable services for currently excluded or marginalized groups; (e) strengthen mechanisms for supervision and broader accountability to the community; and (f) train and build capacity, beyond routing training activities supported by national programs.  Reform Area Two: Improving Quality, Efficiency and Cost Control in Service Delivery. The main goals under this reform area were: (a) effective control of health care costs; (b) improved technical quality of care; (c) increased internal efficiency in service delivery; and (d) increased focus on prevention and health promotion. To achieve these goals, the project was to support interventions to: (a) address existing weaknesses in 4 provider capacity to define performance standards, monitor costs and performance, enforce contractual obligations of hospitals and other providers; (b) develop and implement an integrated primary care model; (c) provide technical input to support the development and implementation of new or revised regulations, and of systems for regulatory monitoring and enforcement; and (d) upgrade material conditions and capacity of rural health providers through limited training and civil works to ensure adequate skills and capabilities in the health system to meet higher quality standards and requirements.  Reform Area Three: Strengthening the Financing and Organization of Core Public Health Functions. Under this reform area, the project was to support the development of a coordinated public health strategy at the county level; to test different approaches for improving the quality, efficiency and equity of existing services; and pilot new interventions to address emerging health challenges (such as NCDs and associated behavioral risk factors). The approaches piloted would include: (a) evidence-based and multi-sectoral planning and coordination for public health service provision; (b) purchasing of clinical public health services; (c) financing and service models—including outreach and incentives for health workers—to improve access for remote populations and other vulnerable groups in the community; (d) community mobilization mechanisms for public health emergencies; (e) management of NCDs according to standardized protocols; (f) disease surveillance at the county and village level; and (g) monitoring equity of access to and satisfaction with clinical public health services. 13. Component 2: Project Coordination, Policy Development and Lesson Learning. This project coordination, policy development and lesson learned component aimed to guide project implementation and identify replicable models for rural health reform. The major principles informing the operation of this component were to include: (a) focusing on learning-through short feedback loops, policy debates, and special studies; (b) actively using monitoring indicators to assess progress and manage fund release; (c) facilitating efficient implementation and course adjustment for the project; (d) flexibility based on clear business rules; and (e) an integrated multi-sectoral and cross-component approach with equity concerns integrated throughout. 1.6 Revised Components 14. The components did not change during project implementation. 1.7 Other significant changes 15. The project was restructured three times during implementation. The March 2011 and June 2012 restructurings reallocated loan proceeds based on project progress and need to achieve the PDOs; the June 2012 restructuring also merged the expenditure Category A (block grants) and Category B (goods, consultant services, training, Program Support) into a single Category C in order to ease disbursement management. The final restructuring (April 2013): (a) extended the Closing Date from December 31, 2013 to December 31, 2014 to allow sufficient time for the project to complete the latest pilots, carry out evaluations of the new pilots, and disseminate and roll out to non-pilot areas; (b) 5 removed reference to the National Validation Panel in the Loan Agreement and replaced it with the National Expert Panel (NEP), as the NEP had been carrying out this function during project implementation; and (c) revised the definition of indicator 10 to better align and assess the progress of implementation on the ground. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 16. A thorough and solid sector analysis was beneficial to project design, but preparation was long. Project identification and design were informed by the findings of the Rural Health Reform Analytic and Advisory Activities (AAA) (P082296), which drew on the World Bank’s global knowledge gained in supporting health reform in wide range of countries. The AAA clearly identified policy and institutional deficiencies present in the Chinese health system at the time and pointed to the need to test concrete operational steps to implement reform. Project design took into account the priorities identified in the AAA and focused on leaning-by-doing and on the institutional development needed to support reform. Building on the lessons learned from other countries' efforts to implement health reform, the preparation team suggested various policy innovations that the project might test within three main reform areas: health financing, service provision and public health. Successful pilots could then be expanded to other areas in China or even other countries. Although the AAA was beneficial, delays in reaching an agreement between the World Bank and the GOC on final publication of the AAA report, Reforming China’s Rural Health System, did have the unintended consequence of slowing down project preparation. Project preparation took three and a half years from the pre-identification mission in November 2004 to the appraisal mission in September 2007 to Board approval in June 2008. Delays in finalizing the project design were also partly due to the Government’s decision to expand the scope and coverage of the project by introducing new provinces. It took the Government almost one year to reach a decision about the selection of these additional provinces. In the end, five provinces (Gansu, Heilongjinag, Henan, Jiangsu and Shanxi) were added to the original three provinces at appraisal, which increased number of project counties from 24 to 40 and more than doubled the population in project areas. In retrospect, the long design had some unexpected benefits; it allowed for a thorough analysis and made the project timing align more closely with the Government’s health reform timetable. 17. The project had an innovative and ambitious design. The project had several design features that had not previously been part of World Bank supported health projects in China, namely: (a) provision of block grants to project counties with the central and provincial Government (instead of counties) being responsible for repayment; (b) introduction of the incentive and exit mechanisms to manage performance and encourage healthy competition; and (c) using bottom up approaches to empower counties to design their project activities. Such project design features provided incentives for innovation and for promoting changes at local level, without increasing the financial burden on rural counties. However, the project had its downside. It was designed to pilot very difficult reform measures in three areas (financing, service delivery and public health function) in 6 40 counties in eight provinces. This is a complex design that requires intensive supervision and technical support.  18. Project design incorporated lessons learned from previous operations in the health sector. The Basic Health Services (Health VIII) Project, successfully closed on June 30, 2007, was particularly relevant. It was an important pre-cursor to this project because it pioneered the development of new approaches that built upon government policy initiatives in rural areas. A noteworthy example is the Medical Financial Assistance program, which provided the foundation for the current Medical Assistance scheme that is now been rolled out nationwide. An important lesson from Health VIII concerns the vital role of technical assistance in supporting institutional change at the local level. From the beginning, project preparation fully utilized the capacity of the Government and national experts. A great deal of work was undertaken by the Borrower at central, provincial and county levels. The Center for Project Supervision and Management (CPSM) 12 and the National Experts Panel (NEP) played critical roles in project preparation. There was no Quality Enhancement Review for this project. 19. The project is closely aligned with the Government’s overall reform agenda in rural China. The project was initiated when the GOC was preparing a comprehensive health reform. The aim of the project was to pilot changes on a small scale, document the results and accumulate experiences that could be used to deepen and broaden the reform, including by rolling out the pilots nationwide. 2.2 Implementation 20. The project operated under a highly volatile and fast-evolving policy environment. The project design was intended to test reform measures at a small scale first, to inform the Government’s overall Rural Health Reform. However, when the project was under the second year of the implementation, without waiting for results of the pilots, the Government launched its health reform with a significant increase in investment in NCMS and rapid rollout of the MA. The Government specified some rules and procedures for the NCMS and MA that every county needed to follow. However, the reform launched nationwide was a general policy framework and it was up to the provinces to figure out how to implement the reform in their local areas. The project made adjustments to stay relevant and meet the changing needs of the reform. Taking advantage of the enabling environment at the national level, the project fund allocation and procurement plan for civil works and equipment were changed quickly to avoid duplication in investment and pilots were focused on testing various pathways for implementation of the reform. 21. A midterm review (MTR) and subsequent restructurings of the project made positive changes. The MTR was conducted in September 2011, as originally planned. It was done jointly with the GOC and Department for International Development (DFID). As a result of the MTR, some procedures were simplified. For example, before the MTR, the 12 Previously called the Foreign Loan Office (FLO). 7 annual activity plans at the central and provincial level were approved by the World Bank and county plans by CPSM, which led to delays. The MTR decentralized the power of approval for county plans from CPSM to Provincial PMOs, with CPSM and NEP making spot checks of county plans. The project had also been restructured to reallocate funds according to need. The amount allocated to civil works and equipment was reduced, whereas funds for training and technical assistance were increased. The project was restructured again later to reduced disbursement categories from two to one, which further simplified project design. The speed of implementation was improved as a result. 22. The incentive and exit system built into the project design was very effective. The project innovatively designed an incentive and exit system that rewarded best performers and let poor performers leave the project. Twice during implementation, the project rewarded counties that were making fast progress; this produced excellent effects. Twenty-two project counties were rewarded and their enthusiasm for better performance and innovation was further stimulated. At the same time, five project counties were identified as being poor performers, received targeted support and were asked to carry out rectification measures. Even though in the end no project county was removed from the project, this system put political pressure on poorly performing counties to improve and injected energy and motivation for good performing counties to do even better. As a result, all 40 counties put forth significant effort during implementation. 23. Implementation capacity was uneven across project counties and provinces. Provinces and counties that had past experience implementing World Bank supported projects, or had strong leaders, were more efficient in preparing and implementing the annual plans. These were provinces where capacity was higher in general. Frequent shuffling of personnel in some provinces or counties led to a gap in both knowledge and institutional memory, negatively affecting implementation. Overall, implementation progress varied greatly among provinces and counties. Although some measures were adopted such as providing special TA to support weaker counties to try to narrow the gap, the results were not always satisfactory. 24. Changed exchange rate affected the use of the total projected project expenditures. Due to exchange rate fluctuations, the project experienced a reduction in funding in local currency terms. For instance, the DFID grant for Gansu was in the amount of GBP465,000, which was equivalent to RMB 6.51 million at project appraisal (1 GBP=14 RMB Yuan). By implementation, the grant amounted to RMB 4.67 million; a loss of RMB 1.84 million or 28.3 percent of the value of the grant. The World Bank loan of US$50 million was equivalent to RMB 350 million at appraisal (1 US$ to 7 RMB), but the actual amount was RMB 310 million by the end of the project completion, a loss of RMB 40 million or 11.4 percent. Planned activities had to be adjusted and/or financed with Government funds to account for the shortfall. 25. At the request of the Government, the project was extended for one year. This allowed sufficient time for completion of the latest pilots, for evaluations of the new pilots to be conducted, and for innovations to be disseminated and rolled out to non-pilot areas. 8 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 26. M&E Design. The project adopted a result-oriented management method and had a clearly defined results monitoring and evaluation framework, with well-specified indicators. Most of the indicators in the Results Framework measured high-level outcomes and relied on survey data to be updated; only five out of the 22 indicators in Results Framework relied on routine data from the NCMS or the health management information system. Therefore, the project was designed to include three surveys (at baseline, mid-term and end-of-project) to assess progress toward achievement of the PDOs. In addition, assessment of progress was done by reviewing implementation of activities in the approved project annual activity plan. Project counties identified triggers for each important pilot activity, which was used as a precondition for the disbursement application for project funds. CPSM compiled an M&E Operational Manual that specified the contents, methods and division of tasks between institutions at different levels. CPSM also tracked 23 regular process indicators, in accordance with the national reform monitoring framework. This data was used to inform supervision and decision-making during the life of the project. 27. M&E Implementation and Utilization. A project management information system (MIS) was developed and used to collect, store, process, retrieve, transmit and aggregate information. The MIS was used to provide the project with comprehensive, real-time and interactive information. It had 10 modules including planning management, basic information management, objectives management, procurement management, financial management and data management. Since it became operational in February 2010, it continuously produced six-monthly M&E reports and annual reports. The MIS also established an online reporting system for routine project M&E, which included output and process indicators; this rectified the weakness of the Results Framework that was noted above. It was used not only as a source of data for project supervision, the mid-term review and final evaluation, but also as the basis for the incentive and exit systems. Project surveys were synchronized with the national health surveys to allow comparisons and analyses between the project and the broader national health reform. The project mid-term and final evaluations were conducted at the same time as the 4th and 5th National Health Survey, respectively, allowing comparable information to be collected for non-project areas.13 2.4 Safeguard and Fiduciary Compliance 28. Environmental Safeguards. The project was classified as Category B based on two potential environment impacts: (a) new construction and rehabilitation of village clinics, and (b) medical waste disposal at village clinics. The project followed the Government’s national guidelines on medical waste disposal and chemicals use, and 13 The three surveys were conducted by Center for Health Statistics and Information of NHFPC and the total sample households of 36,000 from 40 project counties in 8 project provinces and with about 110,000 people. The detailed survey method, content and quality control were described in “the Final Survey Report for China Rural Health Development Project” NHFPC October 2014. 9 developed a Medical Waste Management Plan (EMP) as part of the Project Operations Manual. Since the primary focus of the project was on testing the viability of institutional innovations, such as cost control and improving health insurance, a substantial increase in medical waste related to the project was unlikely. Indeed, no significant safeguards issues surfaced during implementation. 29. Social and Indigenous People Safeguards. The Involuntary Resettlement (OP/BP 4.12) and Indigenous Peoples (OP/BP 4.10) safeguard policies were triggered. The Resettlement Policy Framework and the Ethnic Minority (Indigenous People) Development Planning Framework were developed during project preparation. Refurbishment and construction of facilities were mainly done on existing sites and no outstanding issues found during supervision. The ethnic minority development plan was satisfactorily implemented and some preferential policies were given to ethnic minorities such as higher reimbursement rate for their medical expenses. The implementation of safeguards was satisfactory. 30. Fiduciary. The project maintained acceptable financial management systems that provided basic financial information. Project funds were used for the intended purposes and adequate accounts were managed. Some innovative features, such as establishing Operating Accounts in County Finance Bureaus and Report-based Disbursement, were implemented to meet project needs. These mechanisms improved disbursement efficiency and were particularly beneficial in a decentralized project, such as this one. However, because there was no counterpart fund requirement under this project, some counties encountered problems related to lack of working capital that delayed project implementation. 2.5 Post-completion Operation/Next Phase 31. Rural health care reform is ongoing in China and the outputs and outcomes from the project are likely to be sustained and even expanded nationwide. The project piloted various interventions that are well aligned with the Government’s rural health reform policies and there is buy-in at the highest level to scale up successful experiences. Indeed, some innovations developed under the project have become law and are fully institutionalized. Additionally, as confirmed in interviews with the Government, one of the long-lasting contributions of the project is that it built technical and managerial capacity, expanded the knowledge base and created a change in the mindset of implementers, encouraging innovation and experimentation from the bottom up. The health reform process in China is ongoing; project reform experiences will continue to provide a basis for Government to adjust and refine national health policies and implementation of those policies at the subnational level. 32. The World Bank and the Government are discussing a follow-on operation that will build upon the experiences of this project and support the Government in deepening the rural health reforms that have been started. The new project is expected to support selected provincial governments in implementing service delivery reforms to enable the health system to more effectively respond to the health needs of an aging 10 population facing an increased burden of noncommunicable diseases. The project is currently under preparation (the estimated Board date is October 2017. The estimated loan amount is US$400 million to be allocated across three to four provinces. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 33. The PDOs were highly relevant at appraisal and remained so throughout the implementation and consistent with China’s priorities in the rural health system reform. China’s 12th Five-Year Plan has a continued focus on improving quality of growth and addressing economic and social imbalances, including in financial protection from health shocks and access to quality health services. Innovations financed and technically supported by the project have helped operationalize reforms that move China closer to the goals outline in the 12th Five-Year Plan. After reviewed the report on this project prepared by the Ministry of Finance, Vice-Premier Liu Yandong requested rollout of pilots to non- project areas. This further validates the current relevance of the project objectives, implementation and design. 34. The drive toward universal health coverage has gained importance in the post- 2014 development agenda, as the world defines the targets that will replace the 2015 Millennium Development Goals targets. The proposed Sustainable Development Goal 3—to ensure healthy lives and promote well-being for all at all ages—includes a target on achieving universal health coverage (UHC), which is defined as “ensuring that all people can use the promotive, preventive, curative, rehabilitative and palliative health services they need, of sufficient quality to be effective, while also ensuring that the use of these services does not expose the user to financial hardship.”14 This definition embodies three related objectives and project design: (a) equity in access to health services; (b) that the quality of health services is good enough to improve the health of those receiving services; and (c) financial-risk protection. The project is fully aligned with each of these objectives. 35. Project objectives are consistent with the World Bank’s twin goals of ending extreme poverty by 2030 and promoting shared prosperity among the poorest 40 percent of the population. In particular, by improving equitable access to quality and financial protection, the project contributes to achievement of the World Bank’s Health, Nutrition and Population Global Practice pledged to assist client countries in accelerating progress towards UHC as a means to achieve the twin goals. 36. The project supports the implementation of the World Bank’s Country Partnership Strategy for China for the period FY2013-2016 (Report No. 67566-CN, October 11, 2012). The CPS intended to exploit synergies between investment lending and analytical work, while recognizing that demonstration projects supported through lending introduce and localize international knowledge and experience. The CPS, supporting more 14 http://www.who.int/health_financing/universal_coverage_definition/en/ 11 inclusive and balanced rural development, is also aligned with China's 12th Five-Year Plan. In its review of the CPS, the Board noted that its success would hinge on transfer of knowledge through innovative projects that are sustainably replicated across the country. Further, the Board supported continued enhancement of knowledge services and innovation under the CPS, and stressed the importance of drawing broader lessons for World Bank work in other countries, particularly middle-income countries. This project piloted and tested integrated rural health reform programs in 40 counties in eight provinces and has proven to be a useful vehicle for knowledge generation and sharing. As indicated earlier, knowledge from the project has been disseminated to visiting delegation from Uzbekistan, Tajikistan, Laos, Vietnam, Myanmar, Thailand and Cambodia. 3.2 Achievement of Project Development Objectives 37. Project efficacy is evaluated based on achievement of the PDO: increased and more equitable access to quality health services, improved financial protection, and better management of public health threats in pilot provinces and counties, with lessons to support reforms in non-project areas.     38. Achievement of each of these four project intermediate objectives was substantial. Three PDO indicators measured achievement of increased and more equitable access to quality services, of which all were achieved and two were surpassed. The access to and utilization of health care have increased, particularly by the poor. The number of counties where the poor utilized outpatient services less than the rich (despite having greater need) dropped by two-thirds. Quality of services was also enhanced as evidenced by increased overall satisfaction (proxy indicator) with rural health services by both male and female patients (increasing from 63.8 to 85.4 percent for man and 66.6 to 85.5 percent for woman, and surpassing the target by 17.6 and 14.9 percentage points for men and women respectively). Two PDO indicators measured achievement of improved financial protection. The share of households with health expenditures in excess of 20 percent of income decreased and the 15.2 percent target was surpassed by 2 percentage points. Membership in the Medical Assistance Scheme was associated with higher utilization of hospital care, indicating that those covered by the scheme have fewer barriers to access. Better management of public health threats was measured using the public health system scorecard; the score at the end of the project was 90.22, compared with 61.78 at the onset of the project. Finally, a qualitative indicator was used to gauge achievement of the crosscutting intermediate objective that implementation of activities in project areas yield lessons to support reforms in non-project areas. The target was also surpassed. As noted earlier, lessons from the pilots were widely disseminated domestically and internationally. Meeting all PDO targets is a considerable achievement given that these are high-level indicators that require interventions on multiple fronts. PDO indicators IO indicators Surpassed 4 8 Achieved 3 6 Not-achieved/NA 1 Total number met 7 (100%) 14 (93%) 12 39. The project has had a profound impact on rural health reform beyond what can be measured through its PDO and intermediate results indicators. During project implementation, the GOC launched a major rural health reform policy that included rapid scale up of the NCMS15 and MA, backed by considerable increases in public investments in the health sector, which undoubtedly contribute to achievement of PDOs. However, as is typically, the case in China, policies are defined in broad strokes, without providing details on how they should be operationalized. A major contribution of the project was that it supported the design and testing of approaches to operationalize the reform, drawing on international best practices. Additionally, the project achieved important results that are not necessarily captured by the indicators in the Results Framework. These include intangible, but important result such as the shift towards evidence-based management and policy decision-making, more openness to new ideas, and willingness to experiment. These changes will have a lasting effect on the development of China’s health sector.    40. Some of the main accomplishments of the project pilots in each of the three reform areas are highlighted below. Annex 2 has a more comprehensive list of outputs by component and Annex 3 has a discussion of the linkages between project activities and results, which illustrates how the project contributed to program outcomes.    Reform Area One: Improving Rural Health Financing 41. The project played a key role in reforming financing mechanisms under the NCMS, which increased resources available to finance services. Exploring financing modalities for the NCMS at the local level was a core project activity. Project counties were innovative in creating ways to increase the reimbursement rate under the NCMS, which in turn allowed them to expand services coverage. Many models were created and tested, including the ‘Health Service All-in-One Card’, ‘three-fixed financing’, ‘rolling financing’, and ‘bank withholding and collecting’, among others. Some of these models also reduced the financial burden on NCD patients of obtain care for services covered under the NCMS. The hospitalization rate in the eight project provinces increased from 6.6 percent in 2008 to 8.9 percent in 2013, even though the cost of hospitalization increase 76.3 percent during the same period. Affordability of hospitalization increased with expansion of NCMS coverage and out-of-pocket payments for hospitalization declined by more than 50 percent in this period. 42. Under the project, the Government established innovative NCMS payment systems to increase sustainability of the NCMS. Project counties explored various options to pay providers for services covered under the NCMS. Payment methods were tested for both outpatient services (e.g., per capita payments, lump-sum payment for total expenditures) and inpatient care (e.g., per capita hospitalization payments, payment per patient discharge, per diem payments, single disease case-based payments, and payment for diagnostic-related groupings). Among 40 project counties, 22 counties implemented outpatient lump-sum payments, covering 40 county-level medical institutions, and 365 15 During project implementation, health insurance coverage expanded rapidly, reaching 96 percent across the project counties. 13 hospitals and township health centers. In 23 counties, 87 county-level hospitals and 413 township-level hospitals and health centers used inpatient single disease payment systems, while the classified and grouped comprehensive payment system was implemented in 8 counties, covering 36 county-level medical institutions and 154 township-level hospitals and health centers. A book entitled "NCMS Payment Reform" was produced by the project and recommended by the Central Government as reference material for national trainings. The project also produced an "Operational Guide for Payment Reform" for use in project counties and a book, "Take the Lead in Payment Reform Innovation, Realize the Double Coverage of Payment Reform," which summarizes the innovations in payment reform implemented under the project. Some of the models tested are now the standard financing policy for the local NCMS and have been adopted in non-project counties.  43. The project strengthened the rural medical insurance system and increased financial accessibility by the poor. The rural medical insurance system consists of the NCMS and MA scheme. Project counties explored ways to improve coverage and sustainability of the MA. Efforts included expanding benefit packages, increasing reimbursement rates and capturing more beneficiaries under the MA. Project counties established MA networks with 170 county hospitals and 1172 township health centers in the project areas. Some counties lowered the deductible for inpatient services for the poor, reduced or waived their medical bills, or provided short-term assistance for people who are slightly above the qualification threshold for the MA. These measures have significantly improved accessibility to rural health services and equity. For instance, there is now a greater gap in use of hospitalization services by MA beneficiaries and non-beneficiaries than at the beginning of the project, an indication that affordability has improved for those covered by the scheme. Access to outpatient care for the poor also improved. There has been an increase in the number of counties in which the poor use relatively more services than the rich; this is a desirable outcome given that the poor are likely to have greater health needs.     Reform Area Two: Improving Quality, Efficiency and Cost Control in Service Delivery 44. Under the project, the Government formulated a quality-based and service- oriented long-term performance management system. Since 2009, project counties began to implement the system, which included quantitative evaluation indicators. Some project areas like Jiangsu, Chongqing, Qinghai and Henan built a dynamic system in which indicators were given different weights in accessing performance, based on priorities, degree of difficulties of tasks and quality of services. The performance management system built a direct connection between performance and compensation, effectively enhancing the quality and efficiency of health services in the project areas. 45. The project improved rural healthcare infrastructure and expanded physical access to care. The project financed construction (1109), expansion (599) and renovation (582) of village clinics, as well as clinical and office equipment (62,880 pieces in all). The share of the population living within 15 minutes of the nearest heath facility increased by 9.4 percentage points on average. In Gansu, the province with the most significant improvement, the rate increased by 28.9 percentage points, from 53.6 percent in 2008 to 14 82.7 percent in 2013. These investments also improved structural quality of village clinics, enabling them to provide better services. 46. The project strengthened the rural health workforce by developing and implementing plans for recruiting, retaining and building capacity of health staff. Most project areas developed recruitment and management mechanisms for health and technical personnel at the township and village levels. For example, Jiulongpo District in Chongqing standardized the recruitment procedures for village doctors and attracted young village doctors by including them into the civil servant system. Gaochun in Jiangsu and Yongchuan in Chongqing introduced a development plan for village doctors to obtain a college degree, which had a strong impact on the education level and age structure of village doctors in their areas. All project counties developed training plans and provided capacity-building opportunities, adapted to local needs, for rural health workers in their territories. For example, Heilongjiang and Jiangsu built training schools for village doctors. Jiangsu provided education and training that allowed village doctors to meet current educational requirements. Project counties, such as Zhen’an in Shaanxi and Huining in Gansu, set up training bases for village health personnel. Jiulongpo in Chongqing and Linkou in Heilongjiang applied a “master-guides-apprentice” model that updated staff technical skills, strengthening the 3-tier rural health system. 47. The project has supported comprehensive public hospital reform in some project counties with a focus on changing the payment system from fee-for-service to case-based payment. Implementation of IT-based clinical pathways has been carried out with the aim to standardize quality of services and control costs. Along with the introduction of performance-based payment system, these reform activities have led to better care, shorter length of hospital stay and lower costs for rural patients. Reform Area Three: Strengthening the Financing and Organization of Core Public Health Functions 48. The project improved management of essential public health functions. An “Operational Guide for Management and Implementation of Basic Public Health Service” was developed and implemented early on in the project. Taking into consideration the specific public health issues relevant to an area, the area’s level of economic development and local capacity for service provision, project counties created tailored approaches to incentivize supply of public health services, including by introducing performance evaluation and linking performance to payment. 49. Management improved by promoting a culture of evaluation. All project counties developed and implemented evaluations of basic public health performance by 2010. The project conducted a “comprehensive evaluation” connecting essential public health functions with basic medical care, which included a third-party evaluation mechanism to assess results in providing basic public health services. Survey results show improvement in NCD control and prevention. From 2008 to 2013, the hypertension prevention and control guidance rate under the project increased from 76.8 to 91.3 percent and the compliance rate for diabetes treatment reached 94 percent. 15 50. Under the project, counties operationalized the “Healthy Village” model as a means to promote healthy lifestyles and prevent NCDs. A Healthy Village Pilot Construction Brochure and a Manual for Healthy Village Construction were developed to guide counties in defining local requirements to establish a Healthy Village that would take into consideration the county’s health problems, environment, and capacity. Each project county explored a different "healthy village" format that could be feasibly implemented given their particular circumstances. At the end of the project, 35 out of 40 project counties initiated Healthy Villages and 277 villages had met the Healthy Village standard. Full implementation is still underway and it is too early to assess its impact, as behavior-change is a slow process. 51. The project explored ways to improve health promotion and communication at the local level. Based on the requirements of the Government’s “National Public Health Service Specification,” the project developed health promotion models and special health education materials. This initiative targeted housewives, motivating them to take charge of family and community health. Some project counties carried out a wide range of health promotion activities, such as financing the local Health Education Association, promoting health self-management, establishing health education websites and setting up a public health service team called “Sunshine Rider,” and biking to villages to promote health education. The project’s final survey indicated a 0.8 percentage point reduction in smoking and an 8.9 percentage point increase in regular exercising compared with the 2003 baseline survey. 52. A multi-sector management platform for NCD prevention and control was established. The project adopted a comprehensive multilevel approach to NCD prevention and control, including multi-sector coordination, comprehensive training, and innovative health promotion and provision of preventive services. The project focused on improving the capacity at grassroots health institutions for NCD prevention and control so that these institutions are able to carry out health promotion and early detection in rural communities. Improving management capacity at all levels 53. Knowledge generation and transfer were major contributions of the project. Project preparation and early implementation included the use of both international and domestic technical assistance for knowledge transfer and this continued to be a key element of the project through implementation. The DFID grant of GBP 1.2 million financed international technical assistance, as well as overseas study tours and training. After the closure of the DFID grant, technical assistance was funded by the World Bank. During project implementation, 22 international technical assistance missions were organized, with international experts providing, after each mission, written reports that summarized their findings and recommendations. The technical assistance covered priority areas for which China needed support, such as clinical path management, performance management and payment system reform. The implementing agencies acknowledged that the most important impact of this project was knowledge transfer and human capacity strengthening at the local level.  16 54. The overseas training and exchanges organized by the project played an important role in introducing innovative ideas and appropriate technologies, and in having pilots that were done in a scientific and orderly manner. The project organized 12 overseas visits and training for relevant project staff. These overseas visits, training and exchange workshops were considered very useful by project staff of all levels. The study tours introduced new ideas in project management, and increased cooperation between the project and international organizations. Exchanging international experiences and ideas with neighboring countries was another effective ways of generating and transferring knowledge. The experiences from the project were shared with other countries and project staff also benefitted tremendously from the process of identifying best practices and summarizing lessons learned. 55. The project strengthened human capacity in the rural health system, which had a positive impact beyond the project areas. A lasting impact of the project is that it build technical and managerial skills among health staff. Project management staff at each level were subsequently promoted and assumed positions of responsibility in health care reform. More than half of the project staff in Jiulongpo District of Chongqing, for instance, were promoted to other managerial positions by the end of the project. Project staff reported that working on this project was like going to graduate school, as it required hard work but that the reward was immeasurable. Focused on Knowledge generation and dissemination 56. Operational research under the project was key to generating knowledge. Eleven operational research studies were organized at the central level covering difficult topics in each project component. Research results have been used for revising and refining reform policies. For example, the public health system scorecard and the ‘Healthy Village’ guidelines developed by the project were based on initial research results and since then these have been developed into operational standards to guide implementation. Noteworthy is the regulation on increasing NCMS financing, promulgated by the Jiangsu government. This regulation was based on operational research carried out by the project. Such pioneering work has provided evidence for the State and other provinces to identify ways to increase the NCMS financing.  3.3 Efficiency 57. The efficiency of the project is deemed to be Substantial. The challenges of conducting the economic analysis for this project also existed at the time of preparation and they remain valid today. Many economic benefits, such as improved equity, are difficult to quantify in monetary terms; there is scant evidence on which to base assumptions (and, indeed, some assumptions made in cost-benefit analysis at project preparation—e.g., a decline in total health expenditure in project counties—did not hold); and, it is difficult to measure quantitatively the spillover effects of the pilot experience, which were about knowledge building and creating an openness to innovation and experimentation. For these reasons, the net present value (NPV) or the project’s economic rate of return (ERR) were not calculated. Instead, the economic analysis in Annex 3, 17 illustrates how the project activities contributed to economic benefits and positive results. Data from before and after the project show improved financial protection (illness-related expenses were less likely to be a cause of poverty); increased service utilization, particularly by the poor; better quality of services; and an improved public health environment. In addition, the project contributed to generation and transfer of knowledge about health sector reform and health management, and to strengthening capacity in the sector. Given its overall achievements relative to the small financial contribution of the project (less than 1 percent of government health expenditures in the project areas during the loan period), the project was highly efficient. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 58. The overall outcome rating of the China Rural Development Project is Satisfactory. This is based on: (i) a high overall relevance; (ii) substantial efficacy based on achievement of results across all intermediate objectives; and (iii) substantial efficiency. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 59. The project had a focus on poverty and gender-related issues in the health sector. The project improved equity in access to care by building new health facilities, as well as improving the structural quality of existing health facilities. The project also reduced direct payments for health services, making services more affordable for the poor and reducing financial barriers to accessing needed services. Project surveys suggest access to and utilization of health services by the poor improved in the project period. The ratio of hospitalization rates of MA and non-MA beneficiaries has increased by 0.6 percent point. The share of households with annual health expenditure in excess of 20 percent of total household income declined from 19.2 to 13.2 percent. 60. The project also showed improvements in indictors to monitor gender equality and addressed special health needs of women. The share of women who have access to a qualified health worker in project areas increased from 92.6 to 95.9 percent. The share of women age 15–49 who undergo gynecological check-up grew from 44.1 to 66.3 percent, with Gansu showing an impressive two-fold increase in the rate. On quality of services provided, the project collected gender disaggregated indicators in order to monitor potential disparities. There was virtually no gender difference in satisfaction and the majority of the population (85.4 percent of men and 85.5 percent of women) expressed satisfaction with the health services provided, a marked improvement from the baseline (63.8 percent for man and 66.6 percent for woman). (b) Institutional Change/Strengthening 61. The project made a significant contribution to improving the institutional infrastructure at the grassroots level. It has strengthened staff capacity to formulate health policies and implement reforms at the county, provincial and central levels. The 18 establishment of an improved information system, bolstered by specific surveys, broadened the evidence base on which to ground decision-making. Significant capacity was built, particularly in these areas: (i) development, implementation and monitoring of annual work plans; (ii) operational research to provide additional evidence for policy-making; (iii) collecting, analyzing and using data to monitor service quality; (iv) effective management of health facilities; and (v) development of complex administrative processes to implement NCMS and MA schemes. (c) Other Unintended Outcomes and Impacts (positive or negative) 62. Project staff at different levels of the system felt strongly that the single largest benefit they derived from the project was that they became more receptive to learning about and testing new ideas. They became more independent thinkers, taking a more active role in the design of their own projects and programs, instead of passively following orders from above.  3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 63. A third party hired by CPSM to conduct an independent evaluation of the project fielded an end-of-project survey in the 40 project counties across eight provinces. Interviews were conducted in 36,000 randomly selected households in townships and villages. As noted above, results showed a higher level of satisfaction with rural health services at the end of the project (above 85 percent), considerably better than the pre-project level (63.8 percent of patients were satisfied). Additionally, compared with 2008, the proportion of outpatients who considered outpatients' waiting time to be short increased by 18 percentage points, and the proportion of outpatients who considered the medical environment to be good increased by 34.1 percentage points. 4. Assessment of Risk to Development Outcome Rating: Negligible to Low 64. Overall, project development outcomes are expected to be sustained and expanded given that the current political and financial environment is favorable to implementation of further rural health reforms. While the degree of sustainability will vary by province and county, the positive outcomes from project areas are likely to continue, as the project contributed to a broader Government reform agenda for which there is strong ownership at the national, provincial and county levels. The Government is committed to rural development (as reflected in the 12th Five-Year plan) and will continue to further refine the NCMS and MA, which are the principal financing mechanisms for the rural health care system. New modalities for reimbursement under the NCMS, provider payments methods and approaches to reduce the financial burden on the poor were tried in the project areas; successful ones have been integrated into the NCMS financing model, while others, such as the use of Diagnostics Related Groupings (DRGs) in provider payment, continue to be developed. The evaluation and management systems established under the project to support the rural health reforms are robust. Legislation, policies and training materials developed under the project, as well as knowledge, and technical and 19 managerial skills obtained by project implementers have built an important foundation for both sustainability of outcomes and their further achievements. 65. It should be noted, however, that some provinces and counties have weak capacity. They would need continued support from the Government to make substantial progress toward achievement of more equitable access to quality health services, improved financial protection, and better management of public health services. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 66. Project design benefitted from extensive sector knowledge and experience accumulated from previous World Bank operations in China, as well as comprehensive analytical sector work. During project design and preparation, the World Bank team worked in close collaboration with the Government to conduct research and analyses of China’s rural health sector in order to identified major challenges and underlying causes. The project preparation team was comprised of strong national counterparts, prominent domestic and international experts, and lead specialists from within the World Bank with experience in implementation of health projects in China and across the world. 67. The World Bank, with support from DFID, introduced the concept of innovation to the project and encouraged implementers to think outside the box to address sector issues and challenges. A series of workshops and trainings was carried to encourage this shift in mindset that ultimately translated into a bottom-up approach to project design, whereby block grants are given to project counties to design their own activities under guidelines that promote local participation and foster ownership and sustainability of the project. Innovative features, which had not been previously used in World Bank health projects in China, were brought into the design, including: (a) use of a single disbursement method to give more flexibility to implementers; (b) no counterpart funding requirement to allow for speedy use of project funding; and (c) no repayment requirement from project counties, reducing the financial burden at the county level. The design was sufficiently flexible, allowing for adaptation to different local settings and for adjustments to be made during implementation.   68. The project design was complex, however, and challenging to implement. The project design covered comprehensive reforms in three major areas—health financing, service delivery and public health. The wide breadth of the project was technical grounded; the aim was to provide a large menu for implementers to select reform areas based on their local needs. Implementation risks were correctly assessed at project design and mitigation measures put in place, including utilization of international technical assistance, guidance and supervision at provincial and central level, and adequate funding for knowledge sharing, among others. However, at the request of the Government, the project was to be 20 implemented in a large number of counties (40) and provinces (8), with varying degrees of managerial capacity and financial and human resources availability. This imposed significant challenges to implementation, pushing the limits of the World Bank’s capacity to provide, within the available resource envelop, the technical and managerial support required for successful implementation.     (b) Quality of Supervision Rating: Moderately Satisfactory 69. The supervision missions done by the World Bank team were regular, diligent and supported the client in addressing implementation challenges. The project benefitted from a committed World Bank team with a close relationship with Government counterparts. However, TTL changes created gaps in knowledge and institutional memory. 16 This exacerbated the challenge posed by weak capacity to implement innovations in some counties and hampered implementation, particularly at the beginning of the project. Despite the slow start, the Bank strengthened supervision and developed many good practices, including setting up a learning platform for project counties, organizing biannual dissemination workshops and facilitating South-South knowledge sharing. Additionally, the World Bank fiduciary team’s oversight, support and timely training were considered by the clients to be very beneficial. The midterm review was timely and simplified some procedures that helped speed up implementation. 70. With financial support from DFID, the World Bank team provided strong technical support and knowledge transfer. However, after DFID funding was exhausted, the World Bank team did not have the resources to draw on as many experts to support supervision. The GOC had a strong and competent counterpart team for this project, with high expectations and demands for the World Bank technical team (including international consultants), which could not always be met within the allocated supervision budget. It should be noted that client expectations were high and the level of support from international experts that was expected is generally beyond the scope of supervision and normally requires an operation with a technical assistance component.    71. Given that a key feature of the project was to use a bottom-up approach that gave counties flexibility to design project activities, it was challenging to construct a Results Framework that would capture activities that were yet to be defined. Therefore, not all indicators selected were in line with the interventions that project counties chose to implement. This was the case, for instance, with the intermediate result indicator on the percentage of deliveries by cesarean section. This issue was discussed during the MTR, but ultimately no consensus was reached on alternatives indicators; only one indicator was updated during restructuring. The Results Framework continued to rely heavily on high-level outcome and output indicators that were not as directly linked to project activities, making it harder to monitor the project’s outputs, as mentioned above in Section 2.3 on M&E. 16 The project had three TTLs during implementation. Though this is not necessarily unusual for a six-year World Bank project, transitions can create challenges for smooth implementation. 21 72. Summarizing, the Moderately Satisfactory rating stems from: (a) weakness in implementation at the onset of the project; (b) insufficient resources to support a non- standard project; and (c) the missed opportunity to revise the Results Framework following the MTR. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 73. Given the Satisfactory rating for quality at entry and the Moderately Satisfactory rating for quality of supervision, the overall World Bank performance is rated as Moderately Satisfactory, according to the World Bank’s ICR rating guidelines. However, given the challenges of implementing a complex project and the strengthening of supervision in later stages of project implementation, as well as the successful completion of the project, a “Satisfactory” rating for Overall Bank Performance would be justified in this case, despite the shortcomings in supervision noted above. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 74. Overall, the Government remains highly committed and supportive of rural health reforms, and has demonstrated strong ownership of the project from beginning to end. The Government has made significant investments in the rural health care system, which created an enabling environment for project implementation. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 75. Center for Project Supervision and Management (Highly Satisfactory). CPSM has successfully led the project to completion, with funding fully utilized and key PDO indicators achieved. It has also provided strong leadership in both project preparation and implementation, drawing on its long experience managing World Bank projects. CPSM has developed and maintained institutional capacity and sector knowledge in coordinating and managing a complex and multilevel project in a dynamic national policy environment. 17 They have monitored implementation progress closely and proactively resolved issues, drawing on expertise from the World Bank and international experts when needed. The CPSM team successfully organized joint supervision missions with the World 17 At the national level, the National Health and Family Planning Commission (NHFPC) through CPSM was responsible for overall coordination of the Project and implementation of national level activities. The NHFPC established a national Leading Group (LG), chaired by a Minister of Health, and comprised of high- level representatives from the NHFPC, NDRC, MOF, and other agencies, who were responsible for high- level oversight of all aspects of the Project. CPSM acted as a secretariat to the LG and operated the national level Project Management Office jointly with the NHFPC’s Department of Finance and Planning. A National Expert Panel provided technical assistance to CPSM and lower level implementers. 22 Bank and DFID, as well as national teams comprised of provincial PMOs and national experts. They coordinated with the Center for Health Statistics and Information to conduct three surveys (at baseline, mid-term and end-of-project) and provided valuable information for project monitoring and evaluation. Under their leadership, the project has reviewed and collected innovative case studies and successfully disseminated the results in project and non-project areas. Project covenants were largely complied with and quickly rectified when there was a risk of noncompliance. There were no major fiduciary issues relating to financial reporting, auditing, and procurement. The Project Manager led the CPSM team from the beginning to the end of the project, providing a continuity that was a key factor in the strong performance of the CPSM and in the successful implementation. 76. Provincial Project Management Offices (Satisfactory). Coordination of project activities at the county level was the responsibility of the provincial government, primarily through the Provincial Bureau of Health (BOH). Each BOH established a Project Management Office (PMO) responsible for logistics of implementation, preparation of work plans, progress monitoring and reporting at the provincial level; the PMOs were supported by the Provincial Expert Panel and the Central level. All eight PMOs provided strong implementation support that made achievement of project results possible; though some provinces, namely Henan and Chongqing, excelled in this respect.     77. County Project Management Offices (Satisfactory). County level implementation was the responsibility of the Project Management Office at the county level, with guidance from the provincial PMOs. Most participating entities were fully committed to achieving project objectives and worked diligently, although capacity and performance varied greatly across provinces and counties. Financial management had a slow start at the county level because staff need to learn World Bank procedures, especially the payment of advances which was a new disbursement method introduced by the project. Intense training by the World Bank team as well as CPSM was instrumental in addressing this shortcoming. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 78. The Overall Borrower Performance is rated Satisfactory for reasons outlined above. 6. Lessons Learned 79. The project was able to capitalize the existence of a policy environment favorable to rural health reform and push the envelope, delving into difficult issues such as provider payment and incentives. The project covered comprehensive elements of the rural health reform and encouraged counties and provinces to experiment with innovative approaches to resolving local issues. Because the project design was (and still is) consistent with the national reform agenda, the project had the support of local governments, which made it possible for measures to be implemented that would have otherwise not gotten off the ground. Piloting reform measures at the provincial and county levels generated valuable knowledge and experience that can be applied to the design of future reforms. It was an effective way to get impact from the World Bank’s relatively small financial contribution to overall health financing in China. 23 80. The implementation of a complex reform project in such a large number of counties and provinces undermined the effectiveness of the project. This project was designed to pilot very difficult reform measures in three distinct areas—financing, service delivery and public health. Though the breadth of the project is justifiable for a technical standpoint, the design required intensive technical support, as well as close monitoring. Implementation in 40 countries across eight provinces scattered geographically pushed capacity of both the Government and World Bank teams to the limit. It was difficult for the World Bank team to remain fully engaged with implementation across all counties, contribute to the policy dialogue and provide adequate technical input during supervision, particularly given its limited supervision budget. DFID support for technical assistance was instrumental in mitigating some of these constrains for a portion of the project life. 81. Capacity building and strategic thinking has enhanced the project’s impact and its long-term sustainability. The project created an enabling environment in which implementers were encouraged to be open to new ideas and take risks in trying new approaches to health care financing and provision. It also encouraged local implementers to experiment and learn from the experience of designing activities and interventions to address their local issues; and facilitated intensive knowledge exchanges and training to give implementers the tools to design new approaches. Project staff at all levels felt that the longest lasting gain from the project would come from their ability to apply the skills and knowledge they learned to their future work. This one of the ultimate successes of the project. 82. The Results Framework should include lower level indicators that can be regularly monitored using routine data from the management information system. The project Results Framework design closely linked monitoring indicators to overall project objectives and included mostly indicators that measured high-level outcomes. These indicators were dependent on household survey data to be updated. During project implementation, a project management information system was created to monitor implementation progress, but it would also have been beneficial to formally modify the Results Framework. 83. By focusing on evaluation, documentation and dissemination of experiences, results and lessons learned, the project transferred knowledge and fostered a culture of evidence-based decision-making. The rollout of project pilots benefited from use of evidence generated throughout implementation. A learning network for project counties was set up to encourage learning, and knowledge generation and transfer. Dissemination workshops were organized through the network every half-year as part of project supervision. A vast array of knowledge and dissemination materials were generated, including data, reports and videos, which are now housed in a data repository managed by the central PMO, hence ensuring sustainability. Each province focused on collecting best practice case studies from participating counties. The lessons learned from the innovative approaches in health financing and services delivery have been disseminated not only among the 40 project counties, but also to non-project counties. In the last year of implementation, the project also exchanged experiences on health reform with countries in Central Asia and South East Asia. Using data collected from the project baseline, mid- 24 point and final survey, implementation results were evaluated and discussed at the project completion and dissemination workshop. In this way, the project was effective in building and transferring knowledge. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Cofinanciers 84. DFID was a cofinancier of the project, contributing US$6.14 million (4.18 million pounds) in grant funds plus 1.2 million pounds on direct technical assistance, and made a significant positive contribution to the project. DFID was able to fill a critical gap by providing incentives through its grant that made the Government more willing to use loan resources to finance reforms. Specifically, the grant financed: (a) analytical sector work on rural health through its other project; (b) technical assistance, including short-term consultants for project preparation and implementation; (c) operational support for the National Expert Panel, the Provincial Expert Panels and CPSM, which allowed them to provide technical support to project implementation; (d) World Bank supervision missions; and (e) a portion of the project’s annual plans. The DFID- World Bank collaboration was strategic and created a single, cohesive platform for dialogue and partnership with the GOC. Both the GOC and the World Bank agreed that DFID’s support to the project was valuable and a key factor in strengthening the project and its contribution to furthering implementation of rural health reforms in China. (c) Other partners and stakeholders N/A 25 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Estimate Actual/Latest Estimate (USD millions)1 (USD millions)2 Percentage Components WB WB of Appraisal DFID Total DFID3 Total Loan Loan 1. Health Reform 42.00 5.35 47.34 41.94 4.65 46.59 98.41% Innovations 2. Project Coordination, Policy Development and 4.98 4.65 9.63 7.90 4.06 11.96 124.21% Replication 3. Unallocated 2.90 0.00 2.90 0.00 0.00 0.00 Total Baseline Cost 49.88 10.00 59.88 49.84 8.71 58.55 97.79% Physical Contingencies Price Contingencies Total Project Costs 49.88 10.00 59.88 49.84 8.71 58.55 97.79% Front-end fee PPF Front-end fee IBRD 0.125 0.125 0.125 0.125 Total Financing Required 50.00 10.00 60.00 49.97 8.71 58.68 97.80% Notes: 1. Data from the Project Appraisal Document dated 29 May 2008 and Loan Agreement dated 8 October 2008. 2. Data from World Bank monthly disbursement summary as of 15 June 2015, GOC and DFID. 3. Using historical exchange rate at time of disbursement, GBP 1=1.6195 USD. Equivalent to GBP 5.38 million, which includes GBP 1.2 million in technical assistance managed directly by DFID. (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) UK: British Department for International 10.00 8.71 87.13 Development (DFID) International Bank for Reconstruction and 50.00 49.97 99.94 Development 26 Annex 2. Outputs by Component (from the Project) Component Planned Actual Outputs/ Outcomes at ICR Comments/ Outputs at Additional Appraisal Information Component 1-: Health Reform Innovations Supported via Block Grants. Better link the 1. Seamless linkages between the NCMS and MA systems NCMS and established in all project counties and developed policies to MA programs ensure the poor benefit from the two systems. 2. Beneficiaries of MA increased used of hospital care. 3. One-stop connection between county-level hospitals and township/village clinics established under the MA network covering 170 county-level hospitals and 1172 township/village clinics in project areas. 4. The NCMS under the project improved: reimbursement rates increased; coverage expanded and copayments reduced. 5. Established new ways to improve sustainability of the MA in some project areas. Refine 1. Twenty-nine counties in project areas (accounting for information 72.5%) strengthened the IT systems of the NCMS. flows and 2. Internet supervision at the county and township levels strengthen established in some counties and further extended to the village fund level and electronic cards were used for medical treatment and management care at the county, township and village levels such as “Health and control Service All-in-One Card”. capability Introduce 1. Two books based on the piloting international approaches The book “The international were published: “The NCMS Payment Reform"; "Take the NCMS Payment experience on Lead in Payment Reform Innovation, Realize the Double Reform” was Improving feasible Coverage of Payment Reform". recommended by Rural Health approaches to 2. Took a lead in exploring the reform mode of NCMS the Central Financing cost control payment and improved coverage in population and benefits. Government for and pilot 3. Explored options on provider payments including capitation national reference. alternative and diagnostic related groupings (DRG), lump-sum payment， provider capita-based hospitalization payment, etc. and promoted the payment payment reform and developed payment methods that are mechanisms. suitable to the local conditions. 4. Some of project innovative reform models were adopted as the local NCMS policies and are used by non-project counties. 5. Project experiences were widely disseminated through national level training courses. Strengthen 1. Intensive technical assistances, supervisions, close human monitoring and evaluations were introduced and implemented resources for better service delivery. capacity and 2. Training workshops and seminars held at the each level incentives focusing on NCMS and MA management. Training and other forms of capacity building. Improve 1. Explored mechanisms of establishing funds for catastrophic financial illness and a Case Study on this topic was published. protection 2. Some counties have lowered the copayment for inpatient while care for the poor and reduced or wavered their medical improve 27 access to expenses and provided short-term assistance for people who affordable are relatively poor, but are not covered by the MA. services for 3. Ratio of outpatient visit rate of the bottom 20% vs. the top currently 40% in the income distribution has showed an increasing trend, excluded or i.e., utilization of outpatient care increased among the poor. marginalized groups Strengthen 1. Under the project, counties improved the Network of service mechanisms information and the information was used to guide supervision. for 2. Established systems for community-based supervision to supervision report on NCMS disbursements and activities. and broader 3. Information management systems on NCMS and MA accountability established. to the community Effective Pilots on comprehensive reform of county hospitals were External evaluation control of carried out and experiences on improving service quality and report on project health care controlling cost were collected and disseminated. pilots was published costs and well received by Ministers of Health. Improved 1. Quality-Based Performance Management Mechanism The evaluation technical formulated and a case study on this topic published. indicator system of quality of care 2. Operational guidebook on performance management was the project counties developed and disseminated. gave a higher weight 3. The performance evaluation for both institutions and staff to performance on introduced quality measures. quality indicators; in 4. Satisfaction of patients to health services increased in all most counties it project areas. reached 30-40%. 5. Methods on strengthening health human resource widely explored and piloted. Some showed improvement for sustainability and replicability. 6. Basic equipment and civil works procured for village clinics Improving to improve hardware. Quality, Increased 1. Formulated service-oriented and quality-based long-term Many project Efficiency and internal performance management mechanism. counties explored Cost Control efficiency in 2. Efficiency in service delivery was significantly improved, service models such in Service service and the management skills of hospital managers were enhanced as integration Delivery delivery and the standardization of hospital management was promoted. construction, coordinated care, and outpatient clinical pathway, further clarified the responsibility of all levels of the medical service institutions. Increased 1. Awareness of disease prevention has improved in the project Fully strengthened focus on area due to health education and health promotion carried out the grass-root prevention under the project. construction of rural and health 2. Various approaches of health promotion were piloted based healthcare service promotion on the local needs and the construction of “New Rural system and Villages”. consolidated the 3. Innovative pilots on Healthy Village were widely foundation and implemented, and achieved significant results. improved the ability to provide healthcare services. 28 Evidence- 1. Established cross-component multi-department management A cross-sectorial based and platform and service mode of NCDs. mechanism to lead, multi- 2. Explored the working system of multi-departmental coordinate and sectorial interaction, social participation, investigation and recording, replicate the planning and internet management, tracking treatment and multi assistance experiences of coordination for patients suffering mental illness. public health for public provision was health service established. provision Purchasing of Various modes for providing public health services were The vice Minister of clinical public explored and promoted in the project areas including using Health called on health “government purchasing services; contracting out services; every province to services using performance-based payment approach and etc. learn from the project pilots. Financing and 1. Improved performance evaluation and management of The income of service essential public health, and explored the practicable mode of village doctors models - service provision. engaged in public including 2. The enthusiasm of the grassroots medical personnel engaged health service outreach and in public health services increased. accounted for more incentives for than 1/3 of the health annual total income workers – to in the project areas. improve access for Strengthening remote the Financing populations and and other Organization vulnerable of Core Public groups in the Health community Functions Community Improved the direct reporting IT systems for public health mobilization management. mechanisms for public health emergencies Management 1. Established the financing and management of NCMS to The control rate of of NCDs increase reimbursement for chronic diseases. hypertension according to 2. Management and control rates for hypertension and diabetes (63.8%) and control standardized increased. rate of diabetes protocols 3. Promoted better management for NCDs and increased health (50.2%) were staff’s awareness and skills for chronic disease management significantly higher and improved case finding. than the average level in rural areas (54.9% and 38.3% respectively). Improved The self-reported prevalence rate and daily medication rate of disease hypertension and diabetes significantly improved in project surveillance areas. at the county and village level Monitoring 1. Satisfaction rate of patients with mental illness’ family Patients with mental equity of greatly increased. illness registered access to and 2. The population’s knowledge on mental illness was generally and managed in satisfaction improved. project provinces 29 with public 3. The family members’ knowledge on mental health and increased greatly health psychological needs of patients with mental illness has and their services achieved at a higher level. management rate also increased. Component 2: Project Coordination, Policy Development and Lesson Learning. Focus on 1. Established the mechanism to summarize and disseminate About 10% of the learning - project innovation experiences and achieved the transformation experiences were through from experience to policy and from theory to practice. spread to the frequent 2. Developed a series of health policy documents based on municipal level, feedback, experiences from innovative pilots and developed guidelines about 20-25% policy and standards to help local governments to carry out the health experiences to the debates, and reform and implement reform measures. provincial level and special 3. Established mechanisms for sharing reform experiences another 15% studies among project counties and with non-project through mutual influenced the visits, exchange documents and reports. policy formulation, 4. Published at the national level 10 case studies and shared health system pilot experiences. development in 5. Strategically utilized international technical assistance to other areas outside focus on experiment designs and key technical problems of project provinces. resolving. Active use of 1. Project management information system was designed and monitoring developed, to collect project information systematically, which indicators -to provided timely and effective base for project evaluation and assess decision-making. progress and 2. Developed and utilized the Monitoring and Evaluation manage fund Operational Manual. release 3. The indicators for project monitoring and evaluation were identified and data for these indicators were collected Project throughout the project period. Coordination, 4. Triggers for each major pilot activity as preconditions for Policy project fund disbursement were identified and successfully Development implemented. and Lesson Facilitate 1. Cross-sectorial leadership group, a project management The third parties Learning efficient office, an expert panel was established as well as a mechanism carried out implementa- to coordinate and engage all departments support project evaluation on the tion and implementation. internal evaluation course 2. Established mechanisms to summarize and disseminate group for their work adjustment project experiences among project counties and with non- content, medical for the project project countries. services, medical 3. Cultivated a group of project managers and executives in resource security project areas who are dynamic and innovative and became key and policy implementers for health reform in rural China. implementation; in 4. The establishment of the third party supervision mechanism, some areas a third credit evaluation mechanism, supervision platform to ensure party evaluation provision of high quality basic health services in project areas. institution was 5. Advancement of project fund based on the agreed plan jointly formed by facilitated the implementation and avoided some delays in the professional staff implementation. and administrative staff. Flexibility 1. Implemented an innovative reward system and US$2.9 based on clear million were used to reward high-performed project counties. business rules 2. Developed and implemented the Operation Manual for - MOU, Management and Implementation of basic public health Operations services, Guidelines on Performance Management for grass- Manual, root rural health institutions, development of Healthy Villages. performance 30 rewards and exit strategies Integrated 1. Developed reform policies based on pilot experiences and Policies were issued approach jointly issued by multiple departments. by high level of multi- 2. Promoted the county public hospital reform using multi- departments to sectorial and sectorial approach. expand project cross- 3. Innovative activities were carried out by multi-sectorial experiences to non- component managers, policy makers, the staff and the experts in the project areas. with equity project areas. concerns integrated throughout 31 Annex 3. Economic and Financial Analysis 1. The ICR’s economic analysis builds upon the economic analysis at project preparation. It and assesses the economic rationale of the project and assumptions made at project appraisal. 2. The PAD stated a number of challenges of conducting economic analysis for this project. The first challenge was that many important project benefits, such as improving equity or reducing financial risk of falling ill, would be difficult to quantify in monetary terms. Second, there was scant evidence on which to base assumptions about the impact of project activities on desired outcomes. Third, it was difficult to establish a plausible counterfactual—i.e., what would happen in the absence of the project—given the fast changing environment in China. Finally, it was not possible to measure quantitatively the spillover effects of the pilot experiences, in terms of stimulating or informing rural health reform elsewhere, which was one of the project’s key qualitative outputs. Under these challenges, the approach the appraisal team used in the economic analysis was to illustrate the economic logic of the operation by linking project activities to expected results and ultimate economic benefits (Figure 1 below, from Annex 9 of the PAD). Figure 1. Key Project Activities  Enhancement of NCMS compensation arrangements (benefit package) and the MA Scheme  Introduction of new provider payment methods, such as case-based or capitation-based payment, along with other cost control measures  Development and implementation of improved systems and incentives for promoting health care quality – e.g. through the use of clinical guidelines and other quality standards  Development of institutional and purchasing arrangements for core public health services  Strengthening of management information systems and management capacity in areas of health financing, service delivery, and public health Expected Project Results  Improved equity in financing burden and use of health services  Improved financial risk protection  Improved technical quality of care  Improved patient satisfaction  Expansion in coverage of key preventive services  Increased internal efficiency in management of government health financing  Increased internal efficiency in service delivery Expected Economic Benefits of Project  Improved health outcomes and quality of life  Reduction in time lost from illness and improved productivity  Increased consumption due to reduction in precautionary savings  Resource savings (for households and government) due to improved resources allocation, rains in internal efficiency, and cost-control measures  Improved patient satisfaction  Reduction in risks and costs associated with disease outbreaks and other public health emergencies 32 3. The team used the links illustrated in figure 1 to quantify, for illustrative purposes, the net present value (NPV) of one expected project benefit—namely, the potential resource savings of improved resource allocation, cost controls, and internal efficiency. However, some of the assumptions made for conducting the cost-benefit analysis at project design did not hold and repeating the cost-benefit analysis done for the PAD is not meaningful. The following are the major reasons for not recalculating the net present value (NPV) or the project’s economic rate of return (ERR) in the ICR. 4. Lack of the counterfactual. The cost-benefit analysis in the PAD assumed “Annual growth in total health expenditures in project counties will gradually decline from 12.1 percent to 11.1 percent in project counties, without any worsening of health outcomes or risk protection relative to the counterfactual.” In 2009, shortly after the 2008 start of project implementation, the Government began nationwide rollout rural health sector reforms. The government made a strong commitment to transform the health system with the goal to establish universal basic healthcare by providing safe, effective, convenient, and low-cost health services by 2020. Government health expenditure was increased by 28 percent from 2008 to 2009 and more than doubled (2.6 times) from 2008 to 2013.18 These changes swamp any effect the project may have had on reducing the rate of growth of total health expenditures in project counties compared to non-project counties. It is not possible to establish a counterfactual to assess the impact of the project because both project and non-project areas received a massive influx of funds from the Government and many reform measures, such as an expansion of the NCMS coverage, were carried out in both project and non-project areas. 5. Project benefits were difficult to measure in monetary terms. Recognizing that project financing was a drop in the bucket, representing less than 1 percent of government health expenditures in the project period, the project aimed to support innovations that could contribute lessons on practical approaches for implementing evolving government health reform policies. The primary activity of the project was to pilot innovations, rather than supplement government expenditures. The project generated knowledge on a range of new approaches to increase financing of NCMS, tested various provider payment methods and piloted incentive systems to improve quality of health services, among other achievements. The experiences and lessons learned have been evaluated, documented and disseminated to project and non-project areas. These innovations will continue to have impact and generate benefits over the medium-term as China continues to implement health reforms. It would be difficult to capture, in monetary terms, the qualitative contribution of the project in stimulating ideas and informing ongoing reforms. 6. The economic analysis in this ICR focuses on the contribution made by the project. It compares expected results with actual results based on the economic logic outlined in the PAD, without attempting to attach a monetary value to these outcomes. The ICR team did not have access to the national survey data in order to conduct project vs. 18 WHO National Health Accounts Database. 33 non-project comparison analysis to assess attribution of the project. Only before and after project comparisons were conducted. The results indicated that the project contributed to project outcomes and economic benefits.19 1. Expected Project Results a. Improved equity in financing burden and use of health services 7. Utilization of health services increased in project areas. There was a clear rise, for instance, in use of inpatient care in all project provinces (table 1). This may be partly due to expanded coverage of NCMS, as well as increased reimbursement rate for inpatient care. The project supported pilots explored various methods to improve the NCMS and MA benefit packages, contributing to expansion of the programs.   Table 1. Annual hospital admission rate (%) Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai 2008 Admission Rate 6.6 5.2 6.5 5.3 7.5 9.6 6.0 5.7 6.2 2013 Admission rate 8.9 7.8 6.9 7.4 8.2 14.1 8.3 7.2 9.9 Sources: Final Survey Report for China Rural Health Development Project. 2014 8. Underutilization, including due to financial barriers, has declined. Survey data indicates that the share of patients who should have been hospitalized according to medical advice but were not has significantly decreased from 30.6 percent in 2008 to 13.5 percent in 2013. Among the patients who did not use hospital care, the share who reported financial constraints as the main reason for not obtaining care dropped from 76.9 to 48.8 percent during the same period.20 9. Equity improved, as envisaged. The poor typically use health services less than the rich even though they generally have greater health needs. That is, if there were no barriers to access, one would expect the ratio of service utilization between the poor and the rich to be greater than one to reflect greater need among the poor. Taking outpatient visits as an example, the ratio of utilization between the bottom and top of the income distribution increased in five provinces, remained unchanged in two provinces and decreased in one province. The number of the counties with a ratio below 1:1 decreased from 18 in 2008 to 6 in 2013. 19 The ICR team could not obtain an access to the data from the National Health Surveys in order to conduct project vs non-project area comparison. Main analysis was based on before and after the project. 20 Final Survey Report for China Rural Health Development Project. 2014 34 Table 2. Ratio of outpatient visit rate of 20 percent of the population with the lowest income to that of 40 percent of the population with the highest income Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai 2008 1.1 1.0 1.2 1.1 1.1 1.2 1.3 0.9 1.1 2013 1.2 1.2 1.5 1.3 1.5 1.2 0.8 0.9 1.1 Sources: Final Survey Report for China Rural Health Development Project. 2014. b. Improved financial risk protection 10. Coverage of health insurance for rural population has increased. Comparing data from the project’s baseline survey to the final survey, coverage of NCMS increased from 93.9 percent to 97.3 percent in all project provinces (table 3). Table 3. Coverage of NCMS for rural population (%) Heilongjia Chongqi Qin Total Shanxi ng Jiangsu Henan ng Shaanxi Gansu ghai 2008 93.9 93.6 90.5 92.7 92.4 89.2 96.2 96.4 97.6 2013 97.3 98.4 97.8 93.5 95.7 96.0 99.1 99.4 99.0  Sources: Final Survey Report for China Rural Health Development Project. 2014. 11. Out-of-pocket (OOP) payments for health as a share of gross domestic product (GDP) have declined significantly. It is well established that the population in countries that rely heavily on OOP payments to finance health are more exposed to financial risks due to health shocks. Health sector investments to improve insurance coverage in the project area and nationwide have greatly reduced the financial burden of health care on patients. As government spending on health in China increased, OOP spending as a share of total health expenditure shrank, as shown in figure 2. Figure 2. Health Care Financing in China   Source: China Statistical Year Book 2013. 35 12. Even though the cost of care, particularly inpatient services, increased during the project period and medical treatment became more expensive, partly due to quality improvements,21 the share covered by direct payments decreased significantly for farmers covered by the NCMS—from 74.7 to 48.4 percent (table 4). The reduction was more substantial in poorer provinces, such as Heilongjiang and Shanxi. Jiangsu still had the highest proportion of out-of-pocket expenses among all the provinces in 2013. Additionally, OOP payments for inpatient costs as a proportion of the surveyed population’s annual per capita income dropped markedly from 57.7 to 28.1 percent. This is the most significant benefit achieved during the project period. Table 4. Out-of-pocket inpatient costs Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai Average per visit (yuan) 2008 2651 3312 3225 4902 2376 2045 2495 1550 2472 2013 2975 3462 3607 4597 2636 2506 2915 1819 2978 Proportion (%) 2008 74.7 80.2 86.2 74.1 71.9 75.2 75.3 65.0 71.7 2013 48.4 48.6 49.9 55.3 47.6 53.4 50.7 36.2 44.3 Ratio of out-of-pocket expenses to incomes 2008 57.7 87.4 59.7 52.9 72.6 36.1 54.8 69.4 82.5 2013 28.1 39.2 30.6 26.2 31.7 21.1 28.2 24.5 35.7 Sources: Final Survey Report for China Rural Health Development Project. 2014 13. Even though cost of health care has increased, due to the expansion of health insurance coverage, households spend proportionally less of their income on health. In 2013, percentage of households who spent more than 20 percent of their income on health decreased 6 percentage points, down from 19.2 percent in 2008 (table 5). Table 5. Proportion of households that spent more 20 percent of income on health Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai 2008 19.2 19.0 19.3 10.5 23.9 14.9 19.1 27.5 22.0 2013 13.2 16.7 13.8 8.8 15.2 17.4 15.1 8.1 11.6 Sources: Final Survey Report for China Rural Health Development Project. 2014   14. More of the poor received medical assistance (MA). However, the coverage of the MA program remain small and uneven among project provinces. The number of poor people receiving MA under the project increased from 2,148 in 2008 to 3,053 in 2013, representing a little more than 3 percent of the population in the project areas. The poorer provinces, such as Gansu, had the highest rate of MA coverage (7.4 percent), while Jiangsu, a better-off province, had only 0.7 percent of the population covered by MA in 2013 (table 6). 21 As new medical technologies are made available, price may rise but the output has also changed, making prices comparison across time difficult unless a quality adjustment is made. 36 Table 6. Coverage of Medical Assistance Programs Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai No. of poor receiving MA 2008 2148 106 105 101 190 322 213 503 757 112 2013 3053 323 250 93 333 292 123 2 491 MA (%) 2008 3.5 2.0 1.7 1.4 2.2 4.9 2.9 4.9 7.7 2013 3.1 3.8 2.5 0.7 2.4 2.9 1.0 7.4 3.2 Sources: The Final Survey Report for China Rural Health Development Project, 2014 c. Expansion of coverage of key preventive services 15. To address the rising burden of noncommunicable diseases, the project focused on disease prevention and health education. With project support, prevention measures have increased for rural populations, particularly women and the elderly. In 2013, the gynecological examination rate for women aged 15-49 was 66.3 percent, an increase of 22.2 percentage points compared with 2008. The prenatal care rate for pregnant women was 98.9 percent, up 5-percentage point from 2008. The postpartum visit rate was 84.2 percent, an increase of 34.9 percentage points during the same period. The basic vaccination coverage of children aged 1-4 also increased. The hypertension management rate increased from 76.8 percent in 2008 to 91.9 percent in 2013. Physical examination for people aged 65 and over was 80.5 percent, a significant increase from 15 percent in 2008 (Table 7). Table 7. Physical examination for population aged 65+ (%) Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai 2008 15.4 30.5 3.9 32.5 8.8 16.7 7.5 2.2 3.0 2013 80.5 85.0 87.3 84.0 86.9 69.4 70.4 89.4 75.7 Sources: The Final Survey Report for China Rural Health Development Project, 2014 2. Expected Economic Benefits a. Improved health outcomes and quality of life 16. Data on evolution of health outcomes or quality of life in the project areas is limited. Available data is consist with the overall increasing trend in noncommunicable diseases in China and elsewhere. For instance, the project collected information on the prevalence rate of chronic diseases among the population aged 15 and over. Prevalence of chronic disease refers to respondents diagnosed by health professionals as having one or more chronic illness during the six months prior to the survey, including chronic infectious diseases (such as tuberculosis) or chronic noncommunicable diseases (such as coronary heart disease or hypertension). The results show an increase in the prevalence rate of chronic disease from 2008 to 2013, particularly among females. 37 Table 8. Prevalence rate of chronic diseases among female and male residents (%) Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai 2008 Male 21.3 17.6 16.9 25.8 29.9 23.7 17.7 20.2 17.5 Female 26.7 23.9 23.3 27.2 33.3 27.3 22.5 27.9 25.9 2013 Male 27.9 26.7 28.0 31.0 28.9 39.8 24.3 19.0 24.6 Female 33.4 32.9 33.3 33.4 32.9 46.8 29.4 27.3 30.7 Sources: The Final Survey Report for China Rural Health Development Project, 2014 17. Nonetheless, some improvement in risky behaviors that are known to contribute to the onset of NCDs were seen in project areas. The project also initiated implementation of the “Healthy Village Model,” which emphasizes health prevention, improving health education, promoting healthy diet and physical activities, and creating a clean environment. Based on global evidence, these activities should have a long-term impact on improving the health status of the Chinese population. The project survey in 2013 has already shown some improvement among population in project areas: smoking rate was 26.2 percent, down 0.8 of a percentage point from 2008; physical exercise 24.1 percent, up 9 percentage points and the use of sanitary latrines has increased to 70.5 percent, a 24-percentage point increase. b. Reduction in illness as cause of poverty 18. Fewer people had their ability to work negatively affected by illness or reported payment for medical treatment as a cause of poverty. These measures improved overall in the project areas and, for the most part, the poorer provinces of Qinghai, Gansu, and Shaanxi saw a reduction of these indicators. However, improvements were not shared across all provinces. In Chongqing and Henan, more people were not able to work due to illness and fell into poverty due to payment for medical treatment.   Table 9. Causes of poverty (%) Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai 2008 Ability to work affected by illness 31.8 35.3 35.8 36.4 34.4 28.4 32.3 28.7 30.5 Medical treatment as cause of poverty 12.6 14.4 20.5 10.5 14.0 6.6 9.1 11.2 18.5 Others 55.6 50.4 43.7 53.1 51.6 65.0 58.6 60.1 51.0 2013 Ability to work affected by illness 30.6 32.2 37.2 40.3 42.2 42.2 25.0 21.6 23.0 Medical treatment as cause of poverty 10.8 8.4 13.0 12.3 15.4 11.9 9.4 7.1 14.2 Others 58.6 59.4 49.8 47.4 42.4 45.9 65.6 71.3 62.8 Sources: The Final Survey Report for China Rural Health Development Project, 2014 38 c. Improved Patient satisfaction 19. Project investments in improving infrastructure of health facilities and training health workers to improve quality of services had the desired result. The before and after project surveys show that patient satisfaction in project areas has drastically increased. Compared with 2008, the proportion of outpatients who considered outpatients' waiting time to be short increased by 18 percentage points, and the proportion of outpatients who considered the medical environment to be good increased by 34.1 percentage points. Table 10. Proportion of patients who gave a high rating to medical institutions (%) Total Shanxi Heilongjiang Jiangsu Henan Chongqing Shaanxi Gansu Qinghai Short waiting time 2008 62.7 67.2 67.1 57.1 62.4 58.1 59.3 65.9 71.2 2013 80.7 83.5 94.1 74.4 80.7 72.3 83.2 85.4 77.2 Improved environment 2008 47.3 47.4 45.2 49.5 39.0 44.2 46.3 47.9 65.3 2013 81.4 80.8 91.1 84.8 81.8 64.0 81.7 92.1 87.1 Sources: The Final Survey Report for China Rural Health Development Project, 2014 20. In sum, comparing expected and actual project results and economic benefits, the project delivered on expectations. Project areas saw progress in equity of access to care and reduced economic burden of impatient care, as well as improved patient satisfaction with health services. Provinces, particularly poor ones, had a reduction in the population unable to work due to illness or driven into poverty because of payment for medical care. 39 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending Xiaowei Guo Senior Procurement Specialist EASR2 Xiaoping Li Senior Procurement Specialist AFTPW Haixia Li Sr Financial Management Specialist EASFM Magnus Lindelow Sector Leader LCSHD L. Richard Meyers Consultant EASHH Toomas Palu Sector Manager EASHH Margaret Png Lead Counsel LEGLE Sabrina Gail Terry Program Assistant EASHD Shiyong Wang Senior Health Specialist HDNHE Lansong Zhang Operations Analyst AES Shuo Zhang Senior Health Specialist EASHH Supervision/ICR Rosario Aristorenas Senior Program Assistant AFTEE Michael Joseph Borowitz Consultant EASHD Yi Geng Sr. Financial Management Specialist EASFM Xiaowei Guo Senior Procurement Specialist EASR2 Haixia Li Sr. Financial Management Specialist EASFM Vivian Lin Consultant MNSHH Yunguo Liu Consultant EASHD Qingyue Meng Consultant EASHH Tao Su Program Assistant EACCF Limei Sun Program Assistant EACCF Hope C. Phillips Volker Senior Operations Officer EASHH Shiyong Wang Senior Health Specialist HDNHE Haiyan Wang EASHD Lingzhi Xu Senior Operations Officer ECSH1 Zhuo Yu Finance Analyst CTRLN Fang Zhang Financial Management Specialist EASFM Shuo Zhang Senior Health Specialist EASHH Lansong Zhang Operations Analyst AES Meixiang Zhou Social Development Specialist EASCS 40 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) USD Thousands Stage of Project Cycle No. of staff weeks (including travel and consultant costs) Lending FY05 23.33 114.13 FY06 17.65 133.06 FY07 43.91 240.94 106.68 FY08 21.79 142.25 Total: 106.68 630.38 Supervision/ICR FY09 28.97 119.95 FY10 36.13 128.04 FY11 42.66 159.02 FY12 23.65 125.95 FY13 18.25 140.79 FY14 6.26 86.60 FY15 10.09 81.88 Total: 166.01 842.23 41 Annex 5. Beneficiary Survey Results Not applicable 42 Annex 6. Stakeholder Workshop Report and Results Not applicable 43 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 1. WB/DFID China Rural Health Project (RHP) was launched in October 2008 with a total investment of 50 million USD of WB loan and 5 million GBP DFID/UK grant, and the account was closed on 31 December, 2014 (the DFID grant closed on March 31, 2011).The project implementation period is 6 years (including one year of extension), 8 provinces/municipalities and 40 counties were selected as project provinces and counties. They are Shaanxi, Chongqing, Qinghai and Gansu in western part of China; Henan, Shanxi and Heilongjiang in the middle part and Jiangsu in the eastern part. 2. In line with the project log frame, the project has carried out comprehensive and systematic pilot reforms on rural health protection system, rural health service delivery system, delivery of basic public health service, and comprehensive reform of county-level hospitals, to set up cross-sectorial mechanism to lead, coordinate and disseminate the experience of rural health reform pilots. After 6 years of implementation, the project has gained expected outputs in all components and achieved its designed objectives. Meanwhile, it has practiced result-based project management model. The achievement of the project is briefed as follows: I. Realization of project result indicators 3. The Mid-term survey conducted in 2011 revealed that out of the 22 result indicators, 20 had achieved the expected target and only two not yet achieved. Since mid- term review, CPSM and relevant provinces tried measures such as focused supervision, intensified intervention, real-time monitoring and quarterly reporting to strengthen achievement of expected targets before the closure of project accounts. According to the final survey result and the M&E data between Oct. 2013 and 30 Sept. 2014, all 22 indicators have achieved their targets. II. A set of innovative experience and practice developed 1. The development of rural health protection system 4. Experience and practice on rural health protection system have been developed in term of financing, payment, linkage between NCMS and MFA, medical insurance information management, M&E and capacity building of administrative agencies. 5. The payment reform of the project is characterized by 1) diversified exploration to cover almost all practices at county level in China; 2) continuous improvement to accumulate practical experience; 3) parallel advancement of payment reform and other reform. Project experience and practice are of high value of reference and have offered examples for the country. 6. The experiences explored by the project on NCMS payment reform was recognized and promoted by the former responsible department of MOH. CPSM and Rural Health Management Department of former MOH jointly organized a workshop to promote the experience on NCMS payment reform in April, 2012. The workshop was attended by 44 officials of provincial health department and NCMS administrative offices and experts in 31 provinces, representative from NHFPC pilot counties of NCMS payment reform. 2. Rural health service delivery system 7. Targeted at improving the quality and efficiency of health service delivery and focused on quality control and performance management, the project innovative explorations have been carried out in performance management and compensation mechanism; comprehensive reform of county-level hospitals; supervision and management of health service quality; integration of township-village management and human resource development. 8. The performance management in project areas is characterized by more weight to quality and work load; combination of centralized evaluation and routine evaluation, external evaluation and internal evaluation, evaluation of institution and individual staff; linkage of evaluation results and rectification with government subsidy, NCMS payment and allocation; attention to both economic incentive and non-economic incentive. The concept of performance evaluation and management has rooted in project areas, the efficiency and performance of the health service system in project areas have been greatly improved through diversified measures. 9. In May 2013, CPSM and WB jointly organized a workshop to share project experience on health service performance management. The representatives of System Reform Department and Grass-roots Health Department of NHFPC confirmed the project exploration, expressing that the pilot work had explored ways and means for the performance management in rural grass-roots medical institutions, which were quite operable. 3. Provision of basic public health service 10. Targeted at public health service equity and focused on exploring means to provide equal public health service, pilot activities have been implemented on purchase of public health service, NCD management, management of mental health, mode of health promotion and health village development. 11. Many typical delivery modes such as government purchase, contractual management, performance based payment, township-village integration, contract plus service coupon, one-stop service card were developed. At the same time, the project explored a third-party evaluation mechanism and set up a mechanism to monitor and evaluate the delivery of “service package”. 12. Health village development: All project provinces and counties through a series of innovative practices such as environment improvement, election of health households, various kind of propaganda on health, health promotion targeted at focal groups and key diseases. A group of health villages with typical features have been developed and are replicable. 45 13. A high-level workshop to share experience on basic public health service equity under this project was held in December 2010 in Zhengzhou of Henan. Vice Minister Chen Xiaohong of MOH confirmed the innovative service delivery model of basic public health characterized by “government leadership, management by relevant department, township- village co-action, contractual control, multi supervision and performance based payment”. In October 2013, CPSM and WB jointly organized a workshop to share experience on Health village. The representatives of Disease Control and Prevention Bureau of NHFPC confirmed the explorations. 4. The comprehensive reform of county public hospitals focusing on payment system reform 14. The project explored the comprehensive reform of county public hospitals as a breakthrough to the public hospital reform. A pilot model has been gradually established, in which the patients are the center, service is regulated by clinical path, expense is controlled through comprehensive payment system, the improved supervisory system is used as a guarantee, incentive mechanism as impetus and the information platform as a support. 15. In June 2012, an external review on the pilot reform in Henan was conducted. The Health Reform Office of MOH organized a special survey and research on Yiyang County of Henan about the comprehensive reform of county public hospitals. The survey and research report was commented by five Ministers of MOH who requested for wide dissemination and replication of Yiyang experience. 16. In October 2012, CPSM and Health Reform Office of MOH organized an experience sharing workshop on the comprehensive reform of county public hospitals. Officials from the health reform office of provincial health department and county health bureau from 18 provinces piloting the comprehensive reform of county public hospitals selected by MOH attended the workshop. 5. Project experience widely disseminated 17. China National Radio, Health Daily, Health channel and China Health Pictorial reported the innovative experience and practices of NCMS payment reform, basic public health service delivery, coordinated care and health village development. 6. Project experience caught the attention of the international organizations and neighboring countries (1) CPSM worked with WB to arrange a delegation of 18 officials and experts from WB health sectors in East Asia and Pacific region, Latin America, Middle East and Africa, South Asia to visit Yiyang County of Henan on the pilot comprehensive reform of county public hospitals in November 2012. (2) Project experience shared with countries of Mekong River basin 46 18. In April 2013, CPSM held a high-level seminar jointly sponsored by WB and DFID to share experience on China health reform and universal coverage for delegates from Laos, Cambodia, Myanmar, Thailand and Vietnam. The delegation of 53 participants visited Henan to learn Project experiences. (3) Experience sharing and cooperation carried out with two countries of Central Asia (i) In September 2014, CPSM and representatives from project counties went to Tajikistan to attend a symposium on primary healthcare service delivery jointly held by WB and Health and Social Security Ministry of Tajikistan and shared the RHP experience. (ii) In November 2014, a delegation from Tajikistan and Uzbekistan attended the RHP summary and experience sharing workshop and visited project areas in Henan. (iii) In addition, at the invitation of Laos’s government and Ministry of Health of UZ, Representatives from project provinces and counties visit Laos in 2013. III. Rural health service equity received attention and improvement 19. The project carried out a series of innovative practices centering on improving the social equity. The equity in health financing and the health service for disadvantaged groups(including ethnic minorities) has been improved through reforming NCMS payment system, making the reimbursement easy for the poor, linking NCMS and MFA, and lowering disease-born economic burdens of the poor. All project counties have achieved the seamless linkage between the two systems and worked out corresponding policies to ensure the poor to benefit from the two systems as much as possible. RHP project also explored ways to establish fund for catastrophic diseases, thus to reduce the financial burden of health-seeking for the poor. 20. Supported by the project, gender equity has received wide attention. The project is also dedicated to gradually improving rural health service system, as a result, the service capacity at township and village level has strengthened, service quality improved and the accessibility of health service enhanced. The development of rural health team has produced good effects, the number, structure and quality of health staff increased fairly. To increase the need of health service among women, the project has carried out effective pilots, and the percentage of women receiving service from qualified women doctors in a majority of project provinces has evidently increased. 47 IV.A set of innovative experience and practice on project management achieved 1. Result-based project management and implementation (1) Result-oriented project management 21. The project adopted the result-oriented management, used the achieved project objectives as the basis to measure project effects, arranged project activities and fund according to the results achieved. The prepayment of project fund is also dependent on the realization of periodic objectives. (2) Innovative fund arrangement (i) Project counties are not liable to repay the loan 22. The project arranged for 80 percent of loan fund borrowed and repaid by central fiscal and the remaining 20 percent repaid by provincial fiscal, and the amount of repayment differed according to provincial financial resources. Project counties are not liable to repay the loan or provide counterpart fund. This arrangement has made project counties free from financial constraints and go all out to explore feasible and innovative practices.   (ii) Prepayment of fund 23. The project for the first time adopted prepayment mechanism and abolished payment categories, and used annual activity plan of project counties as basis and condition for fund use and disbursement, thus improving the flexibility and promptness of fund use. (3) Incentive and exit mechanism tried out 24. The project innovatively adopted incentive and exit mechanisms and rewarded twice the project counties with significant progress and excellent effects. 22 project counties (28 county times) were rewarded and their enthusiasm to innovate was further stimulated. At the same time, five project lagging counties received targeted support and were asked to carry out rectification. As a result, all 40 counties made great progress or even a breakthrough with their performance. The incentive mechanism injected vitality and impetus into the project implementation. V. Experience acquired from project implementation 1. A forward-looking project design is a precondition and guarantee for project success (1) The contents of the project design highly conform to objectives and directions of the health reform in China. 25. The project was designed through close collaboration between WB, Chinese government and experts home and abroad. During the two years of project design, Chinese and foreign experts conducted profound research and analysis of Chinese rural health and 48 demonstrated main problems and their reasons in the development of rural health. On this basis, a targeted intervention plan was formulated. Although the design was earlier than the disclosure of opinions on new round of national health reform, the project framework, main contents highly conformed to the objectives and direction of the Chinese health reform, thus allowing the project serve as pilot and catalyst in the process of Chinese health reform. (2) Integration of international experience and national experience to avoid straight copy 26. While introducing international concepts and methods, the project considered different situation in project areas when integrating international practices with domestic practical situation. All project provinces and counties took the initiative designing their own innovative pilot activities based on their situation, sought for a bottom-up route for the pilot reform, thus making the pilots more practical, operable and replicable. 2. A clearly defined mechanism of project management, organization and operation is the basis for the success of project (1) Project management office 27. The project involves central, provincial and county levels, thus a mature, stable and experienced project management office is needed to coordinate relevant agencies to carry out pilot activities. The project management offices play an important role for the project implementation. (3) Expert team 28. As a worldwide challenge, health reform keenly requires experts at all levels to provide technical assistance and guidance. The experts especially central experts need to understand the dynamic health development home and abroad, know well relevant health policies, have strong professional capacity and rich theoretical and empirical experience. At the same time, international experts are needed to provide the project with international experience and best practices. (3) Project coordination mechanism 29. The rural health reform deals with many departments, systems and levels. The project set up a mechanism for smooth communication and coordination. This is normally achieved through regular meeting, work meeting and brief report to inform main project achievements, discuss main problems and come up with solutions and reach a consensus on strategic and directive issues. 3. Innovative management safeguards the project implementation (1) The project adopted the result-oriented management, fund arrangement (Project counties are not liable to repay the loan, the project for the first time adopted prepayment mechanism and abolished payment categories). The project innovatively adopted incentive 49 and exit mechanism and rewarded twice the project counties with significant progress and excellent effects. (2) Timely summary of project experience and practices 30. The project tried to address challenges and difficulties in the implementation; summarized and produced operational guiding books on several important innovative activities based on pilot and exploratory practices to regulate pilot exploration in project counties and outside project areas. Meanwhile, the Project Case Studies of Innovative exploration and practices was written and as a publication issued domestically. (3) Project progress monitored to allow timely adjustment 31. To allow real-time and dynamic monitoring of the overall project progress, the project designed M&E indicators of various kinds including the 22 result indicators to track the realization of project objectives; 23 routine M&E indicators to understand the implementation process (same as indicators of the national health reform in order to allow dynamic comparison); two-year milestone indicators to understand the progress of key implementation nodes, and objectives of annual activity plan. The complete monitoring system has provided important data support for timely detection of problems, formulation of counter measures and timely adjustment. VI. Problems Exist 1. Influence from adjustment of the management system of some project counties on project innovation 32. The project innovation was carried out in the context of national and provincial health reform, some pilots might be so restricted by existing policies or regulations that could not continue or achieve expected effects. For instance when Chongqing was exploring payment reform in general hospitals, there were both life insurance department and social insurance department, due to departmental adjustment followed, health reform policies were all planned at municipal level, this situation made project counties difficult to advance the reform of county public hospitals as scheduled. 2. The clinical pathway coverage of diseases need to be extended 33. The clinical-path-based comprehensive payment system reform is still under exploration, Even though all project counties have carried out payment reform related with clinical path, progress varies a lot. Some areas have only applied clinical path to a few number of simple diseases without setting up supervisory mechanism, there is still room for advancement. 3. Room for improvement in performance management 34. Health staffs in project areas still have different views on the existing performance evaluation system. Village doctors in some areas received the performance management and equitable public health favorably, they could have some stable income from providing 50 public health service, the more people they looked after, the more they would earn. But some other doctors believed that the merit pay led to times of “big rice bowl” and people did not have much enthusiasm. Due to the implementation of performance management and basic drug list, township health centers in some areas did not have any other business except general outpatient service, doctors had less enthusiasm. 4. The prepayment mechanism has not come into its full play. Implementing agencies still lack of working fund and burden to pay for the project activities is still big. 35. Since the prepayment was made to county bureau of finance and did not require counterpart fund, project offices still followed conventional reimbursement system and made advance payment and then got reimbursed from the county bureau of finance. The project implementing agencies had no additional fund as working fund and faced a pressure of advance payment; this led to imbalanced fund use and affected the disbursement of the project. 5. Exchange rate affected the use of project fund 36. Due to the change of exchange rate when the project was appraised and when it was implemented, central project office and provincial project offices all faced with risks brought about by the changed exchange rate. For instance the amount of DFID grant for Gansu was 465,000 GBP and was equivalent to 6.51 million RMB Yuan at the exchange rate when the project was appraised (1 GBP=14 RMB Yuan) , it turned out to be only 4.6733 million RMB Yuan in actual use with a loss of 1.8367 million RMB Yuan; the amount of WB loan was 5.15 million USD when the project was appraised and equivalent to 36.05 million Yuan at the exchange rate of 1 USD to 7 RMB Yuan, but by the time of project completion, the actual amount used was 33.37 million RMB Yuan with a loss of 2.68 million RMB Yuan. VII. Recommendations 1. On future loan project design 37. The RHP project design and implementation have proved that a design built on survey and research and accorded with national macro health development strategy does not only guarantee the realization of project objectives but also allow such a project to push forward the Chinese health reform, thus maximizing the added value of a WB loan project. The system and mechanism innovation, advance concept and technology introduced and influence on future health reform bear much more profound and sustainable significance compared with the scale of capital input. The design and implementation of future international health project may use the mode and path of this project for reference. 2. On project execution 38. Top-level design, guidance and central level coordination and planning are vitally important for innovative projects aimed at pushing forward health reform and providing evidence for national policy-making. The implementation of an innovative project requires 51 inter-departmental coordination and communication, and also requires implementing agencies to timely track the changes and progress with relevant national policies so as to make relevant adjustment and ensure project effects. All these rely on guidance from relevant ministries and departments, coordination of central project management agencies including the coordination and communication with international financing agencies, project provinces and counties, and coordinate team of experts at all levels. 3. On the design of evaluation indicators 39. The design of project result indicators should accord with the project interventions, and the two should be directly related. Otherwise, it is neither objective nor fair to have the realization of indicators evaluate project impact where there have been not corresponding interventions carried out. Take cesarean section rate as an example, and the indicator of gynecological check-up of last year among women aged 15-49 (married and of childbearing age) did not closely correlate with project interventions. 52 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders 53 Annex 9. List of Supporting Documents Center for Project Supervision and Management, National Health and Family Planning Commission (2014), “Project Completion Report”. Beijing, People’s Republic of China Center for Project Supervision and Management, National Health and Family Planning Commission (2014), “Innovative Exploration and Practices – Collection of Project Case Studies”. Beijing, People’s Republic of China School of Public Health (2014), “Final Evaluation Report”, Fudan University November 2014. Center for Health Statistics and Information (2014), “Final Survey Report for China Rural Health Development Project”, National Health and Family Planning Commission, PRC, October 2014. Chongqing Project Management Office (2014), “Final Evaluation Report”, Chongqing Municipality, PRC, August 2014. Gansu Project Management Office (2014), “Final Evaluation Report”, Gansu Province, PRC, August 2014. Heilongjiang Project Management Office (2014), “Final Evaluation Report”, Heilongjiang Province, PRC, August 2014. Henan Project Management Office (2014), “Final Evaluation Report”, Henan Province, PRC, August 2014. Jiangsu Project Management Office (2014), “Final Evaluation Report”, Jiangsu Province, PRC, August 2014. Qinghai Project Management Office (2014), “Final Evaluation Report”, Qinghai Province, PRC, August 2014. Shaanxi Project Management Office (2014), “Final Evaluation Report”, Shaanxi Province, PRC, August 2014. Shanxi Project Management Office (2014), “Final Evaluation Report”, Shanxi Province, PRC, August 2014. 54 To Chita 80°E 90°E 100°E 110°E 120°E 130°E He RUSSIAN FEDERATION ilo ng Jia ng To Chita KAZAKHSTAN CHINA HEILONGJIANG Qiqihar Harbin MO NGO LIA To Bishkek 40° To N Changchun Ulaanbaatar KYRGYZ REP. Urumqi J I L I N To To Bishkek TAJ Dushanbe IK IST Shenyang A Kashi LI AONI LIAON INNG G 40°N L Anshan D.P.R. N GO Sea of X I N J I A N G M ON Hohhot OF NE I Baotou BEIJING Japan AFGHANISTAN Hua ng He BEIJING Bo KOREA Tianjin Hai Dalian To Mardan H E B E I TIANJIN Yinchuan Shijiazhuang REP. OF Taiyuan Jinan KOREA IA This map was produced by NINGX the Map Design Unit of The Xining SHANXI Qingdao PAKISTAN World Bank. The boundaries, colors, denominations and Lanzhou SHANDONG any other information shown QINGHAI Yellow on this map do not imply, on To Zhengzhou Sea JAPAN 30° X I Z A N G ng Luoyang N the part of The World Bank t G A N SU Xi'an JIANGSU ia n Group, any judgment on the Huang He HENAN legal status of any territory, SHAANXI or any endorsement or acceptance of such boundaries. Hefei Nanjing Shanghai 30°N Nu SHANGHAI AN HU I 0 300 Kilometers HUBEI Jinsha SICHUAN Chang Jiang Hangzhou East G Lhasa Wuhan IN La Chengdu n Q INDIA ca China NG ZHEJIANG NEPAL ng 0 100 200 300 Miles O CH Nanchang 80°E To Chongqing Sea Nepalganj Changsha To BHUTAN HU N AN Munger J IAN GXI C HI N A UIZ HOU G UIZHO U Guiyang F U J I A N Fuzhou CITIES AND TOWNS Kunming BANGLADESH TAIWA TAIWAN PROVINCE CAPITALS GUANGDONG YUNNAN G U A N G X I NATIONAL CAPITAL Xun Guangzhou To MYANMAR RIVERS Mandalay Nanning HONG KONG SAR, CHINA MACAO SAR, CHINA 20°N MAIN ROADS 20°N MYANMAR VIETNAM RAILROADS Haikou IBRD 33387R LAO AUGUST 2012 PROVINCE BOUNDARIES P.D.R. HAINAN THAILAND To Da Nang PHILIPPINES INTERNATIONAL BOUNDARIES 90°E 100°E 110°E 120°E