Document of The World Bank Report No.: 88078 PROJECT PERFORMANCE ASSESSMENT REPORT PEOPLE’S REPUBLIC OF BANGLADESH HEALTH, NUTRITION, AND POPULATION SECTOR PROGRAM ( IDA-40520 MULT-56510) June 24, 2014 IEG Public Sector Evaluation Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = Bangladesh Taka (Tk) 2003 US$1.00 Tk58.15 2004 US$1.00 Tk59.51 2005 US$1.00 Tk64.32 2006 US$1.00 Tk68.93 2007 US$1.00 Tk68.87 2008 US$1.00 Tk68.59 2009 US$1.00 Tk69.03 2010 US$1.00 Tk69.64 2011 US$1.00 Tk74.15 2012 US$1.00 Tk80.30 Abbreviations and Acronyms HNP Health, Nutrition, and Population HNPSP Health, Nutrition, and Population Sector Program ICR Implementation Completion and Results report IEG Independent Evaluation Group IEGPS IEG Public Sector Evaluation MDG Millennium Development Goals NGO Non-Governmental Organization PAD Project Appraisal Document PPAR Project Performance Assessment Report SWAP Sector Wide Approach TB Tuberculosis Fiscal Year Government: July 1 to June 30 Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Public Sector Evaluation : Mr. Emmanuel Jimenez Manager, IEG Public Sector Evaluation : Mr. Mark Sundberg Task Manager : Mr. Erik Bloom iii Contents Principal Ratings ............................................................................................................... vii Key Staff Responsible....................................................................................................... vii Preface................................................................................................................................ ix Summary ............................................................................................................................ xi 1. Background and Context................................................................................................. 1 Socio-Economic Context ................................................................................................ 1 Economic growth ........................................................................................................ 1 Poverty ........................................................................................................................ 2 Demographics ............................................................................................................. 2 Health in Bangladesh ...................................................................................................... 3 Health Status in Bangladesh ....................................................................................... 3 The Health System and Health Financing .................................................................. 4 The World Bank and the Health Sector ...................................................................... 7 2. Objectives, Design, and their Relevance ........................................................................ 9 The Program’s Objectives............................................................................................... 9 Program Design ............................................................................................................ 10 Relevance of the Objectives.......................................................................................... 12 Relevance of Design ..................................................................................................... 13 The program architecture and the design of the SWAP ........................................... 13 Technical elements and the causal chain .................................................................. 14 3. Implementation ............................................................................................................. 17 Background and Preparation ......................................................................................... 17 4. Achievement of the Objectives ..................................................................................... 21 Objective 1— Reducing infant, under-five, and maternal mortality and the proportion of malnourished children, while eliminating gender disparity and strengthening access to reproductive health services...................................................................................... 22 Reduce Infant and child mortality............................................................................. 22 Ensuring increased access to reproductive health services ....................................... 24 Contribute to the Reduction of Maternal Mortality .................................................. 25 Reduce the Proportion of Malnourished Children .................................................... 27 Has Gender Disparity Decreased under the Program? ............................................. 29 This report was prepared by Erik Bloom and Md. Rafi Hossain who assessed the project in February 2013. The report was peer reviewed by Anthony Drexler and panel reviewed by Denise Vallancourt. Viktoriya Yevsyeyeva provided administrative support. iv Objective 2—Lowering the fertility rate with a view towards achieving replacement level fertility by 2010 .................................................................................................... 29 Objective 3—Reducing the burden of priority diseases ............................................... 32 Tuberculosis .............................................................................................................. 32 Malaria ...................................................................................................................... 33 HIV/AIDS ................................................................................................................. 34 Objective 4— Initiate a System to Control Newer Health Threats and Protect Health Risks by Improving Emergency Services ..................................................................... 34 Pandemic influenza ................................................................................................... 35 Objective 5— Improving the prevention and control of non-communicable diseases . 35 Non-communicable diseases ..................................................................................... 35 5. Efficiency ...................................................................................................................... 36 External Efficiency ....................................................................................................... 36 Internal Efficiency ........................................................................................................ 37 6. Ratings .......................................................................................................................... 38 Outcome ........................................................................................................................ 38 Risk to Development Outcome ..................................................................................... 38 Bank Performance ......................................................................................................... 38 Quality at Entry ......................................................................................................... 39 Implementation arrangements ................................................................................... 40 Fiduciary arrangements ............................................................................................. 40 Technical preparation and lessons learned ............................................................... 42 Quality of Supervision .............................................................................................. 42 Supervision of the operation ..................................................................................... 43 Coordination ............................................................................................................. 44 Borrower Performance .................................................................................................. 44 Government performance ......................................................................................... 44 Implementing Agency Performance ......................................................................... 45 Monitoring and Evaluation ........................................................................................... 45 Measuring Results on the Ground: The Design of Monitoring and Evaluation ....... 46 Implementation of the Monitoring and Evaluation Framework ............................... 47 Utilization of Monitoring and Evaluation ................................................................. 47 7. Conclusions and Lessons .............................................................................................. 48 References ......................................................................................................................... 51 Annex A. Basic Data Sheet ............................................................................................... 57 Annex B. World Bank Investment in the Health Sector ................................................... 60 v Annex C. Key Performance Indicators for the Health, Nutrition, and Population Sector Program ............................................................................................................................. 61 Annex D. List of Persons Met........................................................................................... 63 Annex E. Borrower Comments ......................................................................................... 64 Boxes Box 1.1. The Government’s Strategic Investment Plan ...................................................... 5 Box 1.2. Sector-Wide Approaches...................................................................................... 8 Box 4.1. The Results of Other Evaluations of the Program.............................................. 21 Tables Table 1.1. Key Economic Data ........................................................................................... 1 Table 1.2. Population and Demographics ........................................................................... 2 Table 2.1. Program’s Design as a SWAP ......................................................................... 14 Table 2.2. Relevance of the Program to Health Sector Challenges .................................. 15 Table 6.1. Achievement of Objectives .............................................................................. 38 Figures Figure 1.1. Historic Life Expectancy in Bangladesh, 1970 to 2010 ................................... 3 Figure 1.2. Share of Burden of Disease in South Asia, 1990-2010 .................................... 4 Figure 1.3. Share of Health Spending, as a percentage of GDP ......................................... 7 Figure 4.1. Neo-natal, Infant, and Child Mortality Rate, per 1,000 live births ................ 23 Figure 4.2. Maternal Mortality Rate, per 100,000 Live Births ......................................... 25 Figure 4.3. Height for Age (stunting) of Children 60 Months and Younger .................... 27 Figure 4.4. The Number of Children Ever Born ............................................................... 30 Figure 4.5. Prevalence of Contraceptive Use.................................................................... 31 vii Principal Ratings ICR* ICR Review* PPAR Outcome Satisfactory Moderately Satisfactory Moderately Satisfactory Risk to Development Moderate Moderate Moderate Outcome Bank Performance Satisfactory Moderately Satisfactory Moderately Satisfactory Borrower Moderately Satisfactory Moderately Satisfactory Performance Unsatisfactory * The Implementation Completion Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEGWB product that seeks to independently verify the findings of the ICR. Key Staff Responsible Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Kees Kostermans Anabela Abreu Christine I. Wallich Completion Bushra B. Alam Julie McLaughlin Ellen A. Goldstein viii IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. ix Preface This Project Performance Assessment Report (PPAR) assesses the Health, Nutrition, and Population Sector Program in Bangladesh, commonly known as HNPSP. This program was financed by several development partners, with the World Bank supervising most of the financial contributions. The World Bank approved its contribution on April 28, 2005 and the program closed on December 31, 2011. The World Bank’s contribution was US$145 million. The Program was the principal instrument for development partners to support the health sector in Bangladesh and was the second Sector Wide Approach (SWAP) for the sector. This PPAR was done in conjunction with an evaluation of the Second Primary Education Development Program, which is a SWAP operating in the education sector. This report was prepared by Erik Bloom, Senior Economist, IEG and Md. Rafi Hossain, Consultant. The findings are based on a field visit from February 20 to March 17, 2013, which was done concurrently with an evaluation mission of the Second Primary Education Development Program. The mission visited Dhaka and the immediate environs and was carried out during a period of hartals (political protests), which limited the possibility of field visits during the mission. From April 16 to 19, 2013, Md. Rafi Hossain visited several health facilities in the Chittagong Division. As much as possible, the PPAR cites publically available documents, and when appropriate, it refers to interviews and internal documents. Following standard IEG procedures, copy of the draft PPAR was sent to relevant Government officials and agencies for their review and feedback. Their comments are presented in Annex E. xi Summary Background This Project Performance Assessment Report reviews the impact of the World Bank’s Health, Nutrition, and Population Sector Program in Bangladesh, commonly known as HNPSP. The World Bank Executive Board approved the program on April 28, 2005 and it became effective on June 14, 2005. It closed on December 31, 2011, one year after the original closing date. The project was designed as a follow-on project to the Bangladesh Health and Population Project (1998 to 2005) and the National Nutrition Project (2000 to 2006) and the HIV/AIDS Prevention Project (2000 to 2007). The Program was designed as a Sector Wide Approach, with the participation of the government of Bangladesh as well as a number of development partners. Bangladesh saw a significant improvement of its health status in the last decade. From 1970 to 2010, life expectancy increased by around14 years. From 1997 to 2007, spending on health from all sources increased from around 2.7 percent of the GDP to 3.4 percent, implying an increase in per capita spending from US$9.20 to US$16.20. In 2007, 65 percent of health expenditures came from households and private firms. The public health sector is highly centralized, with most public facilities under the direct authority of the Ministry of Health and Family Welfare. In 2007, the Ministry accounted for fully 97 percent of all public expenditure on health. The private sector plays a major and largely unregulated role in the health sector. According to official statistics there are about 2,600 private hospitals and clinics as well as 4,500 diagnostic laboratories. A large number of private sector health providers are “moonlighting” from public sector positions. Bangladesh has an influential NGO community, considered to be the largest and most dynamic in the world. These NGOs provide a wide variety of services, ranging from the provision of community health workers to support for hospitals. Objectives and Design The Program was at the center of the government’s strategy for health, population, and nutrition and remains highly relevant to the national development strategy. The program and its successor were identified in the World Bank’s country strategy. The Program had at least five different sets of development objectives in various official World Bank and government documents. This evaluation uses the objectives in the legal agreement, which closely corresponds to the “headline objectives” contained in the government’s Health, Nutrition, and Population Strategic Investment Plan. This indicates that the Program intends to: (i) reduce infant, under-five, and maternal mortality and the proportion of malnourished children; (ii) eliminate the gender disparity in child malnutrition and mortality; (iii) ensure increased access to reproductive health services; (iv) lower total fertility with a view towards achieving replacement level fertility by 2010; (v) reduce the burden of tuberculosis, HIV/AIDS, malaria, and other priority diseases; (vi) initiate a system to control newer health threats and protect xii health risk by improving emergency services; and (vii) improve the prevention and control of non-communicable diseases. Implementation, Impacts, and Risks The Program faced many implementation challenges during its first years. Many key positions were either left vacant or had a high turnover. Development partners were initially quite concerned with financial management and procurement. The midterm review noted that while the Program was supporting service delivery, it was having limited impact on the reform agenda. The Program faced a number of challenges due to the complicated nature of the administrative structure, both in the health sector and in other areas of the government. The implementation did improve with time and many of the initial administrative problems were resolved. Overall the Program outcome is rated Moderately Satisfactory. The Program supported a large number of interventions in the health, nutrition, and population sector. Given its objective, it aimed to strengthen traditional health, nutrition, and population challenges as well as supporting emerging health issues such as emergency care, preparation for pandemics, and non-communicable diseases. It also had activities to support the Ministry’s stewardship of the sector. Infant and child mortality as well as maternal mortality declined during the Program period and a significant part of this decline can be attributed to the Program’s activities. The child mortality rate decreased 23 percent, from 65 per 1,000 births in 2004 to 53 per 1,000 births in 2011. Access to reproductive health services increased due to increased training and the introduction of maternal vouchers. The maternal mortality rate declined from 322 deaths per 100,000 births in 2001 to 194 per 100,000 births in 2010. While many factors were behind this increase in health status, including increases in education, income, reduced fertility, and women’s growing participation in the workforce, the Program contributed to this reduction. The Program appears not to have had any impact on child nutrition, which remains a significant problem. The Program supported the continued reduction in fertility, which contributed to Bangladesh reaching the replacement fertility rate. During the Program period, the fertility rate decreased from 3.0 births per woman in 2004 to 2.3 births in 2011, essentially reaching replacement fertility. While many other factors contributed to this reduction, the Program clearly played a role in this. Program support for vertical programs also played a major role in reducing tuberculosis and malaria, with less impact on HIV/AIDS. This was done through support from the Program as well as complementary financing from the Global Fund to Fight AIDS, Tuberculosis, and Malaria, which operated within the Program’s framework. The Program represented one of the first large-scale initiatives to address emerging health issues and non-communicable diseases. It took the first steps to strengthen the health sector’s response to non-communicable diseases and strengthening emergency services. Its main contribution to addressing non-communicable diseases was increasing the focus on new health risks. xiii It was reasonably also efficient and represented good value for money. The estimated economic rate of return of at least 18 to 28 percent, suggesting good external efficiency. It also contributed to the internal efficiency of the sector. Most observers concluded that the Program was generally successful in bringing together different actors to coordinate and jointly plan. However many issues remain, including underutilized equipment and low levels of synergy among some programs. The Program’s monitoring and evaluation system was complicated and disorganized. It had at least two different and unrelated results frameworks, neither of which appeared to have played a major role in the oversight of the Program. In addition, the Program tracked many other indicators that were not included in the formal result frameworks. The Risk to Development Outcome is rated moderate. Overall, the Program had broad political support and its impact is not likely to be dissipated. A follow-on health program was approved in 2011. Bank and Borrower Performance There were a number of shortcomings in the preparation of the Program, including a complicated and unrealistic design that was unlikely to lead to the desired outcomes. Many of the issues from the previous Program were not fully resolved, which led to delays. In addition, the World Bank had to diffuse tensions left from the previous Program, in part as the result of a decision to suspend disbursements in 2004. The Bank team had limited preparation time. Supervision was relatively strong, with the World Bank playing an active role in administering the fiduciary aspects of the Program, providing technical assistance, and serving as a liaison between development partners and the government. The government provided the necessary budget as agreed. It also intervened when the Program was facing difficulties in the first year, leading to greater stability. The Ministry faced a number of challenges, including strengthening financial management. Ministry staff were frequently rotated. There were also concerns about the lack of qualified staff. The Ministry’s own budget process was complicated and inflexible, and there appeared to be little coordination across the different operational plans. Lessons  A complex challenge such as child nutrition is beyond the scope of the health sector by itself and requires a multi-sectoral approach. While there was a general understanding of the importance of nutrition and clear statistics about the magnitude of malnutrition, interviews suggest that it was essentially treated as a “side issue” with little high-level ownership or attention. The Program design was built on the assumption that the country would focus on nutrition given its importance. A more detailed analysis of political economy and stakeholders could have led to a better design of nutrition activities. An organization like the World Bank can play a role in convening development partners and to ensure that the evidence is shared across different sectors, in much the same way that was done with HIV/AIDS and the “Three Ones” approach. xiv  Not having a clear development objective can impact adversely on implementation and results. In the case of HNPSP, there appear to be at least five sets of program objectives from different official documents. In practice, many policymakers simply followed country-wide health statistics and used these to judge the Program’s impact. Without the guidance provided by an objective, the Program can easily slip into preserving the status quo.  The lack of intervention-level evaluations limits the Program’s flexibility and responsiveness. This evaluation shows the difficulty of evaluating a large Program such as HNPSP. There were many aspects of the Program that did not have strong evaluations. For example, the Program encouraged the ultimately unsuccessful contracting of NGOs and increasing hospital autonomy. There was very little evidence base on why these interventions might work and little attempt to do demonstration pilots.  Having a well-designed results framework is important for planning and monitoring. The Program had a complicated and often contradictory results framework. It had several sets of objectives, drawn from different partner documents as well as the Investment Plan. Without a better roadmap of what the Program wants to accomplish (an objective) and how it plans to accomplish it (a theory of change), it is difficult to generate much political support for the Program or encourage stakeholders to take necessary risks to achieve the objectives.  Without a structure for partners and the government to design, debate, and evaluate new policy, it seems unlikely that there will be structural reform in the future. One longstanding challenge in supporting the health sector is the difficulty in carrying out significant structural reforms, such as granting autonomy to hospitals. The SWAP did include some ambitious reforms, but most were never implemented. After some attempt to support health sector reform, the program largely focused on increasing financing and improving performance and service delivery. Establishing a reform infrastructure requires building trust and respect on both sides, which is a long term process. It also requires development partners to strengthen their understanding of political economy and to develop strategies to gradually expand their dialogue. At the same time, the government needs to identify its priorities and openly discuss its political limitations. Caroline Heider Director-General Evaluation 1 1. Background and Context 1.1 This Project Performance Assessment Report (PPAR) reviews the impact of the World Bank’s Health, Nutrition, and Population Sector Program in Bangladesh from 2005 to 2011. The program was at the center of development partners’ efforts to coordinate their activities in the health sector in Bangladesh, using what is commonly called a Sector-Wide Approach (SWAP). The Program was the second SWAP in the health sector and builds on the first SWAP, the Health and Population Project (1998 to 2005). It also incorporated activities from the National Nutrition Project (2000 to 2006), and the HIV/AIDS Prevention Project (2000 to 2007). 1.2 The World Bank provided a credit of US$293.4 million during the life of the Program and administered another US$387.8 million through a trust fund that received contributions from seven development partners. The World Bank also worked closely with other partners who supported the Program but administered their own resources. The World Bank Executive Board of Directors approved the Program on April 28, 2005 and it became effective on June 14, 2005. It closed on December 31, 2011, one year after the originally planned closing date. Socio-Economic Context ECONOMIC GROWTH 1.3 Despite significant economic growth, Bangladesh remains a low income country. When Bangladesh gained independence in 1971, it had a gross domestic product (GDP) per capita of US$92 (US$383 in constant 2010 dollars), making it the eighth poorest country in the world. By 2010, the per capita income was US$1,652, placing Bangladesh above the average for low income countries (World Bank 2012). Table 1.1. Key Economic Data 1992 2000 2003 2005 2006 2007 2008 2009 2010 GDP per capita (2010 US$) 860 1,076 1142 1,293 1,361 1,432 1,505 1,575 1,652 GDP growth rate (percent) 5.0 5.9 5.3 6.0 6.6 6.4 6.2 5.7 6.1 Current account balance 0.6 -0.6 0.3 -0.3 1.9 1.3 1.2 4.0 2.1 (percentage of GDP) Net development assistance 5.6 2.4 2.5 2.1 1.9 2.1 2.4 1.3 1.3 (percentage of GDP) Net worker remittances 2.9 4.2 6.1 7.2 8.8 9.6 11.2 11.8 10.8 (percentage of GDP) Source: All data from (World Bank 2012). 1.4 Economic growth has been driven by both the industrial sector and services. Central to the growth in the industrial sector is the rapid expansion of garment production, accounting for 78 percent of total exports (International Monetary Fund 2011). The second 2 major source of foreign exchange is from remittances, which have grown substantially and now account for 10 to 11 percent of GDP. POVERTY 1.5 While Bangladesh remains a low income country with an estimated 47 million people living below the national poverty line, it has seen a constant and dramatic reduction in poverty. In 1991, an estimated 59 percent of the population lived below the poverty line; by 2010, this had decreased to 32 percent. Extreme poverty has dropped even more sharply, from 41 percent in 1991 to 18 percent in 2010 (Government of the People's Republic of Bangladesh 2011). The decline in poverty has been relatively well-distributed and has benefited both poor and less poor regions and the Gini index has been relatively stable in the 0.30 to 0.35 range since 1995. The gap between urban and rural areas remains significant although it has been gradually closing (Government of the People's Republic of Bangladesh 2011). 1.6 Increasing real wages in rural areas in the past decade as well as increasing wages in urban areas from 2001 to 2005 have made an important contribution in reducing poverty (Zhang, et al. 2013). The growth in remittances has also played a major role in transferring resources to poorer households. Changes within the household have also played a major role, as families become smaller and women enter the workforce (Government of the People's Republic of Bangladesh 2011). Women account for 80 to 90 percent of the employees in the garment sector and the increase in female employment in Bangladesh has played an important role in improving the status of women and reducing poverty (Government of the People's Republic of Bangladesh 2012). DEMOGRAPHICS 1.7 With the exception of a few small island countries and city states, Bangladesh has the highest population density in the world, at 964 inhabitants per square kilometer (Government of the People's Republic of Bangladesh 2011). Since independence, Bangladesh has undergone a major demographic transition. While the population growth rate is still positive, it has slowed down substantially. This, combined with the increase in life expectancy, has led to a gradually aging population. In 1971, 45 percent of the population was under the age of 15, by 1991 this decreased to 42 percent, and to 31 percent by 2011 (World Bank 2012). Table 1.2. Population and Demographics 1974 1981 1991 2001 2011 Population (millions) 71.5 87.1 106.3 124.4 142.3 Population growth rate 2.6 2.3 2.0 1.6 1.3 Percentage urban 9.4 15.4 20.2 24.0 28.4 Source: Data on the urban population are from (World Bank 2012). Other population estimates are taken from (Government of the People's Republic of Bangladesh 2011) and author’s calculations. 1.8 Although the majority of the population of Bangladesh is Bengali, Bangladesh has a significant number of ethnic minorities. The most recent census estimates that around 1.1 3 percent of the population belongs to a tribal (ethnic minority) group (Government of the People's Republic of Bangladesh 2011). These groups have distinct languages, cultures, and customs. They are primarily located in the hilly areas of the southeastern region and the north (Rahman, et al. 2012). In 2010, an estimated 89.5 percent of the population was Muslim, with most of the remaining population Hindu. A small percentage of the population is Buddhist (primarily among the Hill Tribes) or Christians (Bangladesh Bureau of Statistics 2011). Health in Bangladesh HEALTH STATUS IN BANGLADESH 1.9 Bangladesh has seen significant improvement in the health status of its population and has made substantial progress towards meeting the health-related Millennium Development Goals (Government of the People's Republic of Bangladesh 2012). Starting from a very low base, Bangladesh has seen a decrease in both child and maternal mortality, as well as a reduction in fertility. Life expectancy is a common summary measure of a population’s health status. Figure 1 shows estimates of the life expectancy of males and females in Bangladesh from 1970 to 2010. The results show a rapid increase in the life expectancy of the population over the past four decades increasing around 13 or 14 years. Only a few countries have seen faster gains in the life expectancy than Bangladesh. Figure 1.1. Historic Life Expectancy in Bangladesh, 1970 to 2010 Source: (Wang, et al. 2012) 1.10 Bangladesh is passing through a rapid epidemiological transition (Harper and Armelagos 2010), including a reduction in the mortality and morbidity, as well as a change in the country’s health profile. Longitudinal data from 1986 to 2006 from the rural areas of Matlab, mortality due to traditional communicable diseases such as respiratory infection and diarrheal diseases decreased by around 80 percent. At same time, mortality due to non- communicable diseases such as cancer, strokes, and cardiovascular diseases increased substantially (Karar 2009). 1.11 While there are not precise data on the changing pattern of disease for Bangladesh, a recent burden of disease study provides an estimate for all of South Asia on the changing 4 patterns of disability-adjusted life years lost between 1990 and 2010.1 In South Asia, between 1990 and 2010, there has been a noticeable decline in the impact of communicable diseases as well as death due to maternal and neonatal factors. The burden of non-communicable diseases and injuries and violence play a more important role. This changed happened in an environment when the total number of years of life lost dropped by about 15 percent between 1990 and 2010 (Wang, et al. 2012). Figure 1.2. Share of Burden of Disease in South Asia, 1990-2010 Source: (Lozano, Naghavi, et al. 2012). Note: Burden of disease is measured in the number of disability-adjusted life years. For example, if diabetes reduces the life expectancy of 1,000 people by 10 years each, the total burden of the disease is 10,000 life years. The years of life lost is compared to an ideal life expectancy, usually around 80 years. The impact of both permanent and temporary disability can also be included, representing years of healthy life lost due to a particular condition. THE HEALTH SYSTEM AND HEALTH FINANCING 1.12 The health sector is disjointed with a highly centralized public health sector and a largely unregulated private sector. The public health sector is highly centralized, with most public facilities under the direct authority of the Ministry of Health and Family Welfare. In urban areas, primary health is provided by the Ministry of Local Government, Rural Development, and Cooperatives through its own network of health providers. In 2007, the Ministry of Health and Family Welfare2 accounted for fully 97 percent of all public expenditure on health. Local governments are estimated to provide about 1 percent of health financing. The health system is often described as “vertical” with many health services reporting directly to the Ministry and their particular program rather than to local authorities (Independent Review Team 2008). 1 Although there are important differences among the countries in the region, there are similarities in the life expectancy in Bangladesh, India, and Pakistan over the past twenty years. 2 For this evaluation, the term Ministry refers to the Ministry of Health and Family Welfare. For the sake of simplicity, the evaluation refers to the “health sector” to include health-, nutrition-, or population-related activities under the Ministry of Health and Family Welfare. 5 Box 1.1. The Government’s Strategic Investment Plan In 2004, the Ministry introduced the Health, Nutrition, and Population Strategic Investment Plan, 2003-2010 (Government of the People’s Republic of Bangladesh 2004) as a guide to the health sector’s medium term goals, outlining how these priorities will be addressed. Under the rubric “better health for a prosperous society,” the Investment Plan’s objectives include “…define an overall strategic framework to guide investments…irrespective of political philosophies” and to “…provide the basis for public investment plan revisions…over the next ten years.” The Investment Plan was also designed to “…define the Government’s intentions…as the basis for negotiation Development Partner assistance in the medium term.” The Investment Plan’s long-term goal is “to achieve sustainable improvement in health, nutrition and reproductive health, (including family planning) status of the people, particularly of vulnerable groups, including women, children, the elderly, and the poor with ultimate aim of their economic emancipation and physical, social, mental and spiritual well-being.” The Investment Plan was also prepared within the context of the Interim Poverty Reduction Strategy, whose objective focuses on increasing public financing for health care as well as improving the effectiveness, efficiency, and equity of this expenditure. The targets in the country-specific Millennium Development Goals were also included in the Plan. The Investment Plan outlined seven long term strategies: (i) stimulating informed demand; (ii) improving quality and scope of services; (iii) restructuring services; (iv) mobilizing more resources; (v) improving equity; (vi) improving efficiency; and (vii) improving sector governance and management. It also had ten medium term strategies (policy responses) for 2010 and five Headline Objectives: (1) Reducing maternal, neonatal and childhood mortality, and improving maternal and childhood nutrition; (2) Reducing total fertility to replacement level; (3) Reducing the burden of [tuberculosis] and malaria and preventing and controlling HIV/AIDS; (4) The prevention and control of major non-communicable diseases; and (5) Reducing injuries and implementing improvements in emergency services. These Headline Objectives are effectively the Strategic Investment Plan’s components; with the exception of some technical assistance activities, all of the Program’s activities are grouped under these objectives. The Program had 38 Operational Plans. These Plans, which contained a variety of activities, do not necessarily correspond to the Headline Objectives. For example, Micronutrient Supplementation (#18 of the Operation Plans) and the National Nutrition Program (#32) clearly fits under Objective 1, which covers nutrition. In contrast, Pre-service Training (#10) and Health Services Procurement (#11) are active in all objectives except 2, which covers Family Planning. Concretely, the Investment Plan calls for a total 315 billion taka, around US$5.5 billion from July 2003 to June 2010. 1.13 At the lowest level are community clinics, which are designed to support a population of 6,000 to 10,000. In principal, community clinics have a staff of three health workers. In 2010, there were 413 Upazalia Health Complexes, with a total of 14,557 beds. The health complex provides a mix of out-patient and in-patient care, including emergency services and emergency obstetric care, minor surgery, and basic laboratory services. At the secondary and higher levels, there are district hospitals (located in 59 out of Bangladesh’s 64 districts) and 9 general hospitals, with a total of 8,900 beds. In addition, the public sector operates 17 6 medical colleges and hospitals, which are the primary source of training for physicians and other health professionals.3 1.14 The private sector plays a major and largely unregulated role in the health sector. According to official statistics there are about 2,600 private hospitals and clinics as well as 4,500 diagnostic laboratories. A large number of private sector health providers are “moonlighting” from public sector positions. As cited in Bank supervision documents, there are around active 17,600 doctors (63 percent of all active doctors) who do not have a public sector appointment and presumably work full time in the private sector. Bangladesh has a large pharmaceutical industry that has grown substantially in the past three decade, with current sales of around $1.3 billion and exports generic products throughout the world. This industry supplies around 75 percent of the domestic supply and helps to maintain low drug prices in the country (Ahmed, et al. 2013). 1.15 Bangladesh has a large, dynamic, and influential NGO community (Mushtaque, et al. 2013). These NGOs provide a wide variety of services, ranging from the provision of community health workers to support for hospitals. While these organizations have worked in partnership with the government in a number of areas (particularly tuberculosis, family planning, and urban health), they often work in parallel with the government or focus on populations that have problems with access (Ahmed, et al. 2013). 1.16 Pharmacies and traditional healers also play an important role in providing health care at the primary level. Most rural communities have a “village doctor” and have access to a ready supply of drugs. For example, in Chakaria, a poor district in the southeast of the country, a study found that there were around 2 formal sector health workers (physicians, nurses, trained midwifes, etc.) per 10,000 population compared to around 11 village doctors, 17 traditional birth attendants, and 5 traditional healers per 10,000 inhabitants (Wahed, Rasheed and Bhuiya 2008). 1.17 From 1997 to 2007, as outlined in Figure 3, spending on health from all sources has increased from around 2.7 percent of the GDP to 3.4 percent or an increase per capita spending from US$9.20 to US$16.20. In 2007, 65 percent of health expenditures came from households and private firms. The largest portion of private health expenditure goes to “drugs and medical goods retail outlets;” in 2007 (the last year that is available) this accounted for 66 percent of expenditures by households or 1.4 percent of the GDP (Health Economics Unit 2010). 1.18 Development assistance plays an important role in the health sector, accounting for around one third of all public expenditure on health in 2007. This share has been growing, in part due to the influx of financing due to HNPSP as well as due to support from the Global Fund. (Health Economics Unit 2010). 3 Bangladesh has 7 divisions, divided into 64 districts. These districts, in turn, are divided into 500 upazilas and 509 thanas. An upazila is a unit of local government, usually translated as “sub district.” In urban areas, sub-districts are called thanas. 7 Figure 1.3. Share of Health Spending, as a percentage of GDP Source: (Health Economics Unit 2010) THE WORLD BANK AND THE HEALTH SECTOR 1.19 Since independence in 1971, the World Bank has been quite active in the health sector (see Annex B). At independence, population growth was seen as one of the most important development challenge and it not surprising that the World Bank started focusing on population. Soon after the World Bank started to include primary health (known as “family health”) in addition to population. In the 1990s, the World Bank started a number of specialized projects, focusing on nutrition and HIV/AIDS. The World Bank has always worked with other development partners through co-financing (with the partner providing resources directly to the Bank) or on a parallel basis (with development partners following the same outline but providing independent financing). Each project built on the previous projects, while trying to increase coordination among development partners. 1.20 Bangladesh was the host of the one of the first health sector SWAPs in the world, from 1998 to 2005. The first SWAP, the Health and Population Program, aimed to bring together development partners under the auspices of the government’s health strategy. It builds upon a long tradition of jointly financed health projects as well as a desire to reduce the number of development projects. The large number of projects was seen as a burden on the Ministry as it taxed its administrative capacity. In addition, uncoordinated development partners tended to “cherry pick” the most attractive areas in the health sector and ignore the Ministry’s own priorities (Buse and Gwin 1998). 8 Box 1.2. Sector-Wide Approaches As the name implies, a Sector Wide Approach (SWAP) is an approach to organizing development support for a specific sector (Kostermans and Geli n.d). While definitions of SWAPs vary, there seems to be a general consensus that a SWAP should encompass a number of characteristics, including (World Health Organization 2006, Ahmed 2011):  A clear sector policy and strategy;  A government-led process for donor coordination at the sector level;  A medium-term expenditure program and annual budget;  A performance-monitoring system to measure progress;  A process for moving towards harmonized fiduciary systems; and  A broad consultation mechanism that involves all stakeholders. A well-designed SWAP should bring a number of benefits. First, it should contribute to reducing the transaction costs for the government as it eliminates partner-specific fiduciary rules and multiple implementation arrangements. Second, a SWAP should ensure consistency across projects to ensure that partners are not “cherry-picking” issues and are working together. Finally, a SWAP should build ownership since it mainstreams the government program and aligns external support to the government’s program (Buse and Gwin 1998). In practice setting up a SWAP can require several cycles (World Health Organization 2006). For development partners, a SWAP may represent additional work as they spend more time coordinating both with the government and with other development partners. It can also be difficult for a development partner to explain its individual impact and attribution within the context of SWAP to their domestic stakeholders. How can SWAPs be evaluated? A recent approach (Vaillancourt 2012) proposes evaluating SWAPs against three broad criteria:  Establishment of better tools for sector coordination. The SWAP should help improve the relevance of strategies and expenditure programs.  Greater harmonization and alignment of development assistance. The SWAP should improve the coordination of the development partners and encourage the greater use of country systems.  Enhanced sector stewardship. The SWAP gives the government a central leadership role and an ability to measure results and accountability. 1.21 Despite the promising start, the first Program was generally considered a failure. The World Bank’s Independent Evaluation Group (IEG) gave the program an “unsatisfactory” rating, which mirrors the Bank’s own internal rating for the program. The IEG evaluation identified tension between the government and the partners over policy issues as well as unutilized infrastructure as emblematic of the Program’s lack of impact. The evaluation further argued that the program caused disruptions in the health sector, leading to a reduction in family planning and in access for the poor to health facilities (Independent Evaluation Group 2006). In 2005, a second SWAP was established, the Health, Nutrition, and Population Sector Program, the subject of this evaluation. 9 2. Objectives, Design, and their Relevance 2.1 Evaluating a SWAP can be complicated due to the large number of actors and the difficulty assigning individual attribution of the Program’s impact. For the purpose of this evaluation, the government’s overall Program (including the government financing and the Program financed by the World Bank and other partners) will be referred to as the Program, the second Program, or HNPSP. 4 The consortium of development partners is referred to as the Partners. Finally, the share financed by the World Bank (through its International Development Association) will be called the World Bank Program or the World Bank’s contribution to HNPSP. Because of the extensive use of basket funding on the part of development partners as well as the close coordination between the partners and government on investment, the evaluation will focus on outcomes that are attributable to the Program. As the lead financial partner, it is especially difficult to separate the World Bank from the Partners or the Program. Whenever possible, the evaluation will identify the value added provided by the World Bank to the Program. The Program’s Objectives 2.2 The Health, Nutrition, and Population Program had two different sets of objectives. The first comes from the legal agreement between the Government and the World Bank, known as the Development Credit Agreement, which states that the Program’s objective were to (page 19): (i) reduce infant, under-five, and maternal mortality and the proportion of malnourished children; (ii) eliminate the gender disparity in child malnutrition and mortality; (iii) ensure increased access to reproductive health services; (iv) lower total fertility with a view towards achieving replacement level fertility by 2010; (v) reduce the burden of tuberculosis, HIV/AIDS, malaria, and other priority diseases; (vi) initiate a system to control newer health threats and protect health risk by improving emergency services; and (vii) improve the prevention and control of non- communicable diseases. 2.3 The second set of objectives is drawn from the Program’s Project Appraisal Document (PAD), which states that the Program (pages 4-5 and 33): …assists the Government of Bangladesh in the implementation of its Strategic Investment Plan (SIP), 2003-2010… and …the main purpose of SIP (2003-2010) will be to increase availability and utilization of user-centered, effective, efficient, equitable, affordable, and accessible quality 4 This evaluation will refer to three different programs: the first Program, the Health and Population Sector Program (P037857, from 1998 to 2005); the second Program, the Health, Nutrition, and Population Sector Program, which is the subject of this evaluation; and the third Program, the Health Sector Development Program (P118708, approved in 2011). 10 services, be it the Essential Services Package, improved hospital services, nutritional services, or other selected services. 2.4 While these two sets of objectives are complementary, they have different focuses. The Development Credit Agreement concentrates on higher level, health-related outcomes. Five of these, known as the five Headline Objectives, were taken directly from the Strategic Investment Plan as accelerated service priorities (Government of the People’s Republic of Bangladesh 2004).5 In the World Bank’s legal agreement, two objectives (number ii, focusing on gender disparity and number iii, focusing on reproductive health) are logical extensions of the other five. 2.5 The PAD objectives are more focused on the health sector and improving its functioning. While the Strategic Investment Plan does mention each of the elements in the PAD objectives, they do not receive special attention nor are they identified as objectives (Government of the People’s Republic of Bangladesh 2004). 2.6 Thus it appears that the Development Credit Agreement reflects a better understanding of the government’s Strategic Investment Plan. In evaluating World Bank projects, the standard practice is also to give precedence to the legal agreement, unless there are compelling reasons to do otherwise. 2.7 The objectives for this evaluation will be expressed as: a) Reduce infant, under-five, and maternal mortality and the proportion of malnourished children, while eliminating gender disparity and strengthening access to reproductive health services; b) Lower the fertility rate with a view towards achieving replacement level fertility by 2010; c) Reduce the burden of tuberculosis, HIV/AIDS, and malaria; d) Initiate a system to control newer health threats and protect health risks by improving emergency services; and e) Improve the prevention and control of non-communicable diseases.  2.8 This reflects the fact that the elimination of gender disparity in child malnutrition and mortality (number ii) and increasing access to reproductive health services (number iii) are extensions of the first objective. Program Design 2.9 The Program had three components. The first component focused on objectives for service delivery and traditional public health issues in Bangladesh, including the achievement 5 There are several minor differences. The Strategic Investment Plan includes “reducing maternal malnutrition” in its Objective 1. Its Objective 2 does not specifically announce 2010 in its headline; the date is made clear in the description. Finally, its Objective 3 only includes tuberculosis, malaria, and HIV/AIDS as the priority diseases. These differences have little effect on the actual program. 11 of the health-related MDGs. The second component aimed to develop and implement policies and strategies to address the changing health pattern of the country, due to urbanization and aging of the population. The third component was to address major policy reforms and strategies in order to achieve better equity and efficiency in the sector. 2.10 Component 1: Accelerating achievement of HNP-related MDG and Poverty Reduction Strategy Paper goals. The component was intended to support the delivery of a package of essential health services. The package would focus on actions that aimed at: (a) reduction of maternal mortality; (b) reduction of neonatal mortality; (c) reduction in childhood morbidity and mortality; (d) improvement in the nutritional status particularly of adolescent girls, pregnant and lactating women and children; (e) reducing fertility to replacement level; and (f) reducing the burden of TB and malaria and preventing and controlling HIV/AIDS. 2.11 In the case of child nutrition, the Program planned to incorporate the interventions under the National Nutrition Program which was a separate project financed by the World Bank that was active from 2000 to 2006. The National Nutrition Program was a community- based nutrition Program that provided support, through NGOs, to rural beneficiaries. Support for HIV/AIDS was to be provided by incorporating the HIV/AIDS Prevention Project, which was active from 2000 to 2007 and also used NGOs for part of its activities. 2.12 The first component maintained the focus on traditional public health issues that continued to plague Bangladesh, including direct health interventions (for example, the provision of vaccinations to children or deworming treatments), interventions to stimulate demand (for example, vouchers for pregnant mothers), and improved information to providers and beneficiaries. This was largely a continuation of the support provided by the predecessor Program, the Population and Health Program (World Bank 1998).6 2.13 Component 2: Meeting emerging HNP sector challenges. This component was designed to address emerging health challenges. The component focused on: (a) the reduction of injuries due to accidents and the improvement of emergency services; (b) prevention and control of major non-communicable diseases; (c) urban health service development; and (d) improvement of the health sector’s response to natural and manmade disasters. The component consisted of activities that were not included in the previous Program. The interventions included support for communication and policymaking, as well as several targeted health insurance pilots. 2.14 Component 3: Advancing HNP sector modernization. This component included support to reforms that improved health sector management. In particular, it supported efforts to improve the Ministry’s stewardship of the sector, moving it away from its traditional role as the provider of health services. The component also included activities to improve the efficiency and equity of health care financing. The component would focus on improving institutional and personal skills for (i) better planning and monitoring; (ii) improved budget management through a medium term budgetary framework process; (iii) 6 A rough estimate shows that around 98 percent of the resources of the previous Program were used for traditional health services. 12 reform management; (iv) improved aid management; (v) development of proper contract documents and management of contracts with private and non-government organization (NGO) providers; (vi) information management; and (vii) development of alternative financing mechanisms. 2.15 The Program used the structure of an investment loan, with partners providing financing according to the agreed plan of activities. In order to ensure sector-wide impact, the World Bank managed funds from several development partners through a dedicated trust fund. The Program used the Ministry’s Strategic Investment Plan (Government of the People’s Republic of Bangladesh 2004) as a roadmap for investment. This was an important element of sector-wide coordination. Partners that did not contribute to trust funds could use the Strategic Investment Plan to guide their own investments. To strengthen this process, the Program design included a formal annual review that involved a technical analysis, a review of activities and achievements, and agreement on future activities. The Health Sector Consortium was to engage regularly with the Ministry and other government agencies. This was to be complemented by various events, such as the Health Users Forum. 2.16 The Program was organized into 38 different Operational Plans.7 Each Operational Plan had a director and was typically placed under either the Director General of Family Planning or the Director General of Health Services, although a few plans were “independent.” Some Operational Plans focused on a type of HNP service, such as Essential Service Delivery, National Eye Care, or Family Planning Field Services. Others are cross cutting, such as Pre-service Education, Human Resource Management, and Health Economics. In services, the Directorate General of Health and the Directorate of Family Services had parallel Operational Plans, such as Procurement, Financial Management, and Sector Wide Management. 2.17 As the name suggests, each plan had detailed activities and identified the necessary procurement and other support from the Program. While 38 programs appear to be a large number, it represents a significant decrease from the number of free standing projects and programs that existed before the SWAPs. While there was certainly room to consolidation (the current health Program, the Health Sector Development Program, has 32 Operational Plans), the plans were reasonably well focused and represent a reasonable division of the sector. Likewise, despite the fact that each plan had certain autonomy, procurement and financial management were to be managed centrally which would generate some synergies. Relevance of the Objectives The relevance of objectives is defined as the extent to which an operation’s objective are consistent with the country’s current development priorities and with current Bank…strategies and corporate goals (OPCS 2011). 2.18 The relevance of the objectives is rated as high. This rating reflects its relevance to the World Bank’s country program as well as its relevance to the country’s development strategy. 7 Some documents refer to them as Operation Plans. 13 2.19 Conceptually, the Program was well aligned with high-level national priorities and priorities in the health sector. It was also well aligned with the World Bank’s Country Assistance Strategy (World Bank 2000, World Bank 2006) and it’s Health Sector Strategy (World Bank 1997) at the time of approval. 2.20 The Program still remained relevant at the time of closing. The World Bank’s current Country Assistance Strategy for Bangladesh, covering the period from 2011 to 2014, makes specific reference to the successor Program. The health challenges remain similar as does the Bank’s approach to these problems (World Bank 2010). The World Bank’s strategy for the sector also emphasizes the importance of strengthening health systems, which is very much in line with the Program’s objective and design (World Bank 2007). 2.21 The Program was developed closely around the Strategic Investment Plan, which is the government’s health sector strategy. The different national development strategies, such as the 2005 National Strategy for Poverty Reduction and the 2009 National Strategy for Accelerated Poverty Reduction (Government of the People's Republic of Bangladesh 2009) as well as the Sixth Five Year Strategy (Government of the People's Republic of Bangladesh 2011), all make specific reference to the Program, its approach, and the specific interventions supported by the Program. It is clear that the Program played an important role in the preparation of subsequent development plans. Relevance of Design The relevance of design is defined as the extent to which the project’s design (its planned activities or policy areas) are consistent with the stated objectives, including assessment of the Results Framework. The Results Framework represents the underlying project logic linking the inputs and outputs to the outcomes (OPCS 2011). 2.22 The relevance of design is rated as modest. This rating reflects the design of the Program, including the World Bank’s Program. THE PROGRAM ARCHITECTURE AND THE DESIGN OF THE SWAP 2.23 As the name implies, a Sector Wide Approach (SWAP) is not a specific development product, it is an approach (Kostermans and Geli n.d). While there is no single ideal model for a SWAP, there are a number of characteristics that are commonly part of SWAPs. Table 3 presents the design of the Program against six key characteristics that SWAP should have (World Health Organization 2006). 2.24 The design balanced a number of different issues. First, there was a consensus that the Program should be relevant to the government’s development and health strategies. Second, development partners wanted to assure that the Program was protected from corruption. Third, all stakeholders agreed that the Program should move away from project financing, with financial support for a well-defined set of inputs and move towards budget support, which provides financing through the government budget. Fourth, all stakeholders wanted to incorporate the lessons from the previous health sector programs (World Bank 1998) in the new design to avoid a repeat of the same problems. 14 Table 2.1. Program’s Design as a SWAP SWAP Characteristic Remark Clear sector policy and There was a single strategy, the Strategic Investment Plan, which strategy outlined the investments for the Program as well as providing its strategic guidelines. Some vertical health programs were formally in the SWAP but are effectively managed independently. Government led process of There was an independent health consortium. While there was a wide development partner ranging dialogue, policy discussions focused primarily on the coordination development (capital) budget. An appropriate medium term The development budget was discussed among all partners, although and annual budget was inflexible. The revenue (current) budget was effectively out of the SWAP. A performance monitoring There were joint review missions that include a review of indicators. It system is not clear how these were used in practice. Progression towards Most resources were pooled, with the World Bank managing resources harmonized systems and handling fiduciary aspects for the development partners. While government procedures were used for some transactions, World Bank procedures prevailed. Consultation mechanism The health consortium included both development partners and civil with all stakeholders society and met with the government through joint missions and other meetings. Source: (Lozano, Naghavi, et al. 2012). 2.25 The decision to use a project approach reflected concerns about transparency and governance. A traditional investment loan provides fiduciary control over the investment, through the World Bank’s normal review process of the procurement as well as audit of the financial statements. While the government was responsible for carrying out activities, the Bank provided assurance to development partners. At the same time, the Program did include some elements of performance-based support. This was done by creating a specific disbursement category to disburse on the basis of results following achievement of a number of indicators (World Bank 2005, 11). While this was not designed to be the primary form to disburse funds, it did allow some experimentation with result-based financing. However, the MandE framework was weak and did not provide the necessary instruments to carry out meaningful results-based financing. TECHNICAL ELEMENTS AND THE CAUSAL CHAIN 2.26 The Program aimed to address all of Bangladesh’s health, nutrition, and population challenges. Given the complexity of the health sector in Bangladesh, this led to a program with a large number of activities. The Program’s division into traditional health issues, emerging health issues, and policy initiatives was a reasonable approach to address the country’s epidemiological transition. Table 4 outlines how the different components addressed the health challenges that the government health sector strategy identified. 15 Table 2.2. Relevance of the Program to Health Sector Challenges Challenge Description Relevance of Design Stimulating informed The demand for certain key The Program included initiatives to improve demand services was quite low leading to communication and to introduce demand-side poor health outcomes. vouchers. Improving the quality The quality and coverage of The Program included investment for medical and scope of services public services was general low equipment, training, and refurbishing medical and not improving. facilities. Restructuring the way The country had limited coverage The Program included the use of contracting for services are provided for the growing urban population primary health services. and limited emergency care. Mobilizing more Public spending for health was There was little focus on mobilizing new resources resources low and inadequate. beyond planned technical assistance in health financing. Improving equity There were significant The Program focused on health problems of the differences in health outcomes by poor, such as nutrition, as well as coverage in geographic areas and social remote areas. groups. Improving service Public health providers were Contracting was proposed to improve how services delivery inefficient. were delivered. Improving governance The sector needed to improve its The Program included significant investment in and management financial management as well as financial management. The use of a SWAP was informal payments and also an important element to improve governance absenteeism. and management. Source: World Bank 2005. 2.27 A review of the Program’s different activities and of the Operational Plans shows that the Program covered all of the technical areas that were included in its objective. Several Operational Plans contributed to several elements of the Objectives at the same. Nutrition, in particular, did not receive much attention in the Bank’s PAD or in the Strategic Investment Plan. 2.28 There were several major cross-cutting areas that were not included in the Program. The Program did not touch on human resources, which play a critical role in delivering health care to the population (Independent Review Team 2008). While the Program did support training, it did not fundamentally adjust how staff were deployed. With the exception of reproductive health services, the Program did not address the incentives that staff faced. A second, related area was the lack of focus on health expenditures through the revenue budget.8 It is not clear why this was excluded from the Program. The Program took a strategic risk by contracting out health services, particularly primary health services and nutrition. This drew from the positive experience in the Asian Development Bank’s Urban 8 The budget in Bangladesh is divided into the revenue budget, which roughly corresponds to current expenditures and the development budget, which roughly corresponds to capital expenditure. In many areas the division is arbitrary Invalid source specified.. Both the revenue and development budget are about equal in size. 16 Health Project in Bangladesh as well as from contracting in other contexts (Lovinsohn and Harding 2005). 2.29 The Program did not have a well-articulated approach to reducing child malnutrition. The Program planned to incorporate the National Nutrition Program (an Operational Plan) into the SWAP when the World Bank’s investment loan ended, in 2006. While the World Bank’s National Nutrition Program still had an internal rating of satisfactory in 2004, internal reports make it clear that the project was facing difficulties. In addition, the project’s focus was limited geographically, which made it difficult to achieve national impact (World Bank 2007). The National Nutrition Program’s technical approach focused largely on medical interventions which are important but generally not sufficient to improve malnutrition on their own. 2.30 Another concern was the complicated causal chain articulated in different program documents. Central to this was the division between the results framework, which was designed to be used by the World Bank for its own reporting purposes, and the logical framework, which was expected to be used “…by a majority of development partners...” (Martinez 2008). 2.31 The results framework, which is described in Annex 3A of the PAD, was built around the objective outlined on pages 4 and 5 of the PAD on improving the “…availability and utilization of…HNP services.” Its outcomes focused on the utilization and allocation of resources to the poor. It did include higher level outcomes, such as the maternal mortality rate although there is no clear path how these outcomes were to be “produced.” 2.32 The logical framework, in Annex 3B had an overall goal (poverty eradication) as well as program goal (sustainable increase in health and nutrition status). Many of the activity indicators would have been more appropriately expressed as outcomes, such as “deliveries attended by skilled personnel” or “percentage of [STD] cases among targeted groups.” This follows the traditional approach used in developing logical frameworks. 2.33 The PAD indicated that the results framework used a subset of indicators from the Strategic Investment Plan. It argued that this would help ensure that the Program would focus on indicators largely under its control. The PAD also indicated that the World Bank would use the results framework while others would largely use the broader logical framework (World Bank 2005, 5). It is difficult to place the results framework as a subset of the logical framework. Although the results framework included many elements that would likely to contribute to the desired outcome, it did not appear to have a causal chain that would be useful to identify the Program’s success and areas that needed more detailed attention. 2.34 The overall picture is of an ambitious and complicated design that had a number of important gaps. HNPSP introduced a number of new elements that were not included in the previous Program or in most other health projects, including nutrition in component 1 as well as most of the activities in components 2 and 3. Many activities do not appear to be coordinated. Some elements of the objectives, such as maternal health, receive a significant attention and a large percentage of resources while others, such as non-communicable diseases, appear to receive very little attention. 17 3. Implementation Background and Preparation 3.1 In 2001, the newly elected government expressed interest in changing directions in the health sector. While the World Bank, through the first Program (the Health and Population Program), showed flexibility in some areas, there were several areas where there was significant disagreement between the government and development partners (Kostermans and Geli n.d). In particular, the development partners were committed to merging family planning workers with health workers and with granting public hospitals autonomy. As a result of these policy disagreements, the World Bank suspended the loan from May to July, 2003. According to the PPAR, there was “…a common perception amongst many that the Bank and other pool donors over-reacted and handled the matter in an inflexible matter” (Independent Evaluation Group 2006, 45). The PPAR goes on to state that “…the Bank in particular, exhausted all their political capital in the unsuccessful fight over [the ] unification [of health and family planning services]… Post-reunification HPSP was virtually a curse word, and nothing associated with it could expect support” (Independent Evaluation Group 2006, 16). 3.2 While the suspension was widely condemned in Bangladesh, it also planted the seeds for the Health, Population, and Nutrition Program. Effectively, the suspension showed that each side was committed to health sector reform (albeit with different visions) and increased the interest in moving away from financing inputs following a rigid implementation plan. One senior Bank staff commented that… “ [the] intensive dialogue subsequent to the partial suspension of disbursements has contributed to [the government’s] commitment to move toward a results/outcome oriented…health sector reform agenda…”9 3.3 Within the World Bank, the Program’s concept review meeting was held on February 11, 2004 with appraisal held nine months later on November 17, 2004. The Program was approved by the World Bank’s Board of Directors on April 28, 2005. This was a relatively short period, especially given the size and complexity of the Program compared to the approximately two years that was spent designing the Health and Population Program. The preparation was also plagued by disputes over the Health and Population Program.10 3.4 Given the tensions over the previous Program, the World Bank appointed a senior staff member with broad health and implementation experience, supported by another internationally recruited staff based in Dhaka. A total of 22 professionals (including five consultants) participated in preparation of the project. The total preparation budget was approximately US$812,000, of which US$214,000 was financed by trust funds. “…a very ill patient in need of intensive care” 9 Comment taken from Project Status Report, June 25, 2003. 10 By July 1, 2004 the Health and Population Program has almost fully disbursed its resources although it did not formally close until June 30, 2005. 18 3.5 The Program became effective on June 14, 2005. Almost immediately, implementation was delayed in several critical areas. One of the first challenges that the Program faced was the audit results from the previous Program, which had not been fully addressed. Internal records show that disbursements were suspended for around a year as these observations were being addressed. 3.6 The World Bank was responsible for supervising many areas of the Program and this required significant work. Supervising the Program accounted for approximately half of the World Bank Dhaka’s office procurement workload and the staff was similarly taxed in other areas. The World Bank had a number of staff members supporting the Program, both in Washington and in the Dhaka office. 3.7 The Program put pressure on the Ministry and many key positions were either left vacant or had a high turnover. Internal records showed concern with the Program’s financial management and procurement. There was particular concern about three major procurement packages that were important to support the Program’s management and its plan to use NGOs to provide basic health services. The packages were to contract organizations that were considered critical to the success of the Program and the reforms that it supported. 3.8 The annual review for 2006 (Independent Review Team 2006) was the first review dedicated solely to the Program, as the 2005 annual review was focused on closing the previous program. The Team raised concern about the lack of focus on human resources in the Program. The review indicated that key staff had little knowledge of how a SWAP works or of the details of the overall Program. The review recommended that the Ministry contract more line directors. The review also made recommendations about improving MandE and the financial management of the Program. 3.9 Financial management was also a concern during the first years of implementation. The first joint aide memoire on the Annual Program Review (held from April 10 to April 24, 2006) notes that “Financial management has not improved sufficiently to cope with a program of the magnitude of [the Health, Nutrition, and Population Sector Program” and further states that “This issue needs to be addressed urgently…” (HNP Donor Consortium 2006). 3.10 By the second annual review in 2007, the situation was seen as being more critical. The Independent Review Team described the Program as “…a very ill patient in need of intensive care...” and quite possibly at the risk of becoming “…terminally ill” (Independent Review Team 2007). This description stayed in the mind of many stakeholders and observers. In particular, the review described a number of “fundamental bottlenecks” that were preventing the Program from reaching its objectives. The review also found that the technical assistance was uncoordinated and ignored many “neglected” areas. The review also expressed concern about the lack of progress in establishing the support agencies that were to support the Program and the efforts to contract out certain services. 3.11 While there were many concerns about Program’s impact and its capacity meet its objectives, it saw significant disbursement. By 2007, the disbursement of World Bank was largely as expected and the government had made necessary contribution to the Program. The 19 disbursements from the World Bank-supervised trust fund were low and the partners decided to temporarily suspend disbursements from IDA to allow the trust fund to start disbursing. The Ministry’s Annual Implementation Report for 2007 outlined the activities carried out by the 38 Operational Plans (Planning Wing 2007). The report indicated that there were often delays in contracting and procurement as well as a lack of skilled personnel in many areas. There were also a number of important achievements, including the rapid response to avian influenza (H5N1), submission of required financial management documents, and expansion of the tuberculosis control program. The Directorate of Family Planning also began to take over the procurement of contraceptives, which were traditionally provided as in kind assistance by development partners. 3.12 The first years of the Program also saw the production of a significant amount of policy research and technical assistance. Some of this work focused on the contracting model and supporting ways to introduce contracting, although there was work on other areas such as improving the targeting of resources to the poor. Most of this work was financed and carried out by the World Bank and other development partners. “The patient is still in critical condition but recovering” 3.13 The Program’s midterm review was held in early 2008, around three years after the Program was approved by the World Bank’s Board of Directors. Continuing with the medical analogy, the Independent Review Team reported that “The patient is still in critical condition but [is] recovering.” The Review Team felt that while the Program itself was not in danger, it was not clear how much of this was due to actions by the government and partners or was the result of inertia (Independent Review Team 2008). 3.14 Specifically, the midterm review noted that while the Program was supporting service delivery, which was generally improving, it was having limited impact on the reform agenda. The Program was challenged by the complicated nature of the administrative structure, both in the health sector and in other areas of the government. The Program also suffered from the lack of management information systems, with the exception of the family planning program, which established a procurement tracking system in 2008. 3.15 Human resources remained an issue. At the program level, there was still a high level of turn-over in key positions as well as concerns about the capacity of Ministry staff. The government and partners agreed (in 2007) to identify ten major operational plan (line) directors and to try to reduce their turnover and provide them with basic training. Likewise the Chief Adviser’s11 Office carried out monthly review meetings to monitor the progress of the Program (Implemnetation Planning and Monitoring Division 2011). There was a general feeling that the government’s ownership of the Program was increasing as the Ministry came to understand its role in the health sector, such as the procurement of contraceptives. 3.16 Internal reports showed that disbursement was strong and met targets. Procurement plans were prepared in a timely fashion however they seemed to be biased towards the 11 The Chief Adviser is the non-political equivalent of the Prime Minister during a period of caretaker governments, when Parliament is not in session. 20 purchase of goods with little focus on services. The establishment of the Program Support Office, which was to provide technical support, and of Management Service Agency, which was to supervise the contracting of health services from private sector providers, was delayed and these organizations were not created until the first semester of 2008. Likewise, there were also delays in contracting an organization (including a Procurement Cell) to provide administrative support to the Program. 3.17 In addition to the delays in these key contracts, there were a number of official complaints filed. This is common in Bangladesh and the health sector does not appear to be an outlier. The World Bank carried about an investigation of the ingredients of one drug that was procured under the Program and also suspended a NGO that was hired under the HIV/AIDS sub-component. Development partners remained concerned about financial management issues but there was consensus that quality in this area was improving. While external audits were carried out on a regular basis, internal audit was often delayed. “…modest optimism” 3.18 Despite a difficult start and first years, the Program closed with what the 2009 Annual Review described as “modest optimism.” In part, this was driven by Bangladesh’s strong health outcomes seen in the 2007 Demographic and Health Survey. This was combined with a general feeling that the most serious financial management and procurement issues were largely resolved and that the Program was moving as expected. The Program still had some financial management issues, reflected in the number of audit observations. The Bank indicated that these concerns were less substantial and raised its internal financial management ratings. Likewise, internal records show that the quality of the procurement process had improved. 3.19 Following the mid-term review, the Bank accelerated its support for independent procurement surveys and assessments, which improved the knowledge on the impact of procurement on the health sector. This feedback was important to both the government and the development community to understand what was working and what was not. Around the same time, the Bank and other partners supported a health sector assessment and a study of health financing (Health Economics Unit 2010). 3.20 After the 2009 election, the new government decided to refurbish the community clinics that were built under the previous Program and were largely abandoned. The government carried out the refurbishment of these facilities outside of the Program’s structure and did include its investments in its operational plans. The Program provided equipment and drugs to these clinics after they were refurbished. 3.21 Development partners and the government agreed to modifications of several controversial parts of the Program. As a result, the Program did not close on the acrimonious note that marked the end of the first Program The World Bank, partners, and the government devoted significant resources to prepare the follow on operation (known as the Health Sector Development Program), which was to be the third Program in the health sector. As a result, the new Program was developed with greater coordination than had been the case with the 21 second Program. By many accounts, the government had a higher sense of ownership in the design of the third Program than in that of second Program (World Bank 2012). 4. Achievement of the Objectives 4.1 Health outcomes are the results of many factors, including income and education, place of residence, environmental factors, and the interaction with both public and private providers. Research has shown that Bangladesh has a myriad of health services providers and that many different factors have explained the Bangladesh’s success in health (Chowdhury, et al. 2013). This is particularly true of outcomes related to child mortality, nutrition, and fertility although it affects all of health outcomes. The evaluation of the efficacy will focus on the attribution of the Program towards the objectives and this will be the measure of impact. When appropriate and possible, the evaluation will identify the contribution of the World Bank-financed Program on reaching the objectives. Only one rating will be assigned for efficacy. Box 4.1. The Results of Other Evaluations of the Program HNPSP is one of the largest health SWAPs in the world and has been evaluated by a number of organizations, each with its own evaluation methodology and focus. The World Bank’s Implementation Completion and Results Report (World Bank 2012) focused on the project’s objective identified in the PAD (page 4-5), which aimed to increase the “…availability and utilization of user-centered, effective, efficient, equitable, affordable, and accessible quality services.” The evaluation finds the program’s outcome satisfactory, based on achieving the high- level indicators including more spending at the local level as well as an increase in antenatal care and attended births. In addition, the report cites increase in public health activities such an increase in tuberculosis case detection, immunization, and vitamin supplementation. The United Kingdom’s Department for International Development also prepared a post-program evaluation that was based on the logical framework (Department for International Development 2012). Overall, the program was given a total impact score of 68.75 percent, with a medium risk. The evaluation gave strong ratings for the achievement of many of the sector goals while recognizing that the program had limitation and that “…improving health systems was not as expected.” The government’s Implementation Monitoring and Evaluation Division, which is under the Ministry of Planning, prepared an evaluation of the program that reviews both inputs as well as outcomes (Implemnetation Planning and Monitoring Division 2011). The report does not include ratings. It recognizes the strong achievement that Bangladesh has made in most health outcomes and generally benefiting. It also states that “[it] is however not possible to appropriate these improvements to health sector interventions or specifically to [the program]…” (page xii). The evaluation also acknowledged that the program has been flexible and has helped the government to respond to health issue. It also expressed concern about Ministry’s governance and leadership in the SWAP. 22 Objective 1— Reducing infant, under-five, and maternal mortality and the proportion of malnourished children, while eliminating gender disparity and strengthening access to reproductive health services 4.2 The achievement of this objective is rated modest. This reflects good progress on improving infant, child, and maternal mortality, while strengthening reproductive health services. However, the Program had no impact on reducing child malnutrition. There is no significant gender disparity in achieving these objectives. The Program contributed to the improvement in access to reproductive health services. REDUCE INFANT AND CHILD MORTALITY 4.3 Infant and child mortality declined during the Program’s period and a significant part of this decline can be attributed to the Program’s activities. Bangladesh is one of the best performers in terms of improving child mortality in the world (UNICEF 2010) and is one of the few low income countries that will reach the fourth Millennium Development Goal (Lozano, Wang, et al. 2011). During the period covered by the Program, there was a reduction in infant mortality from around 52 to 43 per 1,000 live births (21 percent reduction) and child mortality from 65 to 53 per 1,000 live births (23 percent reduction). Figure 4 outlines the patterns of neo-natal, infant, and child mortality over the past two decades. The broad pattern is clear, with a constant decrease in mortality over the past decade, with the exception of 1999 to 2003 period; the period under the Program saw an increase in the reduction of early mortality and a return to previous trends. 4.4 The gap between urban and rural areas has decreased. The child mortality rate is estimated to be around 24 percent higher in rural areas than in urban areas; in 1991 the difference was around 37 percent. While regional disparities have decreased, there still remain important differences (World Bank 2010, Government of the People's Republic of Bangladesh 2010) 4.5 The Program contributed to this improvement as did the increase in living standards and improved availability of private health providers. The public health sector played a major role in promoting good health practices, through public information campaigns and as well as changes in training for medical personal. The Program supported efforts to mainstream an integrated approach to child health in medical curriculum as well as providing training to practicing health workers (Ministry of Health and Family Welfare 2010). One of the Program’s major initiatives was the expansion of the Integrated Management of Childhood Illness at both facility and community level.12 This methodology was introduced at the facility level and now operates in 343 upazilas and 40 district hospitals. At the community level, implementation was less successful due to the difficulties contracting NGOs, although coverage increased from 7 upazilas in 2009 to around 50 (Implemnetation Planning and Monitoring Division 2011). This has led to a gradual improvement in care for newborn 12 Integrated Management of Childhood Illness brings a case-based approach to addressing child’s health that attempts to bring together different elements of the health system, including immunization, the provision of drugs, and clinical management of diseases. It generally does not directly focus on the care of newborn health (Government of the People’s Republic of Bangladesh 2004). 23 infants, such as using sanitary blades to cut the umbilical cord. In 2007, 6 percent of births used a cord cutting kit and 82 percent used a boiled instrument; by 2011, this had increased to 14 percent and 84 percent respectively (National Institute of Population Research and Training, Mitra and Associates, and MEASURE DHS 2013). The cord cutting kits were largely provided by the public sector. Figure 4.1. Neo-natal, Infant, and Child Mortality Rate, per 1,000 live births Note: Neo-natal mortality rate refers to the mortality rate of children in the first month in life, infant mortality rate is the mortality rate during the first year of life, and child mortality rate is the mortality rate during the first five years of life. Source: National Institute of Population Research and Training (NIPORT), Mitra and Associates, and MEASURE DHS, 2013 and author’s calculations. 4.6 Children are at the greatest risk during the week after birth. The Demographic and Health Surveys show that the coverage of post-natal care for children within the first two days from medical providers increased from 13 percent in 2004 to 30 percent in 2011 (National Institute of Population Research and Training, Mitra and Associates, and MEASURE DHS 2013). Most of this increase was in the public sector, which was particular important for the marginal population. 4.7 The public health sector has played a central role in both procuring and administering vaccinations to children. Although the immunization program often suffered from weak middle management, it was generally successfully in distributing traditional vaccinations as well as introducing new vaccinations such as hepatitis B and Hib (Ministry of Health and Family Welfare 2010). The Demographic and Health Survey show that in 2011, 86 percent of children under the age of two have received four standard courses of vaccination13, an increase from 60 percent in 2000 and 73 percent in 2004 (National Institute of Population Research and Training, Mitra and Associates, and MEASURE DHS 2013). The Program had 13 This includes BCG (tuberculosis), DPT3, Polio3, and Measles. 24 a target of 85 percent of children fully immunized before their first birthday, compared to the reported coverage of 80 percent by the end of the Program. 4.8 Childhood diarrhea has traditionally been a major source of mortality in Bangladesh; the World Health Organization estimates that around 20 percent of child mortality is due to diarrhea (World Health Organization n.d.). In 2011, 4.6 percent of children had diarrhea in the previous two weeks. Around 18.5 percent of children from the poorest two quintiles seek care from formal providers, compared to 12 percent in 2004. Likewise, 82 percent of poor children receive oral rehydration packets, compared to 58 percent in 2004. These were typically procured through the Program although much of the supply was distributed through the private sector and NGOs. 4.9 Children’s health status is sensitive to the quality of the health sector and to the education level of caregivers, which accounts for the big difference in infant and child mortality between low and high income economies.14 While it is not possible to quantify the overall impact of the public sector on child mortality, the evidence suggests that without the support from the program, child mortality would have been higher. ENSURING INCREASED ACCESS TO REPRODUCTIVE HEALTH SERVICES 4.10 Under the Program, the access to and availability of reproductive health services increased. Improving the quality and availability of reproductive health services was one of the key priorities of the Program and received substantial attention. This included training throughout the country for skilled birth attendants, hospital staff, and nurses in a variety of topics ranging from the operation of incubators to emergency obstetrics procedures. While this training increased the capacity of the health sector, retention of trained personnel was a major issue (Ministry of Health and Family Welfare 2010). This training was complemented by capital investment, particularly for emergency obstetric facilities. 4.11 The Program also supported demand-side financing through the Maternal and Neonatal Health Voucher Scheme. This scheme, which was launched under the Program, provided vouchers to pregnant women entitling them for free antenatal services, delivery, and postpartum care. It also provided incentives for care providers and cash stipends for women to cover transportation costs. The scheme operated in 28 districts and covered an estimated 180,000 women per year (Ministry of Health and Family Welfare 2010).15 An evaluation of this scheme argues that the vouchers had an impact in raising the demand for institutional deliveries, when comparing participating upazilas with non-participating upazilas (HERA 2013). Additional evaluations show that vouchers did lead to an improvement in the quality of services without increasing the number of Caesarean section deliveries, as was feared. The vouchers disproportionally benefited poorer mothers, lowering inequality in areas where they are available (Ahmed and Khan 2011, Schmidt, et al. 2010). 14 Young children are at particular risk of diarrheal diseases, respiratory infections, and accidents. Appropriate maternal action and timely immunization play a major role in reducing the risk that children face (UNICEF 2010). 15 Four additional districts are covered by a parallel program outside of the SWAP. 25 CONTRIBUTE TO THE REDUCTION OF MATERNAL MORTALITY 4.12 Bangladesh saw a continued decline in maternal mortality and the Program contributed to this decline. Using data from the Maternal Mortality Survey, the maternal mortality rate has declined from 574 per 100,000 live births in 1990 to 194 in 2010 (Government of the People's Republic of Bangladesh 2012). Based on these trends, Bangladesh appears likely to reduce the maternal mortality rate to 149 deaths per 100,000 live births in 2015, which is slightly above the Millennium Development Goal target of 143 (Government of the People's Republic of Bangladesh 2010). Figure 4.2. Maternal Mortality Rate, per 100,000 Live Births Source: Government of the People’s Republic of Bangladesh, 2012 and author’s calculations. 4.13 Reducing maternal mortality has proven to be one of the most difficult Millennium Development Goals (World Bank 2009).16 Maternal mortality is particularly sensitive to the provision of ante-natal services, the identification of high-risk births, and access of formal birth facilities. This combination helps to ensure adequate care for both the mother and the newborn child. 4.14 Data from the Bangladesh Maternal Mortality and Health Care Surveys, carried out in 2001 and 2010, provide a detailed look at the causes of maternal mortality and what steps households have taken to reduce the risk and treat maternal health threats. Overall, 54 percent of the reduction is due to demographic changes— fewer women are at risk as the number of 16 It has also proven to be one of the more difficult indicators to measure accurately, due to the large sample size needed and the tendency to underestimate maternal mortality (United Nations, 2003). For example, estimates for the maternal mortality rate in Bangladesh in 1990 range from 724 per 100,000 live births (Hogan, et al., 2010) to 574 from the Maternal Mortality Survey and 478 from the Sample Vital Registration System (Government of the People's Republic of Bangladesh 2012). 26 children born decreases. Likewise women face a lower health risk as they delay the age at which they have their first child (NIPORT, Measure Evaluation, and ICDDR,B 2011). 4.15 An improved health sector response is responsible for the decline in two leading medical causes of maternal mortality. The remaining 46 percent is due to a real reduction in maternal mortality, largely due to medical interventions. Eclampsia and post-partum hemorrhaging account for about half of maternal mortality in the country. Eclampsia, which can be easily prevented, is often fatal if left untreated. Between 2001 and 2010, mortality due to eclampsia declined 23 percent. Hemorrhaging presents itself after the delivery. If care is delayed, the condition can be fatal. Mortality due to hemorrhaging declined by 58 percent during the same period (NIPORT, Measure Evaluation, and ICDDR,B 2011). 4.16 Timely antenatal care plays a major role in addressing eclampsia. Between 2005 and 2010, antenatal care from medically-trained providers has increased from around 46 percent to 56 percent. Overall, all antenatal care increased from 60 percent to 75 percent.17 Most of the non-medically trained providers (NGOs and public health workers) use protocols to indicate when to provide referrals to medical providers (NIPORT, Measure Evaluation, and ICDDR,B 2011).18 Most of these protocols were developed by the public sector and most of the referrals were made to public sector providers. 4.17 Having a birth in a formal setting is an important step to reduce the risk of hemorrhaging. Often times, the local health facility can treat hemorrhaging. Alternatively, a facility can quickly refer the woman to an appropriate facility. In the last decade, the use of public medical facilities for deliveries has increased from 5.8 percent to 10.0 percent of births. At the same time, the coverage of private attended medically-qualified births increased from 2.7 percent to 11.3 percent, with more than half of the increase coming from the 2001 to 2005 period, before the Program. Part of the growth in private delivery is likely to be due to information and referrals received from publically-provided antenatal care. 4.18 It is clear that the public sector, which was financed through the program, played a role in reducing preventable maternal mortality. While the growth in private delivery also contributed to the reduction in maternal mortality, it is not sufficient to explain the reduction. Based on the data presented here, it would be appropriate to assign at least 15 percent of the maternal mortality reduction to the public sector and the Program. This is the equivalent to around 1,200 to 1,300 lives per year.19 The actual impact is likely to be higher, public information campaigns and beneficiaries of public antenatal services may end up using private delivery facilities. 17 For women who receive non-medically trained antenatal care, about a third is from NGO providers and a third from the public sector. 18 Non-medical health providers refer to individuals that have medical training but do not have a formal medical degree. This typically refers to community health workers. 19 This is a conservative estimate that assumes that around a third of the reduction in maternal mortality (after demographic changes are removed) is due to increase in the coverage and efficiency of public health service. 27 REDUCE THE PROPORTION OF MALNOURISHED CHILDREN 4.19 While Bangladesh has made substantial progress with most health indicators, it still has high levels of malnutrition and there has been little advance in the last decade (World Bank, 2011). The Program did not contribute to nutritional outcomes for children in a meaningful way. The Program depended on a model that had proved unsuccessful in a previous project and it did not have either the scope or the coverage to make a difference in nutritional outcomes. 4.20 Stunting (low height for age) is considered a measure of long-run nutritional status since it captures the combined effect of cumulative nutrition and health shocks (Dasgupta 1993). Figure 6 presents the percentage of children under the age of five that are stunted and severely stunted. The overall trend shows a rapid decline in malnutrition in the 1990s followed by a much slower decline in the past decade. However there are several important considerations. First, the proportion of severely stunted children has continued to decline—in 1996, more than 50 percent of stunted children were severely stunted; the proportion is now down to 37 percent. Second, recent Demographic and Health Surveys show that there is no gender gap in stunting (National Institute of Population Research and Training, Mitra and Associates, and MEASURE DHS 2013, NIPORT, Mitra and Associates, and Macro International 2009). Third, at the same time, there is evidence to suggest that other forms of inequality in nutrition are decreasing. In the past decade, most of the gains in the reduction of severe malnutrition have come among children in the rural areas. Likewise, severe malnutrition has dropped for children of mothers with low levels of education. The overall picture that this presents is of rapid decrease in malnutrition in the “easy” population in the 1990s followed by an improvement in the nutritional status of the poorest children in the country in the 2000s. There is a similar trend for other indicators of child nutrition. Figure 4.3. Height for Age (stunting) of Children 60 Months and Younger Note: Following international practices, children more than two standard deviations below the mean are considered stunted and more than three standard deviations are severely stunted. The severely stunted children are included with the stunted children. Source: Mitra, Al-Sabir, Cross, and Jamil, 1997; NIPORT, Mitra and Associates, and ORC Macro, 2001; NIPORT, Mitra and Associates, and ORC Macro, 2005; NIPORT, Mitra and Associates, and Macro International, 2009; NIPORT, Mitra and Associates, and Measure DHS, 2012; and author’s calculations. 28 4.21 While it is possible that certain groups have benefited during the program period, there appears to be no systematic improvement for children. At the same time, it does appear that the nutritional status of adults has improved. What explains this disparity and what does this indicate about the effectiveness of the program? 4.22 Nutritional status is the product of numerous factors that are outside of the control of the health sector, such as the consumption of calories, protein, and nutrients. The health sector can influence nutrition by providing nutritional supplements, preventing diarrheal diseases, and providing information about good nutrition. In recent years, there has been a substantial increase in exclusive breastfeeding combined with a decrease in the prevalence of diarrhea in children (NIPORT, Mitra and Associates, and ORC Macro 2005, NIPORT, Mitra and Associates, and Measure DHS 2012). 4.23 The Program built its support for nutrition on two different Operational Plans: Micronutrition Supplementation Program and the National Nutrition Program. The Program supported a number of activities to distribute micronutrients. Among the most successful were the distribution of Vitamin A to around 98 percent of newborns and the iodization of 84 percent of salt. 4.24 The Program also built on the activities of the World Bank-financed National Nutrition Project, which closed in 2006 with a moderately unsatisfactory rating. This project financed the government’s contracting of NGOs to provide nutritional services to the poor population in some areas of the country, this included training to field staff, communication and social media, and provision of basic equipment. After the project closed, several of the participating NGOs were contracted under HNPSP. However, given the Program’s shortcoming in managing other types of contracting, it was difficult for the Program to supervise the nutrition-related activities properly. According to internal records, at least one major NGO decided not to join the program and at least one NGO was investigated for possible corruption. The Ministry eventually developed a strategy to mainstream nutrition in the health sector, which effectively ended the previous approach (Independent Review Team 2009). 4.25 The government’s implementation report outlined that while the Program carried out significant procurement to support nutrition activities, problems with contracting NGOs reduced the effectiveness of nutrition-focused activities, which included social mobilization and training for field staff. At the beginning of the Program, these NGOs were active in 110 upazilas and by the end of the Program, this had risen to 167 upazilas. Many of these contracts were interrupted for several months. (Ministry of Health and Family Welfare 2010). The final program evaluation, prepared by the Ministry of Planning confirmed problems with procurement of goods and contracting, as well as a lack technical staff and leadership (Implemnetation Planning and Monitoring Division 2011). 4.26 A final evaluation of the National Nutrition Program, covering activities financed by the previous World Bank project and HNPSP, showed that there were substantial differences in nutrition knowledge and practices in covered and non-covered areas. While there is likely to be some placement bias, the difference is large and is likely to reflect the impact of the 29 program. Having said that, these differences in knowledge and practices did not appear to have any impact on child nutrition outcomes (Mbuya and Ahsan 2012). It is likely that the low coverage (around 25 percent of upazilas), the limited scope of the activities (communication and training) as well as problems with contracts have led to limited impact. Likewise, while the distribution of micronutrients was beneficial it could not have had the desired impact on nutrition outcomes. HAS GENDER DISPARITY DECREASED UNDER THE PROGRAM? 4.27 There are no gender differences in infant and child mortality rates. Since at least 1993, there have been no major differences between key health outcomes among boys and girls. In 1993, boys appeared to be about 10 percent more likely to receive medical inputs, including a higher probability to receive the full course of vaccination or to receive proper treatment for diarrhea. However both boys and girls had similar under-five mortality rates, around 15 percent (Mitra, et al. 1994). By 2004 (the baseline for the Program), the under-five mortality drop to around 9.5 percent, with girls (9 percent) having a lower probability of early mortality than boys (10 percent). Likewise, there appear to be no significant difference between access to medical care (NIPORT, Mitra and Associates, and ORC Macro 2005). 4.28 This trend has continued throughout the Program. At the close of the Program, girls continued to have lower under-five mortality rates than boys (5 percent vs. 6 percent). There appear to be no significant difference between boys and girls with access to health inputs (NIPORT, Mitra and Associates, and Measure DHS 2012). By design, HNPSP attempted to be gender neutral in its approach to child health through the integrated management of childhood illness approach (World Bank 2005). Documents show that the Program included training activities for medical providers to address gender concerns in their practices (Ministry of Health and Family Welfare 2010). Objective 2—Lowering the fertility rate with a view towards achieving replacement level fertility by 2010 4.29 The achievement of this objective is rated high. This reflects a continued reduction in the fertility rates, building on already impressive gains. Although much of this decline is due to social changes and the work of NGOs, the public sector played an important role in providing contraceptives and information about family planning. 4.30 Bangladesh has seen a substantial reduction in its fertility rate and has reached its replacement fertility rate. In 1975, the total fertility rate was 6.3 birth per women (NIPORT, Mitra and Associates, and Measure DHS 2012) and the annual population growth rate was between 2.5 to 3 percent (World Bank 2012). As shown in figure 7, the total fertility rate had dropped to 3.4 births per woman in 1993 and has subsequently dropped to an estimated 2.3 birth per woman by 2011, which is essentially the replacement fertility rate. The rural population and poorest quintiles saw the largest decrease in fertility between 2004 and 2011. The evidence shows that around two-thirds of the decline in the fertility since 1993 happened after 2004, a period that corresponds roughly to the Program. Thus, while there is a long term decline in fertility, the trend appears to have accelerated when the Program was active. 30 Figure 4.4. The Number of Children Ever Born Source: Mitra, Al-Sabir, Cross, and Jamil, 1997; NIPORT, Mitra and Associates, and ORC Macro, 2001; NIPORT, Mitra and Associates, and ORC Macro, 2005; NIPORT, Mitra and Associates, and Macro International, 2009; NIPORT, Mitra and Associates, and Measure DHS, 2012; and author’s calculations. 4.31 To what degree did the Program contribute to the reduction in fertility? Fertility is the product of many decisions and factors—primarily the change in marriage patterns and the use of contraceptives. Evidence from Bangladesh suggests that during the period from 1975 to 1993, more than 80 percent of the decline in fertility was due to an increase in the use of contraceptives (Romo 2003). The trend was similar in the period from 1993 to 2007, with the bulk of the fertility decline due to changes in marital fertility and not due to changes in the age of marriage (Kamal 2012). While there are still women who feel that they do not have adequate access to family planning services, it is clear that the supply has kept up to the gradually increasing demand. 4.32 Figure 8 shows the proportion of women using any modern method of contraceptive as well as those using long-term modern contraceptives that require contact with the health sector (basically excluding pills and condoms). While there has been a general increase in the use of contraceptives, most of the growth is due to an increase in commercially-available contraceptives such as pills and condoms. There has been little growth in longer-term contraceptive methods (primarily sterilization, IUD, injectable methods, and implants) although there has been a major shift towards injectable methods (which are not permanent) away from sterilization. 31 Figure 4.5. Prevalence of Contraceptive Use Source: Mitra, Al-Sabir, Cross, and Jamil, 1997; NIPORT, Mitra and Associates, and ORC Macro, 2001; NIPORT, Mitra and Associates, and ORC Macro, 2005; NIPORT, Mitra and Associates, and Macro International, 2009; NIPORT, Mitra and Associates, and Measure DHS, 2012; and author’s calculations. 4.33 The public sector is still perceived as the main source of contraceptives in Bangladesh. More than 70 percent of adults who do not use contraceptives identify public sector as a source of contraceptives. In 2004, the public sector provided 57 percent of contraceptives to users, including most long term contraceptives. By 2011, the public sector provided 52 percent of contraceptives. While the public sector saw a decline in its share of contraceptives, it increased its share as the providers of permanent contraceptive methods (National Institute of Population Research and Training, Mitra and Associates, and MEASURE DHS 2013, NIPORT, Mitra and Associates, and ORC Macro 2005). 4.34 Under the Program, the government began the process of managing the public procurement of contraceptives. Previously this had been managed by different development partners that purchased contraceptives directly and then transferred to the government. This initially led to some delays in procurement. The United States provided technical support to strengthen the capacity of the Directorate General of Family Planning. The annual program review indicated that this did lead to an increase in the government’s capacity to carry out procurement. Even though private and NGOs providers are major providers of contraceptives, the government procures at least 55 percent of pills (National Institute of Population Research and Training, Mitra and Associates, and MEASURE DHS 2013). The Program also supported efforts to reach out to the isolated population, through satellite clinics and targeted outreach efforts (Ministry of Health and Family Welfare 2010). 4.35 While there were not any major reforms in population policy, the Program was able to meet the growing demand for contraceptives through its procurement and the distribution efforts. Fertility declined most rapidly among the poor, suggesting that the government’s efforts were successful since the public sector is the major source of contraceptives for poor and rural. While there was not much of an increase in the use of longer term contraceptives, 32 the Program did support this effort. The Program also played an important role in strengthening the government’s capacity to lead procurement. Objective 3—Reducing the burden of priority diseases 4.36 The achievement of this objective is rated substantial. Overall, the Program made a major contribution in controlling both tuberculosis and malaria. The Program had less of an impact on HIV/AIDS. 4.37 At the time of approval, malaria and tuberculosis were major public health challenges and Bangladesh was considered at risk from HIV/AIDS. The Program aimed to bring together different free-standing, disease-specific “vertical programs” under a united health system. Administratively this occurred as most of the vertical health programs were integrated in the Program and began to coordinate their activities with other Operational Plans. This included the various grants that were approved by the Global Fund for Tuberculosis, HIV/AIDS, and Malaria, which disbursed around US$270 million from 2005 to 2011. Despite this formal integration with the Program, the vertical programs continued to operate with some independence in many areas. TUBERCULOSIS 4.38 With its high population density, Bangladesh is susceptible to the rapid spread of TB. It has one of the highest burdens of tuberculosis (TB) in the world. Starting in 1980s, the country developed an influential and successful public-private partnership to provide the necessary treatment for TB patients (Ahmed, et al. 2013). The public health system plays an important role by tracking TB in the population. The private sector plays a role in the diagnosis of cases (usually using government laboratories) and starting the treatment. The public sector generally provides the drugs. It also detects and diagnoses cases in individuals. It also starts the treatment for many patients with follow-on care provided either by government or community health workers (Ahmed, et al. 2013). 4.39 Overall, the country has seen a gradual decline in the prevalence rate dropping from 503 (per 100,000) in 2001 to 469 in 2005 and 435 in 2011. Similarly, the mortality rate due to TB dropped from 57 (per 100,000) in 2001 to 50 in 2005 and 46 in 2011 (World Health Organization 2013). The decline in TB during the implementation period seems to be following the medium-term trend. Did the Program contribute to this trend or would it have happened without its support? 4.40 The Program supported various activities, including training for government and NGO staff, support to laboratories, and expansion of TB (DOTS) centers from 534 to 774 (Ministry of Health and Family Welfare 2010). The Global Fund, which officially operated within the SWAP arrangements, provided support to organize training and procure drugs (through the Ministry), which were distributed to government providers and NGOs (Global Fund 2008). 4.41 The ICR reports that the TB case detection rate increased from 46 percent in 2003 to 74 percent in 2011, at the time the Program ended. This exceeds the targets of 70 percent, 33 which was updated to 72 percent in 2009 (World Bank 2012). During this period, the number of laboratories that could detect TB nearly doubled, from 0.4 per 100,000 population in 2005 to 0.7 per 100,000 in 2012 (World Health Organization 2013). 4.42 The treatment success rate for TB increased from 85 percent in 2003 (around the time that the Program started) to 92 percent in 2011 (World Health Organization 2013). Although the increase in the treatment success rate during the Program (7 percentage points) was similar to the gains in the previous period (8 percentage points from 1994 to 2003), this understates the difficulty of increasing effectiveness from an already high level (World Health Organization 2013). The treatment of TB requires repeated and uninterrupted contact with the patient and the increase of 8 percent from a relatively high rate shows an important advance. It is likely to imply higher coverage in the poorer segments of the population. MALARIA 4.43 Malaria is endemic in Bangladesh and is one of the leading causes of illness and fever in children. Around 34 percent of the population is at some risk of malaria. Most malaria is concentrated in the East and Southeast of the country, with prevalence rates averaging 11 percent in the Chittagong Hill Tracts, where a large proportion of ethnic minorities live. Overall, the number of detected cases increased sharply in the 1990s, after DDT was banned in 1985, but returned to its original levels around the time the Program started (Islam, Bonovas and Nikolopoulos 2013, ICDDR,B 2012). 4.44 The period between 2006 and 2011 saw a significant reduction in malaria in the country. In 2006, there were an estimated 17.9 million people (11 percent of the population) at high risk of malaria, compared to 4.0 million people (3 percent of the population) in 2011. Likewise the total number of deaths from malaria declined from 6,600 (4.2 per 100,000) in 2006 to 1,300 (0.9 per 100,000) in 2011. Other malaria-related statistics show a similar decline (World Health Organization 2008, World Health Organization 2012). The long-term data show that the drop in confirmed cases and mortality were not part of a long-term trend and represents a secular change. At the same time, the percentage of household at high risk with treated bed nets increased from less than 5 percent in 2006 to around 55 percent in 2011 (World Health Organization 2012). While other sectors undoubtedly played a part in this improvement, clearly the health sector had a major impact. 4.45 There are many factors that can account for the reduction of malaria. These include the use of insecticide and bed nets as well as improvements in access to and the effectiveness of health services. It can also be impacted by climatic factors that change mosquito breeding patterns. The Prevention and Control of Malaria and Other Vector Borne Diseases Operational Plan provided support for training, communication, and equipment (Ministry of Health and Family Welfare 2010). In addition, the Global Fund provided financial support to procure long lasting insect nets (through the Ministry) that NGOs distributed in areas of high malaria risk. 34 HIV/AIDS 4.46 HIV/AIDS was first detected in Bangladesh in the 1990s. By 2003, while Bangladesh still had a low level of prevalence, the prevalence rate had already reached epidemic levels among certain vulnerable groups and there was concern that the disease would become epidemic within the general population (Government of the People’s Republic of Bangladesh 2004). Bangladesh’s geographic position as well as sexual practices contributed to this risk (Caldwell, et al. 1999). 4.47 Following the underlying principle that all external-financed projects should be subsumed under the SWAP, the Program planned to incorporate the World Bank-financed National HIV/AIDS Prevention Project (Government of the People’s Republic of Bangladesh 2004). The project was approved in 2000 and was eventually closed in 2007; by 2003, it was clear that the project was having serious implementation issues. The project used NGOs to carry out its activities. In the end, about half of the credit was cancelled. The project was given a final rating of moderately unsatisfactory by the World Bank’s ICR and unsatisfactory by IEG (World Bank 2008, Independent Evaluation Group 2008). According to the HNPSP ICR, the Program continued to support the approach developed by the HIV/AIDS prevention project. There is no data on whether this approach was more successful. 4.48 By the end of the Program, the number of cases of HIV/AIDS continues to rise. However, due to the very low baseline, the prevalence rate for HIV/AIDS remained similar to the level seen at the start of the Program. As argued by the ICR Review of the HIV/AIDS Prevention Project, this is likely to be due to factors outside of the control of the project (Independent Evaluation Group 2008). Objective 4— Initiate a System to Control Newer Health Threats and Protect Health Risks by Improving Emergency Services 4.49 The achievement of this objective is rated substantial. The Program introduced a focus on improving emergency services, so that the sector can better react to the growing number of injuries and to strengthen the sector’s capacity to respond to environmental crises, such as cyclones and floods (Government of the People’s Republic of Bangladesh 2004). It also provided timely support to reduce the risk of pandemic influenza. 4.50 The Program supported the development of an Emergency Health Response Strategy, which was operational by the end of the Program (Independent Review Team 2009). Since there is no Operational Plan for emergency services or injury prevention, this strategy could play an important role in bridging different areas across the sector to respond to this need. The Ministry also put in place standard operating procedures for disaster mitigation. This included training field staff in vulnerable areas as well as hospital staff on mass emergency management. In addition, the Program trained police, drivers, and other non-health personal in emergency management. There is concern that there are not sufficient institutional and human resources to implement these activities (World Bank 2012). 4.51 The Program supported the construction of four one-stop crisis centers in medical colleges to provide comprehensive services to women affected by violence. In addition, the 35 Program carried out training on gender violence to nurses and hospital staff. Similar training was carried out with field health staff. PANDEMIC INFLUENZA 4.52 The Program’s support for the management of pandemic influenza (H5N1 and H1N1) deserves special attention. The Program supported the establishment of several complementary surveillance systems as well as stockpiling anti-viral drugs. This was planned during project design and the emergence of H5N1 early in the Program (2005) provided further support to this effort. During the outbreak of the H1N1 pandemic influenza, in 2009, the surveillance system functioned reasonably well and was about to detect cases both from patients and travelers. The government provided anti-viral drugs to suspected patients as well as vaccinations when they became available. However, drugs appeared to be underutilized as risk messages for patient and training providers were less effective (Azziz-Baumgartner, et al. 2012). Objective 5— Improving the prevention and control of non-communicable diseases 4.53 The achievement of this objective is rated modest. The Program represents one of the first large-scale initiatives to address emerging health issues and non-communicable diseases (for example, obesity, diabetes, and cancer) in Bangladesh and it was successful in bringing the issue to the table. The Program also supported strengthening the national Institute of Public Health. However there is no clear indication of contributing to outcomes related to non-communicable diseases. NON-COMMUNICABLE DISEASES 4.54 The Program’s main contribution to addressing non-communicable diseases was increasing the health sector’s and public’s focus on new health risks. Although the population is at increasing risk from non-communicable diseases, the public health system largely focuses on traditional health problems (El-Saharty, et al. 2013). 4.55 The Program supported a number of activities to test and treat arsenic in drinking water. Contaminated drinking water was a major health concern in Bangladesh and improving the quality of water was a major focus for many development partners. The Program supported surveillance systems to identify victims as well as training for health workers. While these activities were carried out, it is not clear what the impact of these activities have been or how they were coordinated with other programs to address arsenic contamination. 4.56 In addition to addressing arsenicosis, the Program supported a variety of other activities in non-communicable diseases. This included support to develop an Occupational Safety and Health Strategy, support to the laboratories in the Institute of Public Health as well as surveys on the health status of the elderly. Progress on these activities was mixed and there is no clear measurement of their impact. 36 4.57 Although the Program did establish targets for non-communicable diseases, there is generally no baseline data and little information on progress towards targets. Starting in 2011, the Demographic and Health Survey began to monitor non-communicable health issues for the first time. 4.58 The Program also supported efforts to reduce tobacco use, including training of appropriate staff and providing messages to the media. The Program included a target of reducing adult smoking from 20 percent in 2004 to 15 percent in 2010 (Ministry of Health and Family Welfare 2010). By 2009, 27 percent of adults were smoking tobacco. There were smaller increases in the use of smokeless tobacco. 5. Efficiency Efficiency … asks whether the costs involved in achieving project objectives were reasonable in comparison with both the benefits and with recognized norms [or provides] “value for money” (OPCS 2011). 5.1 The Program’s efficiency is rated substantial. The Program had a positive rate of return and probably reduced government administrative costs. However it also suffered from low internal efficiency with resources underutilized. External Efficiency 5.2 During preparation, the World Bank’s economic analysis estimated that the Program would yield a net present value of US$3.0 billion (with a ten percent discount rate) and an internal rate of return of 51 percent, over a ten year period. (World Bank 2005, 103-104). This is built on the assumption on a decrease in both mortality and morbidity, leading to both direct benefits to individuals and households as well as higher gross domestic growth. This estimate appears to be high and assumes that all of the health gains are impacts of the Program. The ICR argues that these assumptions remained relevant at the end of the Program and that this is a good proxy for the end-of-project cost-benefit analysis. While many of the Program’s targets were met, this evaluation argues that only part of this benefit can be attributed to the Program. 5.3 The data presented in this evaluation allow for a crude and partial ex-post analysis of costs and benefits. Depending on the Program’s impact on child and maternal mortality, the project’s rate of return varies from 18 to 28 percent, with a net present value between US$0.6 billion and US$2.5 billion, at a five percent discount rate. This analysis is based on simple assumptions about future wage dynamics and does not include the impact on improving health on the economy. This estimate is not directly compatible to the estimate provided in the Project Appraisal Document, which includes an estimate for the reduction of morbidity.20 20 The benefits are estimated using the reduction of the child and maternal mortality that can be reasonably attributed to the Program. This is multiplied by the lost income due to mortality and 37 5.4 The World Bank’s Public Expenditure and Institutional Review for Bangladesh (World Bank 2010) argues that on a macroeconomic level, the country receives good value for its health expenditure compared to other countries in the South Asia Region. This is based on the relatively strong progress that Bangladesh has had with improving the health status of its population with a lower level of both public and private spending than other countries. Internal Efficiency 5.5 Despite the setbacks in some areas, such as nutrition and HIV/AIDS, most observers concluded that the Program was generally successful in bringing together different actors to coordinate and jointly plan. The Program was divided into 38 Operational Plans that were “silo-like” according to the 2008 Mid-Term Review (World Bank 2012). This lack of communication across units reduced collaboration and the possibility of synergies. However, this represented a reduction from the number of separate project offices that operated before the first Program as well as a reduction in the Operations Plans in the first Program. The Program also created a common financial management and procurement framework that had not previously existed. Although it is not possible to estimate the changes in the administrative costs for the government, it is likely to be substantial given the large number of programs. 5.6 A report on health infrastructure shows that while many facilities have issues, they are generally maintained and almost always functioning. Issues include poor hygienic conditions (although generally not in surgical theaters or other high priority areas), lack of basic amenities (lights, clocks, etc.), and the like. While this definitely affects the quality of service, it is clear that the facilities are being used as intended. This is an important improvement over the previous Program, where development partners (through the World Bank) financed a large number of community health clinics that were essentially abandoned and unused (Independent Evaluation Group 2006). In contrast, the Health, Nutrition, and Population Sector Program support the rehabilitation of these facilities, providing value to money for the Program. 5.7 The World Bank commissioned a survey on the utilization of medical equipment in 2012, focusing on investments carried out in the program. This survey showed that around 55 percent of medical equipment purchased under the Program is being used effectively. The survey also shows that 32 percent of equipment is not installed or is not being used and another 13 percent of equipment is broken or inoperative. There is a tendency to use and maintain higher value equipment, and when weighed by value, around 76 percent of equipment is being properly used. While this is an improvement compared to what was seen in the previous survey, which was done in 2008, it still represents an important loss of efficiency as resources could have been better deployed. summed over an expected lifespan of 75 years. The costs are based on the total expenditures that can be associated with the Program. 38 6. Ratings Outcome 6.1 Overall, the program outcome is rated moderately satisfactory. 6.2 As a SWAP that was built on the government’s health strategy, the Program had a high relevance of objective. However, the design was overly ambitious and the Program did not have the elements to address all of its objectives. The Program’s performance was mixed—it made a substantial or better contribution to reducing child and maternal mortality, lowering the fertility, and reducing the burden of priority disease. It introduced a focus on emerging diseases and emergency services. It was unsuccessful in improving child nutrition. The Program was substantially efficient, with some improvement in the transaction costs along with evidence that some investment was not fully utilized. Risk to Development Outcome 6.3 The risk to development outcome is rated moderate. 6.4 The Health, Nutrition, and Population Program closed on a positive note and the government is now fully committed to the sector-wide approach in health. A third Program, the Health Sector Development Program, was approved in May, 2011 and is now effective. Unlike the HNPSP, there was widespread support going into preparation. The government continues to take a central role in procurement (for example, contraceptives) and this appears unlikely to change. 6.5 The Program was not successful in introducing many systemic reforms. The Program provided some support to reestablish community health clinics. While these are now functioning, they have been long been an area of political controversy and there remains a real risk that a future government could substantially change the role of these facilities. A second innovation that the Program supported was the maternal vouchers. While these have appeared to be successful, their continued impact will depend on the supply side offering services of value to pregnant women. Both of these initiatives seem likely to continue. 6.6 The reports on infrastructure and on medical equipment show that maintenance remains an issue. While infrastructure is generally used as intended, it is not always well maintained. Medical equipment is not always maintained or used as intended, putting the investment at risk. Bank Performance 6.7 Overall Bank performance is rated as moderately satisfactory. Table 6.1. Achievement of Objectives Outcome Rating Remarks Relevance 39 Relevance of Objective High The program’s objectives were and remain well-aligned with the country’s development strategy and the Bank’s country strategy. Relevance of Design Modest The Program included the entire health sector. However there was no clear causal chain for many objectives. Efficacy Reducing infant, under-five, and Modest The Program’s activities contributed to the maternal mortality and the reduction of both child and maternal proportion of malnourished children, mortality. The Program little impact on while eliminating gender disparity improving the nutritional status of children. and improving access to There was no gender disparity and access to reproductive health. reproductive health improved. Lowering the fertility rate High The Program continued to support the country’s population program and supported efforts to strengthen the government’s procurement and management capacity Reducing the burden of priority Substantial The Program made important contributions in diseases controlling tuberculosis and malaria. It had little impact on HIV/AIDS. Initiate a system to control newer Substantial The Program developed an emergency health threats and protect health response strategy and carried out training. It risks by improving emergency was also effective in responding to the threat services of pandemic influenza. Improving the prevention of control Modest The Program brought attention to these of non-communicable diseases emerging issues, with some new investment. Efficiency Substantial The Program had a positive rate of return. While the Program probably reduced transaction costs, there is evidence that some of the investment was not fully utilized. Project Outcome Moderately Satisfactory Note: consolidated rating from the PPAR. 6.8 The quality of entry suffered due to the tense relations between development partners and the government and the limited time to prepare the Program. After the Program was approved, the Bank provided an experienced team. Supervision was proactive and flexible and provided value added to the government and to partners. QUALITY AT ENTRY Quality at entry refers to the extent that the Bank identified, facilitated preparation of, and appraised the operation such that it was most likely to achieve planned development objectives outcomes and was consistent with the Bank’s fiduciary role (OPCSGuidanceNote). 40 6.9 The quality of entry is rated as moderately unsatisfactory. 6.10 Designing the Program was a complex process as it involved developing both an investment project, which was consistent with World Bank requirements, and a larger program included the government and development partners. The design phase was further complicated by the problems that were inherited from the previous program (the Health and Population Program, 1998 to 2005). In addition to the tense relations between development partners and the government, the preparation was under pressure to develop a program quickly and to continue to provide financing to the health sector. IMPLEMENTATION ARRANGEMENTS 6.11 The Program was designed to administer pooled funds as well as to serve as an overarching instrument to coordinate partners and the government. One tension that was felt during implementation was the degree of independence that the Program would have from the World Bank. In other words, was the Program a Bank program? In the first Program, the Partners tried to distance themselves from the World Bank. During the design phase, the team tried to make it clear that HNPSP was a government program and to clarify the Bank’s supervisorial and fiduciary roles. The Partners initially provided their support in a semi- independent office, to emphasize the Program’s independence from the World Bank. 6.12 The first Program mixed the health and population sectors together despite the fact that they were operated by different General Directorates within the Ministry. HNPSP was even more ambitious, incorporating nutrition as well as vertical programs. While this was logical from a design point of view, as the Program aimed to encompass the sector to develop the Ministry as a steward, there did not appear to be much consideration about the implications of the incorporation of two World Bank projects into the Program. While the Program did have the mechanism to incorporate the projects (through contracting), little attention seemed to be placed on how the incorporation would work in practice. FIDUCIARY ARRANGEMENTS 6.13 The Program had a number of important fiduciary shortcomings that led to some delay in implementation and a series of low financial management ratings. The predecessor project, Health and Population, closed in 2005 with a number of unresolved audit issues. Since the Health and Population Sector Program’s last audit had not been finalized before the time of project negotiations, it was agreed that the government would address any audit observations before the Program would disburse. According to the Project Appraisal Document, the audit issues, this was driven by delays in receiving information from the local level:21 The number of unresolved audit objections continues to be a matter of concern. This is due in part to lack (sic) of understanding by the [Foreign Aided Project Audit Directorate] of the nature of the accounting process for [the Sector Wide Approach]. 21 This is also mentioned in the PPAR for the Health and Population Program (Independent Evaluation Group 2006, 45). 41 Respond to audit objections by spending offices at district and upazalia level to be extremely time consuming. The non-availability of supporting documents incurred directly by [development partners] remained another major concern… (World Bank 2005, 70).22 6.14 While it is common to delay effectiveness until all audit issues are solved, the Bank team did identify this issue during loan negotiations. This allowed the team to take precautions and allow the Program to start disbursing once the audit observations were resolved. 6.15 The financial management team identified 13 fiduciary risks, of which nine were rated high (World Bank 2005, 87). Mitigation factors were also identified and these were appropriate given the nature of challenge. Most of the risks that were identified were concerned with the transparency and the quality of the process. No specific mention was made of the risk of an unresolved audit delaying disbursement even though this was discussed in the loan negotiations. None of the risks suggest that the government would be incapable of carrying basic procurement. Despite the discussion in the Financial Management Annex of the PAD, the problems observed in the previous Program were not cited as a lesson learned in the project preparation (World Bank 2011, 8). 6.16 The Financial Management Improvement Plan is outlined in the PAD (World Bank 2005, 81). This plan included the preparation of new materials and systems, along with the training of staff. The plan had interventions at the Ministry of Health and Family Welfare level and also at the level of institutions. A number of steps are included to improve collaboration with development partners in the area of financial management, including the adoption of a common reporting framework and an end, in most cases, of development partner-specific audits (World Bank 2005, 79). This is also included in the Audit Annex of the Agreed Minutes of Negotiation, which were prepared in February, 2005. A number of other steps were incorporated in the financial management plan that aimed to improve the quality and efficiency of the audit. 6.17 On the procurement side, in Annex 8 of the PAD, the Bank team outlines the approach to be taken with procurement as well as providing a detailed outline of the procurement plan for international procured goods. The procurement team carried out a detailed review of the legal framework for procurement in the country as well as analysis of the problems that the project was likely to face (World Bank 2005, 85). As the part of the process, the procurement team reviewed the most recent Public Procurement Regulations (Government of the People's Republic of Bangladesh 2004). Subsequently, the Bank adopted many of the specific terms of the regulations including adjusting the time required to prepare a bid and the process of bid opening. After discussions with the Bank, the government agreed to allow greater flexibility in the receipt of bid documents. Both parties reached an agreement on the role of state-owned enterprises in the procurement process. 22 This is cited in the Lessons Learned section of the Financial Management Annex. 42 TECHNICAL PREPARATION AND LESSONS LEARNED 6.18 The PAD included a well-developed section of economic analysis in Annex 9 (World Bank 2005). In addition to an analysis of the costs and benefits of the Program, this included an analysis of the justification for public intervention, a discussion of the distribution of benefits from the Program and the health sector, and a review of public spending on health. The cost-benefit analysis was well prepared, based on optimistic assumptions from the Strategic Investment Plan, with monetary benefits. 6.19 The PAD reviewed the lessons learned from previous operations. The formal review of lessons (World Bank 2005, 8) was perfunctory and focused on the importance of understanding the difference between development partners and the government and the importance of properly identifying the role of each. The lessons also focused on the importance of political economy issues. These issues were also raised in the Implementation and Completion Report of the previous project as major shortcomings (World Bank 2005). Although not formally mentioned in the lessons learned section, many other lessons were included throughout the program design. These include extensive review and redesign of the financial management system, the introduction of a formal partnership agreement (with a code of conduct), and better technical development of components. 6.20 The Bank ensured that the government had resources, through an existing trust fund, to prepare the health sector strategy (the HNP Strategic Investment Plan) which is central to developing a SWAP. The strategy provided an outline of the government’s priorities as well as an instrument to discuss the government concerns with the previous Program. In addition to the strategy’s technical role, it was an important signal that the government would take a leading role and would be able to express its views to development partners. QUALITY OF SUPERVISION 6.21 The quality of supervision is rated satisfactory. 6.22 For the World Bank, supervision of the Program was a major challenge. The Program’s design was complex and ambitious, covering most of the health sector and incorporating both government regulation and World Bank operational frameworks. In addition to its own resources, the World Bank had to supervise the resources from several other development partners, essentially making it the leader of Program’s supervision. Finally, the World Bank played a major role in both carrying out its own technical work and promoting technical work of the government and other partners. 6.23 The World Bank had a strong team based both in Washington and in Dhaka. The World Bank had a core technical staff that ranged from two to four international level professionals, mostly based in Dhaka. This group helped to maintained continuity as task team leaders were changed. Of the four team leaders, two were based in Dhaka. In addition to the core technical team, the Program was supported by a large number of staff focusing on fiduciary issues and dedicated fulltime to the Program. These staff members were largely consultants with close support from World Bank staff. As previously mentioned, the World 43 Bank had access to significant resources to supervise the Program as well as to support technical activities. SUPERVISION OF THE OPERATION 6.24 The Program’s design required that the World Bank remain engaged with the government at many levels. This included a careful review of each Operational Plan, the respective procurement plans, and the different stages of the procurements. Many procurement documents had to be revised several times and passed through multiple reviews and no objections. 6.25 World Bank staff who worked on the Program confirmed that a significant amount of their time was dedicated to procurement. In general, the World Bank staff members were diligent in carrying out the review of procurement. For more complicated procurement, this often required contracting specialists to support the process. The World Bank investigated complaints quickly and suspended several contracts, including NGOs supporting both nutrition and HIV/AIDs activities. It also investigated case of adulterated drugs and reported problems with civil works. 6.26 The World Bank team was also proactive with financial management issues. Although the government was prompt with audit reports and financial management reports, there were often observations that needed to be addressed. The World Bank was generally proactive in providing support to help the government address this issue and make the necessary corrections. 6.27 A large part of the Program was financed by development partners through trust funds that were supervised by the World Bank. Although these resources were not formally pooled with the concessionary credit, the World Bank generally managed the two funds in a similar fashion. For example, the Bank stopped disbursing from its own credit to give the trust fund a chance to “catch up” in the middle of implementation. Several development partners provided additional resources to the trust fund during the implementation period. 6.28 One important feature of the Bank’s support to the Program was the use of independent reviews to supplement the supervision. A key part of the Annual Program Review was the preparatory work done by the “Independent Review Team” (the composition of which changed from year to year), hired through a trust fund. These reviews played an important input into the program and facilitated discussion between the development partners and the government on difficult topics, including political economy issues. 6.29 Through the Annual Reviews and internal Implementation Status Reports, the Bank focused on the development objective included in its PAD. As this was a relatively broad set of objectives, focused on improving health service delivery, there is little evidence that influenced operations on the ground. Most of the internal reporting focused on the results framework, which tended to cover higher level objectives as well as analyzing the improving health status of the population. 6.30 In addition to the annual reviews, the World Bank financed a number of specialized reviews of the different aspects of the operation to provide feedback to both the Bank and the 44 government on procurement issues. This included medical equipment surveys (in 2008 and 2012), a review of civil works (in 2011), an assessment of capacity building activities (in 2009), two health facilities surveys (in 2009 and 2011), an assessment of pharmaceutical procurement (in 2007), and a review of immunization effectiveness (in 2008). All of these studies produced high quality reports that looked into detail on utilization patterns and the perception of staff and users. For example, the medical equipment surveys reviewed how medical equipment is being used in the field while the civil works review evaluated the physical conditions of Ministry facilities. Some of these reports were made public (including one as a World Bank publication). They helped to fill the gap while the government was strengthening its procurement monitoring systems. COORDINATION 6.31 In Bangladesh, development partners were organized in the Health Sector Consortium, which was a branch of the larger Donor Consortium. The Consortium’s goal was to discuss views and coordinate actions in the health sector. The Consortium was led by a Chair and by convention, the World Bank never served as either the chair or the deputy chair. The Chair was formally responsible for dialogue with the government. 6.32 In practice, the Consortium asked the World Bank to lead many aspects of the Program without having formal authority. While the Bank did not have a leadership role, it was generally expected to lead in the implementation of most initiatives. In interviews, the Bank was described as a “lightning rod” that took some of the pressure from other organizations and was often in the hot seat. Borrower Performance Borrower performance is defined as the extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes (OPCS 2011). 6.33 Overall borrower performance is rated moderately satisfactory. 6.34 Borrower performance reflects both the actions of both the central government as well as the implementing agencies, in this case the Ministry of Health and Family Welfare. GOVERNMENT PERFORMANCE 6.35 The government performance is rated moderately satisfactory. 6.36 As the Program was organized as a SWAP, the government was the largest provider of resources and these were generally provided in a timely and predictable fashion. This included support for both the development budget, which was co-financed by the Program, and revenue budget, which was technically outside of the SWAP but played an important role in financing the sector. 45 6.37 As became clear that the Program was in “intensive care,” the government began to monitor the Program’s progress closely at the highest level. This led to greater stability in the Ministry and reduced (but did not eliminate) the turnover in the line director positions (Implemnetation Planning and Monitoring Division 2011). 6.38 From 2004 to 2011, there were four governments (including two caretaker governments), which complicated the implementation of the Program. The use of contracting for primary health services was never implemented as the government sent mixed signals about its intention. While this was manifested in the procurement delays at the Ministry, the government could have played a more proactive role in promoting these initiatives or indicated that they were no longer feasible. 6.39 By the end of the Program, there was a much greater sense of government ownership. This became clear as the third Program was being developed, which developed without the same amount of acrimony that went into the HNPSP. IMPLEMENTING AGENCY PERFORMANCE 6.40 Implementing agency (the Ministry of Health and Family Welfare) performance is rated moderately unsatisfactory. 6.41 The first few years of the Program were plagued by financial management and procurement issues, manifested by a large number of audit observations and delays in carrying out key procurement activities. As a result, the Ministry was slow to set up important organizations and even after they were established, development partners expressed concerns about their quality. The rotation of key staff, particularly the line directors, was constantly mentioned in annual reviews and final reports. Perhaps more important was the concern about the lack of training and necessary skill for line directors to carry out the functions. Although this gradually improved over time, turnover of middle management remained a constraint issue throughout the Program (Implemnetation Planning and Monitoring Division 2011). 6.42 Although the Program provided flexibility in terms of financing, the Ministry’s budget process was complicated and often inflexible. Resources could not easily be transferred among plans and most of the plans were only updated once, during the mid-term review. As a result, important initiatives such as the rehabilitation of community clinics were officially done outside of the Program despite their obvious synergies. 6.43 As the Program evolved, the Ministry improved its capacity in a number of areas. While this is typical learning curve for any operation, it appeared that little capacity was developed in the first Program and that the Ministry was basically “reinventing the wheel” with HNPSP. The Ministry supported efforts to expand the maternal voucher scheme as well as to expand the Integrated Management of Childhood Illness approach, which were both seen as successes. Monitoring and Evaluation 6.44 The quality of monitoring and evaluation is rated as modest. 46 6.45 The Program had a complicated MandE system that included multiple frameworks and a large number of indicators. Many indicators did not have baselines or targets and some were never monitored in practice. While the MandE system did generate a large amount of data, there is little reason to believe that these data were used for decision making or “course corrections.” MEASURING RESULTS ON THE GROUND: THE DESIGN OF MONITORING AND EVALUATION 6.46 The Program had two different MandE frameworks, known as the results framework and the logical framework (Martinez 2008). Each one had a large set of indicators that were to be tracked during the life of the Program. In principle, both were driven from the government’s Strategic Investment Plan (Government of the People’s Republic of Bangladesh 2004) although each had a different format. In addition, each of the Operational Plans was to develop its own set of indicators covering inputs, activities, and outcomes. Neither the PAD nor the Strategic Investment Plan provided details of how data were to be collected and analyzed. 6.47 The results framework, which is described in Annex 3A of the PAD and is included in Table C.1 (Annex C) in this PPAR, was designed to be used by the World Bank. It consisted of two outcome indicators and eight intermediate results that were divided among the three components. The World Bank chose indicators that were largely under control of the Ministry and the Program (World Bank 2005, 5). These indicators were collectively known as the priority indicators or the key performance indicators. The outcome indicators respond to the objective outlined on pages 4 and 5 of the PAD on improving the “…availability and utilization of…HNP services” and measured the primary health service utilization rate by the poorest two quintiles as well as spending on health in the poorest upazilas. While these are important outcomes, they do not directly address the objectives in the PAD. In particular, measuring the total amount of spending in the poorest upazilas does not capture the quality or efficiency of the spending or the role of non-government health providers in these areas. 6.48 In addition, the Program had its own results monitoring matrix (the logical framework) with approximately 60 indicators. Part of this matrix is included in Table C.2 (Annex C). The framework mixed both final outcomes (for example, the maternal mortality rate) with outcomes (for example, districts with disease surveillance reports). Many of the quantitative variables did not have baselines at the time of preparation and the targets were often expressed in vague terms. The framework also included several qualitative targets, relating the adoption of strategies and awareness campaigns. There was a substantial amount of overlap between the two monitoring frameworks and many indicators appeared in both. 6.49 Many of the indicators were not classified at the right level. For example, both the maternal mortality rate and a reduction in the number of public health providers without drug stock outs are both reported as “results.” The logical framework did not have a clear division between monitoring the sector and monitoring the Program (Independent Review Team 2006). 47 IMPLEMENTATION OF THE MONITORING AND EVALUATION FRAMEWORK 6.50 The shortcomings with the design of the MandE framework impeded its successful implementation. The first annual review (Independent Review Team 2006) indicated that Many…stakeholders recognized…the [Program’s] monitoring is weak and…tended to link such weakness to limitation of the [Program’s] results framework. As the Program advanced, the monitoring and evaluation system evolved both through formal changes, including restructuring of the Bank’s Program document, and as partners and the government learned which indicators could be collected and could not be collected. 6.51 By the end of the Program, the monitoring and evaluation framework on the ground looked quite different from the ones developed in the Program documents. The World Bank formally changed one of the high level indicators in 2009 in its results framework, as a result of the mid-term review. Previously, the indicator focused on spending targeted to the poorest districts. This was changed to the percentage of expenditure executed at the upazila level. The World Bank also modified several of the baseline and targets for other indicators. 6.52 The Annual Program Review was a comprehensive stock taking exercise that looked at available data and incorporated them into the results framework. Most of the Reviews were carried out with independent consultants to provide an outside perspective of the Program. The Review typically provided a combination of sharp analysis and recommendation. 6.53 Other partners and the government used different sets of indicators that are reflected in the annual program review and the end-of-project report. In practice, not all indicators could be tracked and many were informally dropped. While the final Annual Program Implementation Report in 2010 included many indicators from the original Program results matrix, it also included a large number of indicators taken from the operational plans. These indicators focus on the provision of inputs and activities as well as plan-specific outcomes. UTILIZATION OF MONITORING AND EVALUATION 6.54 There is no evidence that the Program took advantage of the myriad of indicators and monitoring framework to make strategic decisions. Most of the decision making was done following the Strategic Investment Plan and the various Operational Plans. Result-based financing was included in the original design (classified as disbursement category 1) and was to be released on the basis of the government reaching negotiated indicators. However, in practice this system did not take advantage of the indicators as resources could be easily transferred out of category 1 and the targets could be easily modified. 6.55 Although the Program did not make much use of the formal MandE system, the Program did make a significant contribution to strengthen the Ministry’s ability to collect health data. The annual reviews contributed to this effort and fed into the government’s monitoring system for a variety of purposes. The annual reviews’ primary contribution seemed to be at the macro level, by sending a signal to the government and partners on the overall “health” of the Program rather than being used for course corrections. Likewise, the 48 Program did invest in health information systems. The World Bank and other partners carried out a number of studies that also contribute to the database of information in the Bangladesh health sector. 6.56 During implementation, the Program tracked the progress on the indicators for different Operational Plan. These indicators along with program-specific measures, such as procurement data and disbursement levels, were an important part of the discussions in the annual reviews as well as in the aide memoires. From interviews and a review of documents, it appears that these were used to measure Program progress, although many were outside of the formal results and logical frameworks. 6.57 A number of surveys and reports (such as the one that focused on the impact of procurement) contributed significantly to monitoring and evaluation of the health sector. These studies were outside the formal MandE structure but contributed both to the annual reviews and to understanding of the sector. 6.58 In summary, data were used more for operational decisions than for strategic planning. While the Program did make use of data generated by the health sector to monitor progress and to make decisions, it did not make of use of any of the formal MandE frameworks. Instead, decisions makers relied more on data from operational plans and from technical studies. 7. Conclusions and Lessons 7.1 Although the Health, Nutrition, and Population Sector Program did not achieve all of its ambitious goals, it was largely successful and made a number of important contributions to the health sector. The Program remains relevant and it is not surprising that the Health Sector Development Program is built on the successful elements of HNPSP. In many ways, the Program was too ambitious and attempted to cover all aspects of the health sector, with a weak results chain. Despite attempts to introduce structural reforms, the SWAP’s underlying approach was to spend more money to confront a challenge and to try to increase the efficiency of the existing public health system. 7.2 The Program “paid the price” as the follow-up operation to the previous Program. However it has clearly done much to rebuild the idea of a SWAP. The first Program was largely seen as a failure that ended in acrimony between development partners and the government. The failure in the previous program led many to question the value of investing in health in Bangladesh and the use of a SWAP to work with the government. While there remain concerns and disagreements, relations have improved significantly and it is now possible to have a meaningful dialogue. 7.3 Both the preparation and the implementation of the Program provide important lessons and recommendations for the World Bank both in Bangladesh and globally.  A complex challenges, such as child nutrition, is beyond the scope of the health sector by itself and requires a multi-sectoral approach. While there was a general 49 understanding of the importance and magnitude of malnutrition, interviews suggest that it was essentially treated as a “side issue” with little high-level attention. The Program’s design assumed that the Program’s intervention would have a major impact on child nutrition and assigned ambitious targets. In practice, little happened and the Program had little or no impact on nutrition outcomes. In order to engender support from different stakeholders, it is important to marshal a combination of quantitative and qualitative research on successful interventions as well stakeholder analysis. This was clearly missing in the Program’s support for nutrition and may also be an issue for other areas of health, such as non-communicable diseases and child health. An organization like the World Bank can play a role in convening development partners and to ensure that the evidence is shared across different sectors, in much the same way that was done with HIV/AIDS and the “Three Ones” approach.  Not having a clear development objective can impact negatively on implementation and results. Often times, the design of the development outcome and the results framework are not given much priority. In the case of HNPSP, there appear to be at least five sets of program objectives from different official documents. In practice, many policymakers simply followed country-wide health statistics and used these to judge the Program’s impact. Without the guidance provided by an objective, the Program can easily slip into preserving the status quo.  The lack of intervention-level evaluations limits the Program’s flexibility and responsiveness. This evaluation shows the difficulty of evaluating a large Program such as HNPSP. While monitoring and evaluation is important for accountability, it can also help to understand what works well in the Program and what needs to be modified. The maternal health vouchers were separately evaluated and the results of this evaluation have been useful both internally to justify an expansion of the Program and externally to share globally. Likewise the detailed evaluations on specific issues, such as the equipment usage or the state of infrastructure, were used to improve performance within the sector. There were many aspects of the Program that did not have strong evaluations. For example, the Program encouraged the ultimately unsuccessful contracting of NGOs and increasing hospital autonomy. There was very little evidence base on why these interventions might work and little attempt to do demonstration pilots. Likewise, there has been no attempt to quantify the health impact of NGOs in the health sector. The NGO sector in Bangladesh has been quite successful in using evaluations and operations research to refine their activities (Chowdhury, et al. 2013) and this may serve as a lesson for the SWAP.  Having a well-designed results framework is important for planning and monitoring. The Program had a complicated and often contradictory results framework, using a large number of indicators. In addition, different operational plans collected specific data that was not necessarily captured in the official results frameworks. The Program also had several sets of objectives, drawn from different partner documents as well as the Investment Plan. Without a better roadmap of what the Program wants to accomplish (an objective) and how it plans to accomplish it (a 50 theory of change), it is difficult to generate much political support for the Program or encourage stakeholders to take necessary risks to achieve the objectives.  Without a structure for partners and the government to design, debate, and evaluate new policy, it seems unlikely that there will be structural reform in the future. One longstanding challenge in supporting the health sector is the difficulty in carrying out significant structural reforms, such as granting autonomy to hospitals. The SWAP did include some ambitious reforms, but most were never implemented. After some attempt to support health sector reform, the program largely focused on increasing financing and improving performance and service delivery. Establishing a reform infrastructure requires building trust and respect on both sides, which is a long term process. It also requires development partners to strengthen their understanding of political economy and to develop strategies to gradually expand their dialogue. At the same time, the government needs to identify its priorities and openly discuss its political limitations. This matches the observation of IEG’s evaluation of the health sector that noted that the failure to understand political economy considerably contributes to the probability of failure (Independent Evaluation Group 2009). At the same time, the government needs to identify its priorities and openly discuss its political limitations. 51 References Ahmed M. 2011. The SWAp in Bangladesh Primary Education. Sussex, United Kingdom: CREATE PATHWAS TO ACCESS. _____. 2011. The Sector-wide Approach in Bangladesh Primary Education: A Critical View. Sussex, United Kingdom: CREATE PATHWAS TO ACCESS. Ahmed, S.M., Evans, T.G., Standing H., and S. Mahmud. 2013. “Harnessing pluralism for better health in Bangladesh.” The Lancet 382(9906): 1746 – 1755. Ahmed, S.M., and Khan, M.M. 2011. A maternal health voucher scheme. Health Policy and Planning 26(1): 25-32. Alexander, N. 2010. The World Bank Reboots: Sweeping Investment Lending Reforms in the Works. Washington, DC: Heinrich Boell Foundation North America. Azim, T., Khan, S., Haseen, F., Huq, N., and Henning, L. 2008. HIV and AIDS in Bangladesh. Journal of Health, Population, and Nutrition 26(3): 311–324. Azziz-Baumgartner, E., Rahman, E., Rahman, M., Mamun, A., Haider, M., Zaman, R., et al. (2012). Early Detection of Pandemic (H1N1) 2009, Bangladesh. Emerging Infectous Disease 18(1): 146–149. Bangladesh Bureau of Statistics. 2011. Report on Sample Vital Registration System-2010. Dhaka, Bangladesh: Ministry of Planning. ______. Health Watch. 2012. Bangladesh Health Watch 2011: Moving Towards Universal Health Coverage. Dhaka, Bangladesh: BRAC. Baqui, A., Ahmed, S., El Arifeen, S., Darnstadt, G., Rosecrans, A., Mannan, I., et al. 2009. Effect of Timing of Postnatal care home visit on neonatal mortality in Bangladesh: a observational cohort study. BMJ. Baqui, A., El Arifeen, S., Darnstadt, G., Ahmed, S., Seraji, H., Rosecrans, A., et al. (2008). Differntials in neonatal mortality in two adjacent rurals of Bangladesh: Lessons for neonatal health interventions. Global Public Health 3(4):366-382. Barro, R. 2013. Health and Economic Growth. Annals of Economics and Finance 14-2(A): 305–342. Barro, R. and Lee, J.-W. 2010. A New Data Set of Educational Attainment in the World, 1950-2010. Cambridge, MA: NBER Working Paper #15902. Buse K., and Gwin C. 1998a. The World Bank and global cooperation in health: the case of Bangladesh. The Lancet 351(9103): 665 – 669. Caldwell, B., Pieris, I., Barkat-e-Khuda, Caldwell, J., and Caldwell, P. 1999. Sexual regimes and sexual networking: the risk of an HIV/AIDS epidemic in Bangladesh. Social Science and Medicine 48:1103–1116. Chowdhury, A., Bhuiya, A., Chowdhury, M. R., Hussain, Z., and Chen, L. 2013. The Bangladesh paradox: exceptional health acheivement despite economic poverty. The Lancet 382:1734-45. Dasgupta, P. 1993. An Inquiry into Well-Being and Destitution. Oxford, UK: Claredon Press. Department for International Development. 2012. Project Completion Report-- Health, Nutrition and Population Sector Programme (HNPSP). London, the United Kingdom: DFID. Donabedian, A. 2005. Evaluating the Quality of Medical Care. The Milbank Quarterly 83(4): 691-729. El-Saharty, S., Ahsan, K., Koehlmoos, T., and Engelgau, M. 2013. Tackling Noncommunicable Diseases in Bangladesh. Washington, DC: World Bank. Global Fund. 2008. Proposal Form-- Round 8. Geneva, Switzerland: Global Fund. Government of the People’s Republic of Bangladesh. 2004a. HNP Strategic Investment Plan. Dhaka, Bangladesh: Ministry of Health and Family Welfare. ______. 2004b. The Public Procurement Regulations, 2003. Dhaka, Bangladesh: Ministry of Planning. ANNEX A 52 ______. 2009. National Strategy for Accelerated Poverty Reduction. Dhaka, Bangladesh: Planning Commission. ______. 2010a. A Situation Analysis Report on Health (MDG 4, 5, and 6). Dhaka, Bangladesh: General Economic Division, Bangladesh Planning Commission. ______. 2010b. Sixth Five Year Plan, 2011-2015. Dhaka, Bangladesh: General Economics Division, Planning Commission. ______. 2011a. Population and Housing Census 2011. Dhaka, Bangladesh: Bangladesh Bureau of Statistics, Ministry of Planning. ______. 2011b. Sixth Five Year Plan. Dhaka, Bangladesh: Planning Commission. ______. 2012a. The Millenium Development Goals-- Bangladesh Progress Report 2011. Dhaka, Bangladesh: General Economics Division, Bangladesh Planning Commsion. ______. 2012b. The Millennium Development Goals-- Bangladesh Progress Report 2011. Dhaka, Bangladesh: General Economics Division, Bangladesh Planning Commsion. Harper, K., and Armelagos, G. 2010. The Changing Disease-Scape in the Third Epidemiological Transition. International Journal of Environmental Research and Public Health 7(2): 675-697. Health Economics Unit. 2010. Bangladesh National Health Accounts, 1997-2007. Dhaka, Bangladesh: Ministry of Health and Family Welfare. HERA. 2013. Progam Evaluation for Demand Side Financing Maternal Health Vouche Scheme in Bangladesh. Reet, Belgium: HERA. HNP Donor Consortium. 2006. Aide Memoire of First Annual Program Review of the Health, Nutrition, and Population Program, April 10-April 24, 2006. Dhaka, Bangladesh: HNP Donor Consortium. Hogan, M., Foreman, K., Naghavi, M., Ahn, S., Wang, M., Lopez, A., et al. 2010. Maternal Mortality for 181 Countries, 1980-2009: a systematic analysis of progress towards Millennium Development Goal 5. The Lancet 375: 1609-1623. ICDDR,B. 2012. Denegue and malaria prevalence in Bangladesh. Health and Science Bulletin 10(2): 1-6. Implemnetation Planning and Monitoring Division. 2011. End-Line Evaluation of Health, Nutrition and Population Sector Programme (HNPSP). Dhaka, Bangladesh: Ministry of Planning. Independent Evaluation Group. 2006. Project Performance Assessment Report: Bangladesh Fourth Popluation and Health Project (Credidit 2259-BD) and Health and Population Program Project (Credit 3101-BD). Washington, DC: The World Bank. ______. 2007. Annual Programme Review 2007. Dhaka, Bangladesh: HNPSP. ______. 2008a. ICR Review of the Bangladesh HIV/AIDS Prevention Project. Washington, DC: World Bank. ______. 2008b. Main Consolidated Report for Mid-Term Review. Dhaka, Bangladesh: HNPSP. ______. 2009. Annual Programme Review. Dhaka, Bangladesh: HNPSP. Independent Review Team. 2006. Annual Programme Review 2006. Dhaka, Bangladesh: HNPSP. International Monetary Fund. 2011. Bangladesh Article IV Consultation. Washington, DC: International Monetary Fund. Islam, N., Bonovas, S., and Nikolopoulos. 2013. An epidemiological overview of malaria in Bangladesh. Travel Medicine and Infectious Disease 1(1): 29-36. Kamal, S. 2012. Fertility Decline in Bangladesh. Population Association of America. San Francisco, CA. Karar, Z. A. 2009. Epidemiologic transition in rural Bangladesh, 1986-2006. Global Health Action, Available at . Karar, Z., Alam, N., and Streanfield, P. 2009. Epidemiological Transition in Rural Bangladesh, 1986-2006. Global Health Action 1904-1913. 53 Kostermans, K., and Geli, P. (n.d). The Sector-Wide Approach in Action: The Example of the Bangladesh Health, Nutrition, and Population Sector. Washington, DC: The World Bank. Lovinsohn, B., and Harding, A. 2005. Buying Results? A Review of Contracting for Health Service Delivery in Developing Coutnries. The Lancet 366(9486):676–681. Lozano, R., Naghavi, M., Foreman, L. S., Shibuya, K., Lopez, A., Murray, C., et al. 2012. Global and regional mortality from 235 causes of dealth 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet 8(9): e72788. Lozano, R., Wang, H., Foreman, K., Rajaatnam, J., Maghavi, M., Marcus, J., et al. 2011. Progress towards Milennium Development Goals 4 and 5 on maternal and child mortality: an updated systematic analysis. The Lancet 378: 1139-1165. MacDonald, L. 2013. How to Avoid Another Bangladesh Factory Disaster. Retrieved May 7, 2013, from Global Prosperity Wonkcast: http://www.cgdev.org/blog/how-avoid-another-bangladesh-factory- disaster-kimberly-elliott Martinez, J. 2008. Sector wide approaches at critical times. London, UK: HLSP Institute. Mbuya, N., and Ahsan, K. 2012. The National Nutrition Program Survey, 2010- Better, but Not Enough. Washington, DC: World Bank. Ministry of Health and Family Welfare. 2010. Annual Programme Implementation Report (APIR) 2010. Dhaka, Bangladesh: HNPSP. Mitra, S. N., Ali, M., Islam, S., Cross, A., and Saha, T. 1994. Bangaldesh Demorgraph and Health Survey, 1993-1994. Dhaka, Bangladesh: NIPORT. Mitra, S. N., Al-Sabir, A., Cross, A., and Jamil, K. 1997. Bangladesh Demograph and Health Survey, 1996-1997. Dhaka, Bangladesh and Calveton, MD: NIPORT, Mitra and Associates, and Macro International. Mushtaque, A., Chowdhury, A. B., Chowhurdy, M. E., Rasheed, S., Hussain, Z., and Chen, L. (2013). The Bangladesh Paradox: excepctional health acheivement despite economic poverty. The Lancet, 1734-1745. National Institute of Population Research and Training, Mitra and Associates, and MEASURE DHS. 2013. Bangladesh Demographic and Health Survey, 2011. Dhaka, Bangladesh: NIPORT, Mitra and Associates, and ICF International. National Oceanic and Atmostpheric Administration. (n.d.). NOAA’s Top Global Weather, Water and Climate Events of the 20th Century. Washington, DC. NIPORT, Mitra and Associates, and ORC Macro. 2005. Bangladesh Demographic and Health Survey 2004. Dhaka, Bangladesh and Calverton, MD: NIPORT, Mitra and Associates, and ORC Macro. NIPORT, Measure Evaluation, and ICDDR,B. 2011. Bangladesh: Maternal Mortality and Health Survey, 2010. Dhaka, Bangladesh: NIPORT. NIPORT, Mitra and Associates, and Macro International. 2009. Bangladesh Demographic and Health Survey 2007. Dhaka, Bangladesh and Calverton, MD.: NIPORT, Mitra and Associates, and ORC Macro. NIPORT, Mitra and Associates, and Measure DHS. 2012. Bangladesh Demographic and Health Survey 2011: Preliminary Report. Dhaka, Bangladesh and Calverton, MD: NIPORT, Mitra and Associates, and Measure DHS. NIPORT, Mitra and Associates, and ORC Macro. 2001. Bangladesh Demographic and Health Survey, 1999-2000. Dhaka, Bangladesh and Calverton, MD: NIPORT, Mitra and Associates, and ORC Macro. OPCS. 2009. Investment Lending Reform: Concept Note. Washington, DC: World Bank. ______. 2010. Implementation Completion and Results Report: Guidelines. Washington, DC: World Bank. ANNEX A 54 ______. 2011a. A New Instrument to Advance Development Effectiveness: Program for Results. Washington, DC: World Bank. ______. 2011b. Implementation Completion and Results Report: Guidelines. Washington, DC: World Bank. Planning Wing. 2007. Annual Programme Implementation Report 2007. Dhaka, Bangladesh: Ministry of Health and Family Welfare. Rahman, S. A., Kielmann, T., McPake, B., and Normand, C. 2012. Healthcare-seeking Behaviour among the Tribal People of Bangladesh: Can the Current Health System Really Meet Their Needs? Journal of Health, Population, and Nutrition, 353-365. Rahman, S. A., Kielmann, T., McPake, B., and Normand, C. 2012. Healthcare-Seeking Behaviour among the Tribal People of Bangladesh: Can the Current Health System Really Meet Their Needs? Journal of Helath, Population, and Nutrition 30(3): 353-365. Reite, T., Mahon, J., and Reza. 2011. Options for Aid Modalities for the Health Sector Pool Funding Mechanism. Oslo, Norway: Santeam. Romo, V. 2003. Decomposition Methods in Demography. Amerstadm: Rozenberg Publishers. Salomon, J., Wang, H., Freeman, M., Vos, T., Flaxman, A., Lopez, A., et al. 2012. Healthy life expectancy for 187 countries, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet 380: 2144-2162. Save the Children. 2003. Thin on the Ground. London, UK: Save the Children UK. Saxena, S., and Salze-Lozac'h, V. 2010. Competitiveness in the Garment and Textiles Industry: Creating a supportive environment. New York, NY: The Asia Foundation. Sayem, A. M., Nury, A. T., and Hossain, M. D. 2011. Achieving the Millenium Development Goal for Under-five Mortality in Bangladesh: Current Status and Lessons for Issues and Challenges for Further Improvements. Journal of Health, Population, and Nutrition 29(2): 92-102. Schmidt, J., Ensor, T., Hossain, A., and Khan, S. 2010. Vouchers as demand side financing instruments for health care. Health Policy 96(2): 98-107. UNICEF. 2010. Child Survival in Bangladesh. Dhaka, Bangladesh: UNICEF. United Nations. 2003. Indicators for Monitoring the Millennium Development Goals. New York: United Nations. Vaillancourt, D. 2009. Do Health Sector-Wide Approaches Acheive Results? Washington, DC: World Bank. _____. 2012. In Sweet Harmony? A Review of Health and Education Sectorwide Approaches (SWAps) in the South Pacific. Washington, DC: World Bank. Wahed, T., Rasheed, S., and Bhuiya, A. 2008. Doctoring the Village Doctors. Dhaka, Bangladesh: ICDDR,B. Wang, H., Dwyer-Lindgren, L., Lofgren, K., Rajartnam, J. K., Marcus, J., Levin-Rector, A., et al. 2012. Age-specific and sex-specific mortality in 187 countries, 1970-2010: a systematic analysis for the Global Burden of Disease Study 2010. The Lancet 380(9859):2071-94. Wassner, B. 2012. In an Unlikely Corner of Asia, Strong Promise of Growth. New York Times. Wescott, C., and Breeding, M. 2011. Bangladesh: World Bank Engagement on Governance and Anticorruption. Washington, DC: Independent Evaluation Group. World Bank. 1997. Health, Nutrition, and Population Sector Strategy Paper. Washington, DC: World Bank. ______. 1998. Project Appraisal Document on a Proposed Credit in the Amount of SDR 185.5 Million (US$250 Million Equivalent) to the Government of the People's Republic of Bangladesh for a Health and Population Program Project. Washington, DC: Report 17684-BD. 55 ______. 2000. Country Assistance Strategy for Bangladesh. Washington, DC: World Bank. ______. 2005a. Implementation Completion Report on a Credit in the Amount of SDR 185.5 million (US$ 250 million equivalent) to the Government of the People's Republic of Banagladesh for a Health and Population Program Project. Washington, DC: World Bank. ______. 2005b. Project Appraisal Document on a Proposed Credit in the Amount of SDR 196.1 Million (US$300 Million Equivalent) to the People's Republic of Bangladesh for a Health, Nutrition, and Population Sector Program. Washington, DC: World Bank. HEALTH NUTRITION AND POPULATION SECTOR PROGRAM, ______. 2006. Bangladesh Country Assistance Strategy, 2006-2009. Dhaka, Bangladesh: World Bank. ______. 2007a. Healthy Development. Washington, DC: World Bank. ______. 2007b. Implementation Completion and Results Report (IDA-33560 NETH-27743) on a credit in the amount of SDR 68.50 million (US$92.0 million equivalent) to the People's Republic of Bangladesh for a National Nutrition Project. Washiington, DC: World Bank. ______. 2008a. Implementation Completion and Results Report on a Credit in the Amount of SDR 30.8 million (US$40 million equivalent) to the People's Republic of Bangladesh for a HIV/AIDS Prevention Project. Washington, DC: World Bank. ______. 2008b. Poverty Assessmnet for Bangladesh: Creating Opportunities and Bridging the East-Wet Divide. Washington, DC: The World Bank ______. 2009. Reducing Maternal Mortality: Strengthening the World Bank Response. Washington, DC: HNP Network, World Bank. ______. 2010a. Bangladesh - Public expenditure and institutional review . Washington, DC: World Bank. ______. 2010b. Bangladesh Country Assistance Program, 2011-2014. Dhaka, Bangladesh: World Bank. ______. 2010c. Bangladesh Health Sector Profile. Washington, DC: World Bank. ______. 2010d. Bangladesh Public Expenditure and Institutional Review . Washington, DC: World Bank. _____. 2011e. Implementation Completion and Results Report: Guidelines. Washington, DC. . ______. 2011f. Project Appraisal Document on a Proposed Credit in the Amount of SDR 226.40 (US$ 338.90 million equivalent) to the People's Republic of Bangladesh for a Health Sector Development Program. Washington, DC: World Bank. ______. 2012a. Implementation Completion and Results Report on a Credit in the Amount of SDR 196.1 Million and Grant in the Amount of 387.98 million to the Government of the People's Republic of Bangladesh for a Health, Nutrition, and Population Sector Program. Washington, DC: World Bank. ______. 2012b. World Development Indicators. data.worldbank.org. ______. 2013. Gini Index. Retrieved April 30, 2013, from Data: htppL://data.worldbank.org/indicator/SI.POV.GINI World Health Organization. 2006. A guide to WHO's Role in Sector Wide Approaches to Health Development. Geneva, Switzerland: World Health Organization. _____. 2008. World Malaria Report, 2008. Geneva, Switzerland : WHO Press. World Health Organization. 2012. World Malaria Report, 2012. Geneva, Switzerland: WHO Press. ______. 2013a. TB Data. Retrieved October 30, 2013, from http://www.who.int/tb/country/data/download/en/index.html ______. 2013b. Gini Index. Retrieved April 30, 2013, from Data: http://data.worldbank.org/indicator/SI.POV.GINI ______. (n.d.). Bangladesh Child Health Profile. Retrieved June 21, 2013, from Child Health Profile: http://www.who.int/maternal_child_adolescent/data/media/cah_chp_bangladesh.pdf ANNEX A 56 Yardley, J. 2012. Export Powerhouse Feels Pangs of Labor Strife. New York Times. Zhang, X., Rashid, S., Ahmad, K., Mueller, V., Lee, H., Lemma, S., et al. 2013. Rising Wages in Bangladesh. Washington, DC: IFPRI. 57 Annex A. Basic Data Sheet HEALTH, NUTRITION, AND POPULATION SECTOR PROGRAM (P074841) IDA-40520 MULT-56510 Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 760 683.04 90.00 Loan amount 300.00 288.33 96.11 Cancellation 0.00 10.48 0.00 Cumulative Estimated and Actual Disbursements FY06 FY07 FY08 FY09 FY10 FY11 FY12 FY13 Appraisal estimate (US$M) 35.00 90.00 150.00 210.00 270.00 300.00 300.00 300.00 Actual (US$M) 54.56 127.23 124.70 144.58 235.95 292.43 295.23 288.33 Actual as % of appraisal 155.88 141.36 83.13 68.84 87.38 97.47 98.41 96.11 Date of final disbursement: 05/03/2012 Project Dates Original Actual Initiating memorandum 07/10/2001 02/11/2004 Negotiations 12/12/2002 02/14/2005 Board approval 03/18/2003 04/28/2005 Signing 05/17/2005 05/17/2005 Effectiveness 06/14/2005 06/14/2005 Closing date 12/31/2010 12/31/2011 Task Team Members Responsibility/ Names Title Unit Specialty Supervision/ICR Md. Mahtab Alam Program Assistant SACBD Administrative support Anisuzzaman Chowdhury Consultant SASHD Operations Shahadat Hossain Program Assistant SACBD Administrative Chowdhury support Rafael A. Cortez Sr. Economist (Health) LCSHH Health Economics ANNEX A 58 Responsibility/ Names Title Unit Specialty Agnes Couffinhal Economist SASHD Health Economics Md. Raziq Hossain Operations Officer SASHD Operations Farzana Ishrat Nutrition Specialist SASHD Nutrition Preeti Kudesia Senior Public Health Specialist SASHD Public Health Iffat Mahmud Operations Analyst SASHD Operations Dinesh M. Nair Sr. Health Specialist SASHD Team Lead Sandra Rosenhouse Sr. Population and Health SASHD Public Health Specialist. Harvinder Singh Suri Consultant SARPS Procurement Yolanda Tayler Lead Procurement Specialist SARPS Procurement Bina Valaydon Health Specialist AFTH1 Public Health Elfreda Vincent Program Assistant SASHD Administrative support Alejandro Welch Information Assistant SASHD Administrative support Suraiya Zannath Sr. Financial Management SARFM Financial Specialist Management Kees Kostermans Lead Public Health Specialist SASHN Team Lead Albertus Voetberg Lead Health Specialist SASHN Public Health Sameh El-Saharty Senior Health Policy Specialist SASHN Team Lead Bushra Binte Alam Senior Health Specialist SASHN Team Lead Nkosinathi Mbuya Senior Nutrition Specialist SASHN Nutrition Aparnaa Somanathan Senior Economist EASHH Health Economics Ramesh Govindaraj Senior Health Specialist SASHN Pharmaceuticals Tania Dmytraczenko Senior Economist LCSHH Health Economics Michael Engelgau Senior Public Health Specialist SASHN Public health Ruma Tavorath Senior Environmental Specialist SASDI Environment Teen Kari Barua Consultant SASHN Social safeguards Marghoob bin Hussein Senior Procurement Specialist SARPS Procurement Karar Zunaid Ahsan Research Analyst SASHN MandE Hasib Ehsan Chowdhury Program Assistant SARFM Operations Nasreen Begum Team Assistant SASHN Administrative support Mariam Claeson Program Coordinator SASHN HIV/AIDS Tatyana Klimova Senior Resource Management SARRM Trust Funds Officer Maria Gracheva Senior Operations Officer SASHN Operations 59 ANNEX A Responsibility/ Names Title Unit Specialty Nilufar Ahmad Senior Gender Specialist SDN Social issues Shirin Jahangir Consultant Social issues Tahmina Begum Consultant Heath Economics Other Project Data Borrower/Executing Agency: Follow-on Operations Operation Credit no. Amount Board date (US$ million) Health Sector Development Program 49540 358.6 May 26, 2011 60 Annex B. World Bank Investment in the Health Sector Total Amount Final Name of (current US$ million) No. of co- Years Outcome Key Activities Operation financiers IDA Partners Govt. Rating  Technical assistance  Construction of facilities Population 1975 to 15.0 25.4 5.3 6 U  Training and stipends Project 1982*  Communication pilots  Private sector participation  Field service support Second  Training Population 1979 to 30.8 49.2 18.7 8 S  Communication program and Family 1985* Health Project  Evaluation and research  Private sector participation  Family planning Third  Health services Population 1986 to  Communication program and Family 100.9 110.8 34.6 6 HS Welfare 1992*  Income generation Project  Evaluation and research  Private sector participation  Family planning services Fourth  Maternal and child health Population 1991 to  Public health programs 188.4 453.1 303.2 11 S and Health 1998*  Training Project  Communication programs  Institution building  Institutional building Integrated 1995 to  Education and communication Nutrition 58.6 0.0 7.1 0 MU Project 2002*  Local nutrition services  Multi-sectoral programs  Essential health services Health and  Hospital initiative 1998 to Population 238 152.3- 2,445 7 U  Communication 2005* Program  Health information system  Sector restructuring and reform  Local nutrition services National 2000 to  Supplementation and fortification Nutrition 70.3 12.12- 6.8- 2 MU 2006*  Nutrition communication Project  Monitoring and evaluation  High-risk group intervention HIV/AIDS 2000 to  Communication and advocacy Prevention 19.7- 7.2- 0.0- 1 U Project 2007*  Blood safety  Monitoring and evaluation Health, Nutrition, and 2005 to MS  Population 2011* Program Health Sector 2011 to Development n/a  2016 Program NOTE: Years refer to the actual years that the operation was active, including extensions; * means the project was extended at least once. Total Amount from Project Completion Report or Implementation Completion Report indicated, when available; + means that the actual amount was 110 percent or more than original planned, - means that the actual amount was 90 percent or less than original planned. Key Activities refer to activities included in the design (in the Staff Appraisal Report or Project Appraisal Document) and not necessarily the project’s components. Project administration is not included. 61 Annex C. Key Performance Indicators for the Health, Nutrition, and Population Sector Program Table C.1. Key Performance Indicators, World Bank Results Framework Baseline Target Final Value Alternative Indicator Remarks Value Value (project end) Final Value Indicators at the PDO Level Proportion of Ministry expenditures at 45% 50% 47% This indicator is modified replaced the original indicator, which attempted to measure upazila level and below spending in the poorest 25 percent of districts. It was not possible to track the original variable. Revision made during Midterm Review (2009). Proportion of attended deliveries in the 4.1% 10.0% 11.8% These two indicators quantify the original indicator that focused on the increase in the lowest two income quintiles coverage of the essential service package. Revision made during Midterm Review (2009). Proportion receiving antenatal care in 32.5% 40.0% 40.3% the lowest two income quintiles Indicators supporting Component One (Accelerating achievement of Millennium Development Goals Outcomes and Population Policy) Proportion of attended births 15.5% 28.0% 26.5% Original baseline was from 25 percent to 15.5 percent during the Midterm Review (2009). Tuberculosis case detection rate 46% 72% 74% Original baseline was raised from 41 percent to 46 percent during the Midterm Review (2009). Proportion of children (9-59 months 81.8% 90% 92% Definition of age of children was changed from 12-59 months to 9-59 months old) receiving Vitamin A supplementation Proportion of children (0-12 months 68.4% 85% 80% Indicators added during the Midterm Review (2009) old) fully immunized Tuberculosis cure rate 85% 85% 92% Indicators added during the Midterm Review (2009) Indicators supporting Component Two (Meeting emerging sector challenges) Non-Communicable Disease Strategy No Strategy Strategy Strategy was developed and implemented, with actions incorporated in the operational developed and implemented Strategy plan. Indicators supporting Component Three (Advancing sector modernization) Proportion of contracts awarded within n.a. 90% 80% No baseline is available as the indicator is associated with the program. initial bid validity period Proportion of women receiving 7% 60% 64% Indicator changed during Midterm Review (2009) from introduction of the maternal maternal voucher that have attended voucher to a measure of its effectiveness. birth Percentage of districts with disease 52% 95% 95% Indicators added during the Midterm Review (2009) surveillance reports Note: Baseline and final values are taken from the final revision. ANNEX C 62 Table C.2. Key Performance Indicators, Government Program Baseline Target Final Value Most Recent Indicator Value Value (project end) Value Remarks Outcome/Impact Indicators Infant mortality (per 1,000 births) 56 37 52 43 Child mortality (per 1,000 births) 80 52 65 53 Maternal mortality (per 100,000 births) 295 240 194 194 Children (under 5) underweight 48% 34% 41% 36% There are some inconsistencies between the baseline and survey data. The children stunted variable does not appear to be monitored, however it is Children (12 to 60 months) severely stunted 27% 20% n.d. 15% still possible to follow trend of children under weight. Total fertility rate 3.7 2.2 2.7 2.3 Indicators supporting Component One (Accelerating achievement of Millennium Development Goals Outcomes and Population Policy) Proportion attended births 12% 43% 27% Proportion children immunized (DP3) 72% 95% Proportion children immunized (measles) 76% >80% Proportion children receiving Vitamin A 89% >90% 83% Percentage women with long-lasting birth 13% Higher 7% Final evaluation reports that the baseline should have been around 7%. control Tuberculosis case detection rate 41% 70% 72% Proportion children under five using bed net <15% Higher n.a. Higher Increased from 5% of households to 55% of households. (in areas at risk) HIV/AIDS prevalence in general population <1% <1% 0.1% Several different HIV/AIDS indicators were proposed in different documents. Indicators supporting Component Two (Meeting emerging sector challenges) Percentage adults using tobacco n.a. Lower 22% Originally, there was no value for the baseline. It was set later set at 20%, with a target of 15%. Percentage women receiving counseling n.a. Increase after injury Percentage blood screened for transmission n.a. 65% increase Indicators supporting Component Three (Advancing sector modernization) Percentage districts with disease surveillance 2% 80% reports Share of government allocated to Ministry 5% 10% Note: The indicators are based on the World Bank PAD (World Bank 2005). It excludes all qualitative indicators and several quantitative indicators. The baseline values are from 2003 and are taken from the World Bank PAD. They do not necessarily match the World Bank’s results framework. The final values are taken from the Final Evaluation (Implemnetation Planning and Monitoring Division 2011). Most recent values are taken from the best available sources. 63 Annex D. List of Persons Met Anthony Drexler Consultant Dr. Kaosar Afsana Director, Health BRAC Rashedul Haider Choudhury Program Manager, Health BRAC Naima Sultana Monika Asst. Manager, Visitors Communication BRAC M. Musharraf Hossain Bhuiyan Cabinet Secretary and former Cabinet Division Economics Relations Secretary Peggy Thorpe First Secretary (Development) CIDA Dr. Momena Khatun Health Advisor CIDA M.M. Reza Consultant, Ministry of Planning and Consultant Former Health Secretary Naveed Ahmed Chowdhury Social Development Adviser DFID Nadia Farheen Technical Director FHI 360 Dr. Anita Davies Chief Migration Health Officer IOM Timothy Evans Dean James Grant School of Public Health, BRAC University Bertold Liche Senior Program Manager, Health KFW Habibur Rahma Senior Program Manager KFW Dr. S.A.J. Md. Musa Director, Primary Health Care and Line Ministry of Health and Director, Maternal, Child, and Family Planning Adolescent Health Dr. Barendra Nath Mandal Additional Project Director, Community MOFHW Clinics Dr. Makhduma Nargis Additional Secretary and Project MOHFW Director, Community Clinics Ahasan H. Mansur Executive Director Policy Research Institute Simon Rasin Director Save the Children Leo Kenny UNAIDS Dr. Muhammod Abbus Sabur UNDP Representative UNFPA Yuki Suehiro Deputy Representative UNFPA Tauhid Alam Sr. Program Officer UNFPA Gregory Adams Acting Office Director USAID Sharmina Sultana Project Management Specialist USAID Thibaut Williams Health and Population Officer USAID Dr. Md. Kamar Rezwan National Professional Officer (TB WHO Control) Anabela Abreu World Bank Rafael Cortez World Bank Tania Dmytraczenko World Bank Albertus Voetberg World Bank Cornelis Kostermans World Bank Sameh El-Saharty World Bank Christine Wallich World Bank Jacqueline Mahon World Bank Bhushra Alam World Bank 64 Annex E. Borrower Comments 65